Due to a change in tracing-subscriber where it escapes ANSI codes, the
output from rustdoc was changed to be escaped instead of colored. This
fixes the issue by just printing to the console directly.
This does not have a test because it is difficult to emulate a terminal
in the testsuite, and I don't feel like adding color overrides at the
moment.
Ref:
- https://github.com/rust-lang/mdBook/pull/2829 -- Switched to tracing
- https://github.com/advisories/GHSA-xwfj-jgwm-7wp5 -- CVE-2025-58160
- https://github.com/tokio-rs/tracing/issues/3378 -- Discussion of
supporting color codes.
Eventually mdbook's primary output should migrate away from using log
messages, and instead use a structured console system. However, that's a
ways off.
This fixes a problem where custom preprocessors were not being
registered when running tests. This was caused by the test function
rebuilding the preprocessor map.
This removes the code that was rebuilding the preprocessors and removing
the IndexPreprocessor when running tests. Skipping IndexPreprocessor was
added back in https://github.com/rust-lang/mdBook/pull/741 to fix
https://github.com/rust-lang/mdBook/issues/724 which was caused by
https://github.com/rust-lang/mdBook/pull/685 which added the
IndexPreprocessor.
Additionally, https://github.com/rust-lang/mdBook/pull/1986 added
running *all* preprocessors.
The IndexPreprocessor was removed because in the past the code was
testing against the source directly, and the path from `chapter.path` is
the converted `index.md` file, and that filename does not exist in the
source. This isn't a problem anymore because due to
https://github.com/rust-lang/mdBook/pull/891 it is not reading from the
`src` directory.
Note that this results in a minor change where the chapter path changes
from `README.md` to `index.md` in the output and the `--chapter` option.
I think I'm ok with that change, though it would be easy to switch it
back if that's an issue.
This adds several changes to how environment variables are handled to
more closely align with how configs are handled, and to fix an issue
with replacing entire tables. The changes are:
- Top-level tables like `MDBOOK_BOOK` now *replace* the contents of the
`book` table instead of merging it. This adds consistency with how all
the other environment objects work.
- Fixed allowing top-level replacement of `MDBOOK_BOOK` and
`MDBOOK_OUTPUT`. This was inadvertently recently broken.
- Added ability to replace top-level `MDBOOK_RUST`. I don't recall why
that wasn't included.
- Reject invalid keys like `MDBOOK_FOO`.
- Reject unknown keys, like `MDBOOK_BOOK='{"xyz": 123}'`
- Reject invalid types, like `MDBOOK_BOOK='{"title": 123}'`
These docs were slightly drifting from the user guide docs. Instead of
trying to maintain multiple copies of this, I have changed it so that
it just links out to the guide.
(The guide docs could be cleaned up a little, but that's a separate
issue.)
This moves the code for copying the theme to the theme directory to the
Theme type so that the code lives closer to the data definition. This
also then reduces the public API surface of the Theme to give a little
more flexibility for updating it in the future.
This does a little cleanup around the usage of filesystem functions:
- Add `mdbook_core::utils::fs::read_to_string` as a wrapper around
`std::fs::read_to_string` to provide better error messages. Use
this wherever a file is read.
- Add `mdbook_core::utils::fs::create_dir_all` as a wrapper around
`std::fs::create_dir_all` to provide better error messages. Use
this wherever a file is read.
- Replace `mdbook_core::utils::fs::write_file` with `write` to mirror
the `std::fs::write` API.
- Remove `mdbook_core::utils::fs::create_file`. It was generally not
used anymore.
- Scrub the usage of `std::fs` to use the new wrappers. This doesn't
remove it 100%, but it is now significantly reduced.
These functions are only used by the links preprocessor. I'm moving
these functions to put them closer to the code that they are associated
with, and to reduce the public API surface.
This rewrites the HTML rendering pipeline to use a tree data structure,
and implements a custom HTML serializer. The intent is to make it easier
to make changes and to manipulate the output. This should make some
future changes much easier.
This is a large change, but I'll try to briefly summarize what's
changing:
- All of the HTML rendering support has been moved out of
mdbook-markdown into mdbook-html. For now, all of the API surface is
private, though we may consider ways to safely expose it in the
future.
- Instead of using pulldown-cmark's html serializer, this takes the
pulldown-cmark events and translates them into a tree data structure
(using the ego-tree crate to define the tree). See `tree.rs`.
- HTML in the markdown document is parsed using html5ever, and then
lives inside the same tree data structure. See `tokenizer.rs`.
- Transformations are then applied to the tree data structure. For
example, adding header links or hiding code lines.
- Serialization is a simple process of writing out the nodes to a
string. See `serialize.rs`.
- The search indexer works on the tree structure instead of re-rendering
every chapter twice. See `html_handlebars/search.rs`.
- The print page now takes a very different approach of taking the
same tree structure built for rendering the chapters, and applies
transformations to it. This avoid re-parsing everything again. See
`print.rs`.
- I changed the linking behavior so that links on the print page
link to items on the print page instead of outside the print page.
- There are a variety of small changes to how it serializes as can be
seen in the changes to the tests. Some highlights:
- Code blocks no longer have a second layer of `<pre>` tags wrapping
it.
- Fixed a minor issue where a rust code block with a specific
edition was having the wrong classes when there was a default
edition.
- Drops the ammonia dependency, which significantly reduces the number
of dependencies. It was only being used for a very minor task, and
we can handle it much more easily now.
- Drops `pretty_assertions`, they are no longer used (mostly being
migrated to the testsuite).
There's obviously a lot of risk trying to parse everything to such a low
level, but I think the benefits are worth it. Also, the API isn't super
ergonomic compared to say javascript (there are no selectors), but it
works well enough so far.
I have not run this through rigorous benchmarking, but it does have a
very noticeable performance improvement, especially in a debug build.
I expect in the future that we'll want to expose some kind of
integration with extensions so they have access to this tree structure
(or some kind of tree structure).
Closes https://github.com/rust-lang/mdBook/issues/1736
This adds the `Book::chapters` iterator (and `for_each_chapter_mut`) to
iterate over non-draft chapters. This is a common pattern I keep
encountering, and I figure it might simplify things. It runs a little
risk that callers may not be properly handling every item type, but I
think it should be ok.
This switches to using the tracing crate instead of log. Tracing
provides a lot of nice features which we can take advantage of moving
forward.
This also adjusts the output fairly significantly. This includes:
- Switched the environment variable from RUST_LOG to MDBOOK_LOG.
- Dropped the timestamp. I experimented with various different time
displays, but ultimately decided to omit it for now. I don't think
I've ever found it to be useful, and it takes up a very significant
amount of space. It could potentially be useful for basic profiling,
but I think there are other, better mechanisms for that. We could
consider leveraging tracing itself for doing some basic profiling
(like using something like tracing-chrome).
- Dropped the target unless MDBOOK_LOG is set. The target tends to be
pretty noisy, and doesn't really convey much information unless you
are debugging or otherwise trying to adjust the log output.
- Added color.
- Slightly reworked the way the error cause trace is displayed.
- Slightly changed the way html5ever filtering is done, as well as add
handlebars to the list since they both are very noisy. You can
override this now by explicitly listing them as targets.
I still expect that mdbook will eventually change how it displays things
to the console, possibly switching away from tracing and printing things
itself. However, that is a larger project for the future.
This renames the "sections" list to "items". In practice, this list has
contained more than just "sections" since parts were added. Also, the
rest of the code consistently uses the term "items", since the values it
contains are called `BookItem`s. Finally, the naming has always been a
little confusing to me.
This is a very disruptive change, and I'm not doing it lightly. However,
since there are a number of other API changes going into 0.5, I think
now is an ok time to change this.
This changes `with_renderer` and `with_preprocessor` to replace any
extensions of the same name instead of just appending to the list. This
is necessary for rust-lang's build process, because we replace the
preprocessors with local ones. Previously, mdbook would just print an
error, but continue working. With the change that preprocessors are no
longer optional by default, it is now required that we have a way to
replace the existing entries.
This adds the `optional` field to the preprocessor configuration to
mirror the same option for the `output` table. Missing preprocessors are
now an error unless the `optional` field is set. This should help with
inadvertently building a book when a missing preprocessor that you
expect to be installed.
This changes preprocessors so that:
- Relative paths in the `command` value are relative to the book root.
- The process current directory is the book root.
This makes it so that it isn't dependent on the directory where `mdbook`
is executed.
Fixes https://github.com/rust-lang/mdBook/issues/1424
This removes the deprecated support for renderer paths that are relative
to the destination. Relative renderer command paths now must always be
relative to the book root.
This changes it so that it is an error if there is ever an unknown
configuration field. This is intended to help avoid things like typos,
or using an outdated version of mdbook. Although it is possible that new
fields could potentially safely be ignored, setting up a warning system
is a bit more of a hassle. I don't think mdbook needs to have the same
kind of multi-version support as something like cargo does. However, if
this ends up being too much of a pain point, we can try to add a warning
system instead.
There are a variety of changes here:
- The top-level config namespace is now closed so that it only accepts
the keys defined in `Config`.
- All config tables now reject unknown fields.
- Added `Config::outputs` and `Config::preprocessors` for convenience
to access the entire `output` and `preprocessor` tables.
- Moved the unit-tests that were setting environment variables to the
testsuite where it launches a process instead.
Closes https://github.com/rust-lang/mdBook/issues/1595
This switches all public types to use non_exhaustive to make it easier
to make additions without a semver-breaking change.
Some of the ergonomics are hampered due to the lack of exhaustiveness
checking. Hopefully some day in the future,
non_exhaustive_omitted_patterns_lint or something like it will get
stabilized.
Closes https://github.com/rust-lang/mdBook/issues/1835
This removes the `pub` status of the SectionNumber field. The intent is
to make this potentially extensible in the future if we decide to add
more fields, or change its internal representation. With the existence
of the deref impls, generally this change shouldn't be visible except
for the constructor, which hopefully shouldn't be too cumbersome to use
`SectionNumber::new` instead.
This removes toml as a public dependency. This reduces the exposure of
the public API, reduces exposure of internal implementation, and makes
it easier to make semver-incompatible changes to toml.
This is accomplished through a variety of changes:
- `get` and `get_mut` are removed.
- `get_deserialized_opt` is renamed to `get`.
- Dropped the AsRef for `get_deserialized_opt` for ergonomics, since
using an `&` for a String is not too much to ask, and the other
generic arg needs to be specified in a fair number of situations.
- Removed deprecated `get_deserialized`.
- Dropped `TomlExt` from the public API.
- Removed `get_renderer` and `get_preprocessor` since they were trivial
wrappers over `get`.
