Dandandan opened a new pull request, #24338:
URL: https://github.com/apache/datafusion/pull/24338
## Which issue does this PR close?
None. Found while profiling compile times for #24325 / #24326 / #24329 /
#24330.
## Rationale for this change
`datafusion/core/src/bin/` holds three binaries that regenerate the docs
under
`docs/source/user-guide`: `print_config_docs`, `print_runtime_config_docs`
and
`print_functions_docs`. Cargo auto-discovers them and they have no
`required-features`, so **every `cargo build` links all three** — each one
~174MB, since each links the whole `datafusion` rlib.
Nothing in normal development uses them. They are run by
`dev/update_config_docs.sh` and `dev/update_function_docs.sh`, and by the CI
job
that checks the committed docs are up to date.
Two places where this shows up:
**Cold builds.** The three binaries link *after* every other unit has
finished,
so they sit on the critical path with nothing to overlap with.
`cargo build --timings` shows them occupying the last **3.5s** of a
`cargo build -p datafusion` (~8.8s of CPU), after the last library unit
completes.
**The tightest inner loop** — touch a file in core, rebuild. All three are
relinked every time:
```
before: 3.0s 2.4s
after: 1.3s 1.1s
```
## What changes are included in this PR?
The three binaries move behind a new non-default `docs_generation` feature,
and
the two `dev/` scripts pass `--features docs_generation`.
Using `required-features` means declaring the `[[bin]]` targets explicitly,
since
auto-discovered targets cannot carry it.
## Are these changes tested?
- `cargo build -p datafusion` no longer produces the three binaries
- `cargo build -p datafusion --features docs_generation` does
- `./dev/update_config_docs.sh` still regenerates
`docs/source/user-guide/configs.md` byte-identically (empty `git diff`
afterwards), which is what the CI doc check compares
`dev/update_function_docs.sh` uses the same invocation pattern and all three
of
its call sites were updated; CI exercises both scripts.
## Are there any user-facing changes?
The three binaries are no longer built by a default `cargo build`. Anyone who
ran them directly needs `--features docs_generation` — same as the `dev/`
scripts
now do. No library API changes.
If you would rather these lived outside the published crate altogether,
moving
them to a small non-published `dev/` crate would have the same effect on
build
times; I went with the smaller change.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
