This is an automated email from the ASF dual-hosted git repository. github-merge-queue[bot] pushed a commit to branch gh-readonly-queue/main/pr-3562-2135b37730c55ba98fecaae8747bffb7105c0ece in repository https://gitbox.apache.org/repos/asf/iceberg-rust.git
commit 17b303c0cad70e8249f6284a973b48d51cc3c0d3 Author: Kevin Liu <[email protected]> AuthorDate: Wed Oct 28 09:59:01 2026 +0000 ci(python): fail wheel license generation when about.toml needs an update (#3562) * chore(licensing): drop redundant zstd-sys clarification With cargo-about 0.9.2, which CI uses, the generated wheel LICENSE is byte-identical without this clarification: 0.9.2 keeps zstd-sys's detected license files, including the Meta notices. The clarification was only needed with 0.9.1, and its checksums go stale on every zstd-sys bump. Co-authored-by: Copilot App <[email protected]> * ci(python): fail wheel license generation on stale clarifications `cargo about generate --fail` exits 0 when a clarification's checksum no longer matches; it only logs a WARN and falls back to the license template, so the wheel LICENSE gets `<year>` placeholders instead of the crate's license text. CI passed, and this was only caught by the manual check before a release. Fail the action on a WARN or a placeholder, so a dependency bump that needs an about.toml update fails in its own pull request. Co-authored-by: Copilot App <[email protected]> * ci(python): generate wheel licenses once per pull request The pull request test job regenerated LICENSE and NOTICE on every OS, taking 40-70s each, but the output doesn't depend on the OS and the built wheels aren't published. Run it on Linux only; the release and nightly workflows still run it for every wheel. Co-authored-by: Copilot App <[email protected]> * ci(python): fail wheel license generation on any cargo-about stderr Matching `WARN` missed cargo-about's other non-fatal problems: `[ERROR]` logs such as an unreadable license file, and the codespan `warning:` emitted for a clarification whose crate is no longer a dependency. Fail when cargo-about writes anything to stderr instead. Run the cargo-about binary rather than `cargo about`, since the rustup proxy writes to stderr when it installs the pinned toolchain, as it does on the Ubuntu PR job. cargo-about's own `cargo metadata` call captures its stderr. Match only `<year>`: every accepted license template with placeholders contains it. Co-authored-by: Copilot App <[email protected]> * ci(python): set up the toolchain on Linux and call `cargo about` `cargo-about` and `cargo about` read too alike to tell apart in review. The binary was only called directly because the Linux PR job skipped setup-builder, so rustup installed the toolchain inside the cargo about step and wrote to stderr. Run setup-builder on Linux too, which moves that install out of the step. Release and nightly already set up the toolchain first, and the generate step's stderr is empty on all of their runners. Co-authored-by: Copilot App <[email protected]> * ci(python): write cargo about output with -o `2>&1 > file | tee` is easy to misread. With `-o`, cargo about writes the output file itself, so its stderr is piped to tee as usual. The file loses the trailing newline that stdout adds, so the wheel LICENSE has one blank line fewer before the entries from third-party-licenses.txt. Co-authored-by: Copilot App <[email protected]> * ci(python): shorten generate-python-license comment Co-authored-by: Copilot App <[email protected]> * ci(python): install the toolchain in generate-python-license Revert the bindings_python_ci.yml changes, so PR CI runs the action on every OS again and still skips setup-builder on Linux. Instead, the action installs the active toolchain before cargo about, so rustup's install output stays out of the stderr check for any caller. Co-authored-by: Copilot App <[email protected]> * ci(python): set up the toolchain in generate-python-license Use setup-builder before installing cargo-about, like the workflows do, instead of a bare `rustup toolchain install` in the generate step. The Linux PR job doesn't run setup-builder, so without it rustup would install the toolchain during `cargo about` and fail the stderr check. Co-authored-by: Copilot App <[email protected]> * ci(python): drop setup-builder comment in generate-python-license Co-authored-by: Copilot App <[email protected]> * docs(python): rewrite LICENSING.md release checks Say what CI already checks and how to reproduce a failure, then list the hand-maintained entries the release manager checks, and how to check each. Co-authored-by: Copilot App <[email protected]> * ci(python): run setup-builder on the Linux PR job Set up the toolchain in the one job that calls generate-python-license without it, instead of in the action. The other jobs already do, so they no longer run setup-builder twice. Co-authored-by: Copilot App <[email protected]> --------- Co-authored-by: Copilot App <[email protected]> --- .github/actions/generate-python-license/action.yml | 7 ++++++- .github/workflows/bindings_python_ci.yml | 3 +-- bindings/python/LICENSING.md | 14 ++++++++------ bindings/python/about.toml | 14 -------------- 4 files changed, 15 insertions(+), 23 deletions(-) diff --git a/.github/actions/generate-python-license/action.yml b/.github/actions/generate-python-license/action.yml index 112b4243d..ac79233ac 100644 --- a/.github/actions/generate-python-license/action.yml +++ b/.github/actions/generate-python-license/action.yml @@ -30,7 +30,12 @@ runs: shell: bash working-directory: bindings/python run: | - cargo about generate -c about.toml --locked --fail --manifest-path Cargo.toml about.hbs > cargo-about-output.txt + # --fail misses warnings and license template fallbacks. + cargo about generate -c about.toml --locked --fail --manifest-path Cargo.toml -o cargo-about-output.txt about.hbs 2>&1 | tee "$RUNNER_TEMP/cargo-about.log" + if [[ -s "$RUNNER_TEMP/cargo-about.log" ]] || grep -n '<year>' cargo-about-output.txt; then + echo "::error::bindings/python/about.toml needs an update, see bindings/python/LICENSING.md" + exit 1 + fi # Both are symlinks to the root files, remove them so the redirects don't write through. rm LICENSE NOTICE cat ../../LICENSE cargo-about-output.txt third-party-licenses.txt > LICENSE diff --git a/.github/workflows/bindings_python_ci.yml b/.github/workflows/bindings_python_ci.yml index a41d5bf12..7252a0704 100644 --- a/.github/workflows/bindings_python_ci.yml +++ b/.github/workflows/bindings_python_ci.yml @@ -83,10 +83,9 @@ jobs: - uses: actions/setup-python@5fda3b95a4ea91299a34e894583c3862153e4b97 # v7.0.0 with: python-version: 3.12 - # Linux maturin builds may run inside manylinux Docker, so host Rust caches only help Windows/macOS. - name: Setup Rust toolchain - if: runner.os != 'Linux' uses: $/.github/actions/setup-builder + # Linux maturin builds may run inside manylinux Docker, so host Rust caches only help Windows/macOS. - name: Cache Rust artifacts if: runner.os != 'Linux' uses: Swatinem/rust-cache@6323deb102c322ba6fcbdcafc7e3dddab59af2b6 # v2.9.2 diff --git a/bindings/python/LICENSING.md b/bindings/python/LICENSING.md index 685e92111..9eb48ecc0 100644 --- a/bindings/python/LICENSING.md +++ b/bindings/python/LICENSING.md @@ -60,12 +60,14 @@ Note that in the repository, `bindings/python/LICENSE` and `NOTICE` are symlinks ## Keeping entries up to date -Pull request CI runs the action, so it fails when generation fails, for example on a license that isn't in `accepted`. - -Everything else needs a manual check and update before every release: the clarifications and `targets` in `about.toml`, `third-party-licenses.txt` and `third-party-notices.txt`. They don't change with the dependencies. - -Run this from `bindings/python` to check. A `WARN` or `warning:` on stderr means a clarification no longer applies, and `<year>` in the output means no license file was matched: +Pull request CI checks `about.toml`. It fails on a license that isn't in `accepted`, on a `cargo about` warning such as a stale clarification, and on `<year>` in the output, which means no license file was found. To reproduce a failure, run this from `bindings/python`: ```shell -make -s cargo-about | grep -nE '<year>|<owner>|<copyright holders>' +make -s cargo-about | grep -n '<year>' ``` + +CI doesn't check the other entries. Before each release, from `bindings/python`, check that: + +- `targets` in `about.toml` matches the wheel targets in `release_python.yml` and `release_python_nightly.yml`. +- Each crate in `third-party-licenses.txt` is in the `make -s cargo-about` output at the version the entry names. If the version changed, re-check the entry against the new version. If the crate is gone, remove the entry. +- `third-party-notices.txt` matches the `NOTICE` files of the dependencies, found with the command in [Relevant files](#relevant-files). diff --git a/bindings/python/about.toml b/bindings/python/about.toml index c1472318f..a804d6acb 100644 --- a/bindings/python/about.toml +++ b/bindings/python/about.toml @@ -65,17 +65,3 @@ license = "MIT" [[libm.clarify.files]] checksum = "3823dda7cf046602f4b4e77ec8e227863dc4736037cc85bb33d9f19febe16bb7" path = "LICENSE.txt" - -# zstd-sys's LICENSE covers only its Rust code. Without this, cargo-about omits -# the Meta copyright notices for the bindings and the vendored zstd C code. -[zstd-sys.clarify] -license = "BSD-3-Clause" -[[zstd-sys.clarify.files]] -checksum = "ce4868c8e4ebe47a7d4b3c93dbf6b802033e5ae2fa7638098173f80836daac60" -path = "LICENSE" -[[zstd-sys.clarify.files]] -checksum = "bf7017a73a95ea63bf3716b13ec2bcd79c92942356e3408c6fb66332bad6d977" -path = "LICENSE.BSD-3-Clause" -[[zstd-sys.clarify.files]] -checksum = "7055266497633c9025b777c78eb7235af13922117480ed5c674677adc381c9d8" -path = "zstd/LICENSE"
