The GitHub Actions job "Benchmarks" on texera.git/main has succeeded. Run started by GitHub user github-merge-queue[bot] (triggered by github-merge-queue[bot]).
Head commit for run: 3242ac9e3f29d5ceac95443dde39a3aa81bfe3dd / Yicong Huang <[email protected]> test(amber): move state-mat e2e into amber-integration (#5682) ### What changes were proposed in this PR? Two things: **1. Move the state-materialization e2e tests into `amber-integration`.** The two e2e specs that previously ran in the deleted `pyamber-state-materialization-mac` diagnostic job now run under the existing `amber-integration` job, which already provisions postgres + iceberg catalog DB + MinIO + Lakekeeper and runs `pytest -m integration` as its last step. The test's catalog backend switches from sqlite `SqlCatalog` to the real postgres-backed `JdbcCatalog`, matching `test_iceberg_document.py:45`, so we exercise the prod catalog code path instead of an sqlite shim. **2. Add macOS to the `amber-integration` matrix.** We want CI coverage that the integration stack actually runs on macOS (where most dev machines live), not just Linux. GitHub-hosted macOS runners have no Docker, so each docker-dependent step now branches on `$RUNNER_OS`: macOS provisions postgres / minio / lakekeeper natively (`brew install` + the upstream lakekeeper `aarch64-apple-darwin` tarball — published from v0.11.0 onward, same version as the Linux docker tag) while Linux keeps the existing docker path. Protoc on macOS uses brew's arm64-native protobuf because protoc 3.19.4 has no arm64-mac build and running its x86_64 binary under Rosetta breaks the `python_betterproto` plugin (arch / site-packages split between Rosetta'd protoc and arm64 setup-python). For proto3 sources, the plugin output depends on `betterproto`, not protoc, so the protoc version drift on macOS is benign for codegen. | Before | After | | --- | --- | | `pyamber-state-materialization-mac` macOS diagnostic job (build.yml:742) | deleted | | `SqlCatalog` (sqlite) injected in module fixture | real postgres-backed `JdbcCatalog`, matching `test_iceberg_document.py:45` | | Test discovered by an explicit `pytest -sv <path>` from that job | `@pytest.mark.integration` + picked up by `amber-integration`'s `pytest -m integration` | | `amber-integration` runs only on `ubuntu-22.04` | `amber-integration` runs on `[ubuntu-22.04, macos-latest]` with `$RUNNER_OS`-branched service provisioning | `StorageConfig.initialize` is wrapped in a class-scoped autouse fixture (rather than called at module import time) so it co-exists with `test_iceberg_document.py`'s import-time `initialize` regardless of pytest collection order. All catalog + S3 credentials read the same `STORAGE_*` env vars the production code consumes (via storage.conf), with defaults that match storage.conf — so the test stays aligned with whichever identity the surrounding CI infra uses. The unit-style `test_process_start_channel_persists_produce_state_on_start_output` that the deleted mac job also ran is untouched: it monkeypatches the output manager and is already picked up by the regular `pyamber` job's `pytest -m "not integration"` step. ### Any related issues, documentation, discussions? Closes #5681. ### How was this PR tested? Locally against the existing texera-dev infra (postgres on 5432 with `texera_iceberg_catalog` schema initialized via `sql/iceberg_postgres_catalog.sql`): ``` cd amber && pytest -m integration --junit-xml=/tmp/junit-integration.xml -sv # 3 passed, 502 deselected -- test_state_written_by_output_manager_is_replayed_by_reader, # test_state_table_persists_across_writer_close, # test_rest_catalog_round_trip ``` And the regular pyamber suite still green: ``` cd amber && pytest -m "not integration" -q # 502 passed, 3 deselected ``` In CI, the `amber-integration` job picks these tests up automatically because they're marked `@pytest.mark.integration`, on both `ubuntu-22.04` and `macos-latest`. ### Was this PR authored or co-authored using generative AI tooling? Generated-by: Claude Code (Claude Opus 4.7) --------- Signed-off-by: Yicong Huang <[email protected]> Co-authored-by: Claude Opus 4.7 (1M context) <[email protected]> Report URL: https://github.com/apache/texera/actions/runs/27522246716 With regards, GitHub Actions via GitBox
