david-streamlio commented on PR #33:
URL: https://github.com/apache/pulsar-connectors/pull/33#issuecomment-4682207624
I went ahead and added an integration test for this change (pushed as a
commit on the branch) so the driver upgrade is actually validated against a
real ClickHouse server.
**`ClickHouseJdbcSinkIntegrationTest`** spins up ClickHouse via
Testcontainers, creates the `pulsar_messages` table, opens the sink, writes a
record, and verifies the row landed. It lives in `jdbc/clickhouse/src/test`, so
it runs in the existing CI `test` job alongside the other container-based
connector tests (mongo, elasticsearch, cassandra) — no `ci.yaml` changes needed.
The test is a genuine regression guard for #32 — **red on the old driver,
green on the new one**:
- ✅ **Driver 0.9.8 (this PR):** `testOpenDiscoversTableAndWrites` passes.
- 🔴 **Driver 0.4.6 (before):** fails at exactly the #32 location:
```
java.lang.Exception: Not able to find table: pulsar_messages
at org.apache.pulsar.io.jdbc.JdbcUtils.getTableId(JdbcUtils.java:112)
at
org.apache.pulsar.io.jdbc.JdbcAbstractSink.open(JdbcAbstractSink.java:116)
```
One deliberate design choice: the table is created and the result is read
back over ClickHouse's **HTTP interface** rather than JDBC, so the driver under
test is exercised *only* by the sink. That's what makes the failure surface
precisely at `getTableId` instead of in the test harness.
Changes in the added commit:
- `testcontainers-clickhouse` added to the version catalog
- test dependencies added to `jdbc/clickhouse/build.gradle.kts`
- the new integration test
Verified locally on JDK 17 with Docker; full
`:jdbc:pulsar-io-jdbc-clickhouse:build` (spotless + test + nar) is green. Once
CI is approved, this should run automatically.
A couple of notes for whoever merges:
- The test pins `clickhouse/clickhouse-server:24.3` — happy to bump that to
whatever version the project prefers to validate against.
- This needs the CI workflow approved (still `action_required`) for it to
run in the pipeline.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
