peterxcli commented on PR #5407: URL: https://github.com/apache/datafusion-comet/pull/5407#issuecomment-5452449714
Confirmed against the v4.1.3 source @sunchao — `matchIdField` dispatches before any name matching and `_LEGACY_ERROR_TEMP_2093` lives only in `matchCaseInsensitiveField`, so the collision must not affect an ID-resolved `v`. Fixed in e40b939cd: the per-field ID exemption is hoisted above both checks in the referenced-column walk, so an ID-resolved column skips name-duplicate validation and the Unicode-fold guard alike (an ID-bearing requested field never reaches Spark's name matcher — unmatched IDs get a fake name, not a name fallback — so this mirrors the dispatch exactly). Duplicate-ID validation is retained in `remap_physical_schema`. Covered by an adapter-level Rust test (`v`/ID 1 vs physical `v`+`V` succeeds with field-ID reads; the same shape without IDs keeps the ambiguity error) and a `CometNativeReaderSuite` e2e that writes your exact file — case-sensitive `v VARIANT`/ID 1 + `V INT`/ID 2 — and reads `v` by ID case-insensitively, matching Spark natively. The pre-existing no-ID duplicate regression still errors as before. Validated on Spark 4.0 and 4.1. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
