andygrove commented on issue #2457: URL: https://github.com/apache/datafusion-comet/issues/2457#issuecomment-4144871957
This appears to have been fixed through multiple layers of validation that now prevent duplicate struct field names from reaching the Arrow schema conversion: 1. **CreateNamedStruct** (`structs.scala`) — rejects duplicate field names, falls back to Spark (commit abd9f858e) 2. **Shuffle** (`QueryPlanSerde.scala`) — rejects structs with duplicate field names (commit 9203f1734) 3. **Native DataFusion scan** (`CometScanRule.scala`) — rejects duplicates in case-insensitive mode (commit f6d84b138) 4. **Rust error handling** (`errors.rs`) — converts DataFusion deduplication errors to proper Spark errors instead of crashing (commit 9929745d7) @wForget can this issue be closed? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
