adriangb opened a new pull request, #26437: URL: https://github.com/apache/datafusion/pull/26437
## Which issue does this PR close? - Closes https://github.com/apache/datafusion/issues/23109. This picks up the approach of https://github.com/apache/datafusion/pull/23235, which the stale bot closed without a review decision. This PR adds the clone-path, `SET` and expression-path coverage, and documents the precedence rule. ## Rationale for this change `datafusion.sql_parser.dialect` accepts only a closed list of names. `SessionState::sql_to_statement` and `SessionState::sql_to_expr_with_alias` resolve that name with `dialect_from_str` on every parse. Thus an application with its own `sqlparser::dialect::Dialect`, for example one built with `derive_dialect!` on top of `PostgreSqlDialect` (https://github.com/apache/datafusion-sqlparser-rs/pull/2174), cannot plug it in. It must wrap every parse entry point itself. A call to a stock method such as `SessionContext::sql` then parses with the wrong dialect, and the query fails with a parse error. ## What changes are included in this PR? - New `SessionStateBuilder::with_sql_dialect(Arc<dyn Dialect + Send + Sync>)`, plus the getters `SessionState::sql_dialect()` and `SessionStateBuilder::sql_dialect()`. - `sql_to_statement` and `sql_to_expr_with_alias` use the dialect object when it is set. All other parse paths go through these two methods: `SessionContext::sql`, `sql_with_options`, `create_logical_plan`, `create_logical_expr` and `sql_to_expr`. - Precedence: the dialect object wins over the dialect name. This includes an explicit `dialect` argument to `sql_to_statement` or `sql_to_expr`, and a later `SET datafusion.sql_parser.dialect = ...`. `SET` still changes the config value, but not the parser. - `SessionStateBuilder::new_from_existing` and `From<SessionState>` keep the dialect object. - If no dialect object is set, the behavior does not change. `datafusion-cli` resolves the dialect name itself and does not expose a way to set a dialect object, so this PR does not change it. ## Are these changes tested? Yes. `test_custom_sql_dialect` uses a small hand-written dialect: PostgreSQL plus `SELECT * EXCEPT (...)`. The `derive-dialect` feature of sqlparser is not enabled in this workspace. The test checks that: - the PostgreSQL dialect name alone rejects the query (default behavior unchanged), - `ctx.sql(...)`, `create_logical_plan`, `sql_to_statement`, `sql_to_expr` and `create_logical_expr` parse with the dialect object, - a `SET datafusion.sql_parser.dialect` statement does not replace it, - `new_from_existing` and `From<SessionState>` keep it. The rustdoc example on `with_sql_dialect` runs as a doctest. ## Are there any user-facing changes? Yes. This PR adds a new public builder method and two getters. There are no breaking changes. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
