Burak Yavuz created SPARK-50618:
-----------------------------------
Summary: Refactor Data[Frame|Stream]Reader to leverage the analyzer
Key: SPARK-50618
URL: https://issues.apache.org/jira/browse/SPARK-50618
Project: Spark
Issue Type: Improvement
Components: SQL
Affects Versions: 4.0.0
Reporter: Burak Yavuz
Assignee: Burak Yavuz
The DataFrameReader and DataStreamReader typically creates analyzed relations
as part of their respective `.load()` methods.
This creates inconsistencies for what rules get applied to the query plan as
part of Catalyst depending on your API of choice, such as SQL vs Python or SQL
vs Scala.
The goal of this Jira is to refactor the logic in DataFrameReader and
DataStreamReader classes to create unresolved plans that get analyzed as part
of Catalyst.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
