JNSimba commented on issue #223:
URL:
https://github.com/apache/doris-spark-connector/issues/223#issuecomment-3479053761
> I am using Java spark-doris-connector-spark-3.3 version 25.1.0 with Spark
3.3.0. My code below pulls all the data, all columns from table even if I have
selected few columns in the query and added a where clause. They are completely
ignored. Tested with both partitioned and non-partioned table: df =
spark.read().format("doris").option("doris.fenodes", dorisFeNodes)
.option("doris.table.identifier", dorisSchema + "." + tableName)
.option("user", user) .option("password",
password).option("doris.request.tablet.size", tabletSize)
.option("doris.query", query) .option("doris.log.level", "DEBUG")
.option("doris.deserialize.arrow", "false") //Tried setting it, didn't help
.load();
What version of Doris are you using? If it's 2.x, it might be a known issue.
You can upgrade to 3.x or use ArrowFlight to read it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
