sky1229329146 opened a new issue, #14781: URL: https://github.com/apache/doris/issues/14781
### Search before asking - [X] I had searched in the [issues](https://github.com/apache/doris/issues?q=is%3Aissue) and found no similar issues. ### Version 一：用的是这个依赖 spark-doris-connector-2.3_2.11 二： dori版本：1.1.1 spark版本：2.4.0 ### What's Wrong? 在查询doris时使用option("doris.filter.query",s"$queryWhere" )预过滤后，再用where条件过滤，发现doris不会使用预过滤的数据而是之间将where重新作为全量数据的过滤条件，导致之前的预先过滤失效 ### What You Expected? 查询doris时使用option("doris.filter.query",s"$queryWhere" )预过滤后，再用where条件去查询数据时，该预过滤条件不会失效 ### How to Reproduce? 代码如下 工具类： def getDorisSession(tableName:String,queryWhere:String="",queryFileds:String = PropertiesUtil.getPropertiesByKey("QueryFiles")) = { SparkUtil.getSparkSession(this.getClass.getSimpleName) .read .format("doris") .option("doris.table.identifier",tableName ) .option("doris.read.field",s"$queryFileds" ) .option("doris.filter.query",s"$queryWhere" ) .option("doris.fenodes",PropertiesUtil.getPropertiesByKey("DORIS_FE_RESFUL_PORT")) .option("user",PropertiesUtil.getPropertiesByKey("DORIS_USERNAME")) .option("password",PropertiesUtil.getPropertiesByKey("DORIS_PASSWORD")) .option("batch_size","4096") //每次进行批量计算的行数 .option("enable_vectorized_engine","true") //向量化执行引擎;建表时设置NOT NULL或者特殊值取代null,防止性能劣化 .load() } step1.查询限制某个条件的数据， val frame = DorisUtil.getDorisSession(PropertiesUtil.getPropertiesByKey("DORIS_TABLE_NAME_CAR"),"id='51100254001310801150'") step2.打印看下符合当前条件的数据 frame.show() step3.创建临时表 frame.createOrReplaceTempView("temptable") step4.在临时表中限定不是这个id,居然还是能查到其他id的数据 sparkSession.sql( s""" |select |* |from temptable |where id !='51100254001310801150' |""".stripMargin).show() 备注：如果在step1的下面添加 frame.cache() ,则step4中数据为空 ### Anything Else? _No response_ ### Are you willing to submit PR? - [ ] Yes I am willing to submit a PR! ### Code of Conduct - [X] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
