wirybeaver opened a new pull request, #119: URL: https://github.com/apache/datafusion-java/pull/119
## Which issue does this PR close? Closes #118. ## Rationale for this change Java table providers currently receive no physical scan request. A selective source cannot choose an access path from pushed predicates, read only projected columns, or distinguish exact filtering from candidate pruning. Expose that existing DataFusion contract while keeping the Java/JNI binding thin and source-agnostic. ## What changes are included in this PR? Add an opt-in `ScanTableProvider` with per-expression pushdown capabilities and immutable projection/filter/limit requests. It returns a lazy `TableScan`, separating source planning from opening an Arrow reader. Existing providers retain full-schema scans and native projection fallback. ```text DataFusion optimizer -> Java filter capabilities DataFusion planning -> planScan(request) -> lazy TableScan DataFusion execution -> fresh ArrowReader -> Arrow C stream -> Rust operators ``` DataFusion still owns residual filtering and limit placement. Planned readers produce the requested projected schema, including zero-column batches that preserve row counts. Source-specific access-path decisions stay in provider implementations; no storage SDK or DataFusion Rust-core change is introduced. This remains a bounded, single-partition leaf-scan API, not a binding for arbitrary child-consuming physical operators. ## Are these changes tested? A Java/JNI/Arrow integration harness registers an in-memory selective source and inspects its planned requests and actual reads. EXPLAIN plans without opening the source; execution requests only the matching row and projected column. Inexact and unsupported predicates retain residual evaluation before LIMIT, exact predicates permit limit pushdown, and independent scans keep separate requests and readers. The existing full-schema provider path is exercised through the same native boundary. The source is synthetic: this does not establish performance or correctness against an external storage service. ## Are there any user-facing changes? The new `ScanTableProvider`, `TableScanRequest` and `TableScan` APIs are additive. Existing `TableProvider` and `SimpleTableProvider` implementations need no changes. The provider guide documents projection, filter, limit, threading and reader-ownership contracts. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
