smaheshwar-pltr commented on code in PR #3364:
URL: https://github.com/apache/iceberg-python/pull/3364#discussion_r3261834703
##########
pyiceberg/table/__init__.py:
##########
@@ -1916,77 +2056,43 @@ def _min_sequence_number(manifests: list[ManifestFile])
-> int:
return INITIAL_SEQUENCE_NUMBER
-class DataScan(TableScan):
- def _build_partition_projection(self, spec_id: int) -> BooleanExpression:
- project = inclusive_projection(self.table_metadata.schema(),
self.table_metadata.specs()[spec_id], self.case_sensitive)
- return project(self.row_filter)
-
- @cached_property
- def partition_filters(self) -> KeyDefaultDict[int, BooleanExpression]:
- return KeyDefaultDict(self._build_partition_projection)
+def _to_arrow_via_file_scan_tasks(scan: BaseScan, tasks:
Iterable[FileScanTask]) -> pa.Table:
Review Comment:
Introducing this helper + (the one below) specialised for `FileScanTask`. We
don't want to have this be the default implementation on `BaseScan` because it
requires `FileScanTask` specifically and not all table scans will have
`FileScanTask` planned in general (i.e. changelogs)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
