smaheshwar-pltr commented on code in PR #3364:
URL: https://github.com/apache/iceberg-python/pull/3364#discussion_r3260039828
##########
pyiceberg/table/__init__.py:
##########
@@ -1707,20 +1771,163 @@ def __init__(
row_filter: str | BooleanExpression = ALWAYS_TRUE,
selected_fields: tuple[str, ...] = ("*",),
case_sensitive: bool = True,
- snapshot_id: int | None = None,
options: Properties = EMPTY_DICT,
limit: int | None = None,
- catalog: Catalog | None = None,
- table_identifier: Identifier | None = None,
):
self.table_metadata = table_metadata
self.io = io
self.row_filter = _parse_row_filter(row_filter)
self.selected_fields = selected_fields
self.case_sensitive = case_sensitive
- self.snapshot_id = snapshot_id
self.options = options
self.limit = limit
+
+ @abstractmethod
+ def projection(self) -> Schema: ...
+
+ @abstractmethod
+ def plan_files(self) -> Iterable[ScanTask]: ...
+
+ def to_arrow(self) -> pa.Table:
Review Comment:
Materialization stays abstract on `BaseScan`. Both `DataScan` and
`IncrementalAppendScan` implement `to_arrow` / `to_arrow_batch_reader` as
one-line delegations to the module-level helpers
`_to_arrow_via_file_scan_tasks` / `_to_arrow_batch_reader_via_file_scan_tasks`
above.
A `BaseScan`-level default would require `Iterable[FileScanTask]`, but
`BaseScan.plan_files()` returns `Iterable[ScanTask]` — Liskov-widened so that
future non-file scans (e.g. changelog) can return a different task type. Mypy
`arg-type` makes the default-on-base form impossible without specialising the
base. Helpers keep the dedup without that constraint.
`to_pandas` / `to_polars` / `to_duckdb` / `to_ray` *do* get pulled up to
`BaseScan` as defaults — they only need `to_arrow()` on `self`, no
`FileScanTask` typing. [Prior
thinking](https://github.com/apache/iceberg-python/pull/2230#discussion_r2222471562).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
