Fokko commented on code in PR #6590:
URL: https://github.com/apache/iceberg/pull/6590#discussion_r1071885001
##########
python/pyiceberg/table/__init__.py:
##########
@@ -328,14 +331,23 @@ def plan_files(self) -> Iterator[FileScanTask]:
partition_evaluators: Dict[int, Callable[[DataFile], bool]] =
KeyDefaultDict(self._build_partition_evaluator)
- for manifest in manifests:
- partition_filter = partition_evaluators[manifest.partition_spec_id]
- all_files = files(io.new_input(manifest.manifest_path))
- matching_partition_files = filter(partition_filter, all_files)
+ futures = []
+ with ThreadPoolExecutor() as executor:
+ for manifest in manifests:
+ partition_filter =
partition_evaluators[manifest.partition_spec_id]
- matching_partition_data_files = map(_check_content,
matching_partition_files)
+ def read_manifest():
Review Comment:
Not a big fan of the closure here, I think it is very prone to errors.
##########
python/pyiceberg/cli/output.py:
##########
@@ -212,3 +228,6 @@ def spec(self, spec: PartitionSpec) -> None:
def uuid(self, uuid: Optional[UUID]) -> None:
self._out({"uuid": str(uuid) if uuid else "missing"})
+
+ def result_table(self, rows: List[Tuple[Any]]) -> None:
+ print(json.dumps(rows))
Review Comment:
```suggestion
self._out(rows)
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org
For additional commands, e-mail: issues-h...@iceberg.apache.org
