stefnba commented on issue #30: URL: https://github.com/apache/iceberg-python/issues/30#issuecomment-1922014952
Maybe this helps. A PyArrow Dataset can be initiated from a list of file paths: Create a FileSystemDataset from explicitly given files. The files must be located on the same filesystem given by the filesystem parameter. Note that in contrary of construction from a single file, passing URIs as paths is not allowed. More info [here](https://arrow.apache.org/docs/python/generated/pyarrow.dataset.dataset.html). To make it work, the corresponding cloud filesystem, e.g. `pyarrow.fs.S3FileSystem` has to be specified, see [here](https://arrow.apache.org/docs/python/generated/pyarrow.fs.S3FileSystem.html). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org For additional commands, e-mail: issues-h...@iceberg.apache.org
