jiakai-li commented on code in PR #1453:
URL: https://github.com/apache/iceberg-python/pull/1453#discussion_r1903363474
##########
pyiceberg/io/pyarrow.py:
##########
@@ -351,76 +344,146 @@ def parse_location(location: str) -> Tuple[str, str,
str]:
return uri.scheme, uri.netloc, f"{uri.netloc}{uri.path}"
def _initialize_fs(self, scheme: str, netloc: Optional[str] = None) ->
FileSystem:
- if scheme in {"s3", "s3a", "s3n", "oss"}:
- from pyarrow.fs import S3FileSystem
+ """Initialize FileSystem for different scheme."""
+ if scheme in {"oss"}:
+ return self._initialize_oss_fs(scheme, netloc)
Review Comment:
I didn't find an authoritive document explicily saying it's not supported
from pyarrow. But I tested it locally and it doesn't work for Alibaba. Kevin
also helped to check the pyarrow code in [this
comment](https://github.com/apache/iceberg-python/pull/1453#discussion_r1902287419).
Seems pyarrow is using the `x-amz-bucket-region` header to determine the
bucket region, which seems to be an AWS thing.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
