Re: [I] Support S3 Access Points with Access Point to Bucket mapping [iceberg-python]

2024-09-17 Thread via GitHub
github-actions[bot] closed issue #452: Support S3 Access Points with Access Point to Bucket mapping URL: https://github.com/apache/iceberg-python/issues/452 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

Re: [I] Support S3 Access Points with Access Point to Bucket mapping [iceberg-python]

2024-09-17 Thread via GitHub
github-actions[bot] commented on issue #452: URL: https://github.com/apache/iceberg-python/issues/452#issuecomment-2357239777 This issue has been closed because it has not received any activity in the last 14 days since being marked as 'stale' -- This is an automated message from the Apac

Re: [I] Support S3 Access Points with Access Point to Bucket mapping [iceberg-python]

2024-09-02 Thread via GitHub
github-actions[bot] commented on issue #452: URL: https://github.com/apache/iceberg-python/issues/452#issuecomment-2325404746 This issue has been automatically marked as stale because it has been open for 180 days with no activity. It will be closed in next 14 days if no further activity oc

Re: [I] Support S3 Access Points with Access Point to Bucket mapping [iceberg-python]

2024-03-06 Thread via GitHub
JGynther commented on issue #452: URL: https://github.com/apache/iceberg-python/issues/452#issuecomment-1980790119 Testing a very simple wrapper like: ```Python from pyarrow.fs import S3FileSystem class WrappedS3FileSystem(S3FileSystem): def __init__(self, bucket_over

Re: [I] Support S3 Access Points with Access Point to Bucket mapping [iceberg-python]

2024-03-03 Thread via GitHub
JGynther commented on issue #452: URL: https://github.com/apache/iceberg-python/issues/452#issuecomment-1975812563 Finally had a chance to poke this. To me it seems that there is no easy way out to implement this. When creating and scanning a StaticTable the actual location of a parti

Re: [I] Support S3 Access Points with Access Point to Bucket mapping [iceberg-python]

2024-02-21 Thread via GitHub
JGynther commented on issue #452: URL: https://github.com/apache/iceberg-python/issues/452#issuecomment-1956310314 Thinking about this further I see two options: 1. Change the way metadata is read when the target is an access point alias, either by checking for the "-s3alias" suffix (

Re: [I] Support S3 Access Points with Access Point to Bucket mapping [iceberg-python]

2024-02-20 Thread via GitHub
JGynther commented on issue #452: URL: https://github.com/apache/iceberg-python/issues/452#issuecomment-1955236592 I found the same issue. However using Access Point S3 alias it seems to work just fine. Here is a minimal PyArrow example that reads a file: ```python from pyarrow imp

Re: [I] Support S3 Access Points with Access Point to Bucket mapping [iceberg-python]

2024-02-20 Thread via GitHub
hussein-awala commented on issue #452: URL: https://github.com/apache/iceberg-python/issues/452#issuecomment-1955233783 I think the problem is in Pyarrow S3FileSystem, I found this open issue https://issues.apache.org/jira/browse/ARROW-9669 (moved to https://github.com/apache/arrow/issues/2

Re: [I] Support S3 Access Points with Access Point to Bucket mapping [iceberg-python]

2024-02-20 Thread via GitHub
JGynther commented on issue #452: URL: https://github.com/apache/iceberg-python/issues/452#issuecomment-1955209888 Here is my minimal code example (that fails): ```python from pyiceberg.table import StaticTable # Latest metadata file object = "iceberg/metadata/00068-b5e701c2-

Re: [I] Support S3 Access Points with Access Point to Bucket mapping [iceberg-python]

2024-02-20 Thread via GitHub
hussein-awala commented on issue #452: URL: https://github.com/apache/iceberg-python/issues/452#issuecomment-1955204486 Could you share the full exception you get when you run the scan query? -- This is an automated message from the Apache Git Service. To respond to the message, please log

[I] Support S3 Access Points with Access Point to Bucket mapping [iceberg-python]

2024-02-20 Thread via GitHub
JGynther opened a new issue, #452: URL: https://github.com/apache/iceberg-python/issues/452 ### Feature Request / Improvement [S3 Access Points](https://docs.aws.amazon.com/AmazonS3/latest/userguide/access-points.html) are a way to scale data access by providing additional endpoints