Re: [PR] AWS: Change S3FileIO to use SHA1 based checksums [iceberg]

2024-11-06 Thread via GitHub
muddyfish closed pull request #10293: AWS: Change S3FileIO to use SHA1 based checksums URL: https://github.com/apache/iceberg/pull/10293 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

Re: [PR] AWS: Change S3FileIO to use SHA1 based checksums [iceberg]

2024-11-01 Thread via GitHub
github-actions[bot] commented on PR #10293: URL: https://github.com/apache/iceberg/pull/10293#issuecomment-2452745563 This pull request has been marked as stale due to 30 days of inactivity. It will be closed in 1 week if no further activity occurs. If you think that’s incorrect or this pul

Re: [PR] AWS: Change S3FileIO to use SHA1 based checksums [iceberg]

2024-08-13 Thread via GitHub
steveloughran commented on PR #10293: URL: https://github.com/apache/iceberg/pull/10293#issuecomment-2286820292 checksumming input streams on the v2 sdk kills performance, not matter what you use. might be best to turn off. see https://github.com/apache/hadoop/pull/6441 -- This is an aut

Re: [PR] AWS: Change S3FileIO to use SHA1 based checksums [iceberg]

2024-05-13 Thread via GitHub
singhpk234 commented on PR #10293: URL: https://github.com/apache/iceberg/pull/10293#issuecomment-2108558359 > What do you think @muddyfish @singhpk234? Sounds good, @amogh-jahagirdar ! -- This is an automated message from the Apache Git Service. To respond to the message, please lo

Re: [PR] AWS: Change S3FileIO to use SHA1 based checksums [iceberg]

2024-05-10 Thread via GitHub
muddyfish commented on PR #10293: URL: https://github.com/apache/iceberg/pull/10293#issuecomment-2104654017 I think that's a reasonable comment, and I'd be happy going for that path forward. -- This is an automated message from the Apache Git Service. To respond to the message, please lo

Re: [PR] AWS: Change S3FileIO to use SHA1 based checksums [iceberg]

2024-05-09 Thread via GitHub
amogh-jahagirdar commented on PR #10293: URL: https://github.com/apache/iceberg/pull/10293#issuecomment-2103607159 I have similar questions to @singhpk234. I think changing the the checksum default to a more expensive SHA1 validation may be more expensive without any significant benefits. T

[PR] AWS: Change S3FileIO to use SHA1 based checksums [iceberg]

2024-05-09 Thread via GitHub
muddyfish opened a new pull request, #10293: URL: https://github.com/apache/iceberg/pull/10293 ## Issue Using [S3 Express](https://docs.aws.amazon.com/AmazonS3/latest/userguide/s3-express-one-zone.html) for Iceberg was previously failing multipart upload integration tests as well as