ookumuso commented on PR #11112: URL: https://github.com/apache/iceberg/pull/11112#issuecomment-2349883643
> Left some comments but a had a possibly naive question to just check my understanding: > > In the past for object storage provider, we've used a wider character set in the hash portion of the file as a means to maximize entropy and ultimately improve heat distribution (#7128). With this new approach are we saying we can get a good enough heat distribution and at the same time enable s3 to scale capacity more quickly? @amogh-jahagirdar Yes, essentially both base64 and base2 distributes the traffic evenly which is the most crucial thing. With auto-scaling they can both support high TPS workloads. Main difference between them is how fast they auto-scale to the point where there are no more throttles observed. Base2 being more effective there due to every next char in the hash taking ~50% of the traffic. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org For additional commands, e-mail: issues-h...@iceberg.apache.org
