danielcweeks commented on PR #11112: URL: https://github.com/apache/iceberg/pull/11112#issuecomment-2371796750
> @danielcweeks let me know what you think about this. One alternative is to maybe provide both as in having S3LocationProvider as is and a base2 option for the ObjectStoreLocationProvider to keep the option for partition values along with directories @ookumuso I'm still of the opinion that we should just incorporate this into the existing object store location provider. Ultimately, we can change the hashing behavior as the existing behavior is based on earlier discussions with S3, so replacing it or providing alternative options is fine. I do think partition values in the path still needs to be an option. I think we also need to consider adding a path separator to help with some of the maintenance routines. For example `s3://bucket/<prefix>/<db>/<table>/data/00101001/0010100111011...`. The `/` in the hash portion allows for `2^8` prefixes to allow listing under those paths for clients that list. If we omit that, we get `2^24` prefixes that need to be listed under the data directory, which makes maintenance difficult. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org For additional commands, e-mail: issues-h...@iceberg.apache.org
