danielcweeks commented on PR #10283:
URL: https://github.com/apache/iceberg/pull/10283#issuecomment-2103091902
@dimas-b Unfortunately, I agree there will likely be problems if people use
special characters in the pathing prefix that's outside of Iceberg's control,
but overall, I think we want to encourage (if not prohibit through the spec)
usage of unencoded characters. While S3 supports this behavior, it causes a
number of compatibility problems.
For example, this change would likely be incompatible with S3AFileSystem or
any other system that uses the java URI implementation. We have this same type
of issue with GCS, but for different reasons. I'm not sure about other
implementations like pyarrow, OpenDAL in the rust, or Trino.
Overall, I think it's better to fail fast where interoperability is a
concern as that's more important than supporting the full s3 key space.
I would suggest:
1. fix the field name encoding
2. continue the larger community discussion around what are "valid" paths
3. recommend people avoid problematic naming
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org
For additional commands, e-mail: issues-h...@iceberg.apache.org
