javrasya commented on PR #9464: URL: https://github.com/apache/iceberg/pull/9464#issuecomment-1890799411
Another idea is to still use original `writeUTF` since that is covering the most, but write it as chunks. The biggest character can be 3 bytes according to the `writeUTF` function. If we go for the worst case, we can create multiple chunks out of the json value worth of **65kb / 3** and write that each bit which is guaranteed to fit in a UTF. That would entail zero copy pasting from the Flink source code. https://github.com/apache/iceberg/compare/main...javrasya:iceberg:issue-9410-introduce-v3-serde-write-utf-as-chunks?expand=1 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org For additional commands, e-mail: issues-h...@iceberg.apache.org
