gitzwz commented on PR #11280: URL: https://github.com/apache/iceberg/pull/11280#issuecomment-2454158309
> This sounds interesting, is it just the ser-de that causes this ? what about the increase in memory pressure to hold this in memory ? Yes for the first question, in my SDK service, I need to get all the spec id (do ser-de) for every FileScanTask, this is one reason why it is so slow. Sure for the increase in memory pressure, it's a good question. Is there another way to solve ser-de problem? I collect all spec id to check if one column was previously used as a partition field. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org For additional commands, e-mail: issues-h...@iceberg.apache.org
