LionTao commented on issue #6481: URL: https://github.com/apache/iceberg/issues/6481#issuecomment-1364677516
Hi, thank you for your reply, and merry christmas🎄 > Could you elaborate a bit more on what you would like to do here? **I am building a distributed computing engine for trajectory data based on Faas service, Dapr and Iceberg. I want to minimize the network IO. So I think using S3-Select may be helpful.** After digging around the code, I found currently the FileIO API relies on Stream to process content. But the S3-select response seems to be different. So I think if we need to utilize the S3-Select ability, we need some new mechanism to deal with Input files. Also, this new mechanism may not be supported by all storage, so it could be AWS-specific. I'm not sure whether it is ideal for the extensibility of Iceberg. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org For additional commands, e-mail: issues-h...@iceberg.apache.org
