jackye1995 commented on PR #7194: URL: https://github.com/apache/iceberg/pull/7194#issuecomment-1483370758
> this is introducing something that we don't do in the Iceberg project: Invoke an external provider to perform operations on behalf of the library. Yes I 100% agree with that. But first of all this is within the aws module which is an external provider integration. And secondly, without this PR, we will likely also contribute things like CloudWatch, SQS and SNS metrics reporter. So the questions maybe you could help share your opinions are that: 1. would it work if we contribute CloudWatch, SQS and SNS metrics reporters, such that these compaction actions could be triggered from that point, which is outside the Iceberg library 2. would it be okay with the community if we publish a separated open source repository for metrics reporters like this, such that they will be a part of the EMR distribution, and users can plug it into a catalog if necessary. 3. we could also develop a metrics reporter for a specific engine like Spark, and we contact the Spark endpoint to submit such a job instead of doing it through an external service provider's API. Would that be a viable approach? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
