stevenzwu commented on PR #7161: URL: https://github.com/apache/iceberg/pull/7161#issuecomment-1763706777
is the partition time an event time or ingestion/processing time? or asking in a different way, how many active minutes do the Flink writer job process for every commit cycle? I feel this ongoing work might work better for you https://docs.google.com/document/d/13N8cMqPi-ZPSKbkXGOBMPOzbv2Fua59j8bIjjtxLWqo https://www.youtube.com/watch?v=GJplmOO7ULA On the other hand, I also recognize this change imposed a behavior change that doesn't work for your use case. We can revert the distribution mode change in the `FlinkSink` class. To enable the custom bucketing partitioner, users will have to manually apply the custom `keyBy`. ``` input.keyBy(new PartitionKeySelector(partitionSpec, iSchema, flinkRowType)) ``` @kengtin can you create a PR for reverting the FlinkSink change? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org For additional commands, e-mail: issues-h...@iceberg.apache.org
