94Sip opened a new issue, #9791: URL: https://github.com/apache/iceberg/issues/9791
### Apache Iceberg version 1.4.3 (latest release) ### Query engine Other ### Please describe the bug 🐞 I realize this might be a bug with AWS Glue, but thought I would post here as I'm not sure where to go... I have a Kinesis stream that is persisting (inserting) data to an iceberg table, via a Glue streaming job. I'm following the glue streaming pattern as published [here](https://github.com/aws-samples/realtime-serverless-gdpr-ccpa-in-a-data-lake/blob/main/icebergstream.py) After that process completes, I have another Glue job that is using the --extra-jars parameter to use the v1.4.3 release of Iceberg so that I can use the [changelog](https://iceberg.apache.org/docs/latest/spark-procedures/#create_changelog_view) feature. However, when I create the changelog, the activity from the kinesis stream is NOT reflected in the changelog. Is that because Kinesis is writing directly to S3 and bypassing the actual INSERT into the table? I have confirmed that if I do an explicit INSERT, via Athena, or as a part of a MERGE INTO, in a glue job or Athena, that the INSERT is picked up by the changelog. Do I need to use a different method to write to the iceberg table? For example one of the methods shown [here](https://docs.aws.amazon.com/glue/latest/dg/aws-glue-programming-etl-format-iceberg.html#aws-glue-programming-etl-format-iceberg-write)? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org For additional commands, e-mail: issues-h...@iceberg.apache.org
