amogh-jahagirdar commented on PR #11478: URL: https://github.com/apache/iceberg/pull/11478#issuecomment-2474025071
>@amogh-jahagirdar I don't see https://github.com/apache/iceberg/pull/11273 being back-ported to Spark 3.3, 3.4 yet. Shall we skip changes to Spark 3.3, 3.4 until that is done? @manuzhang Definitely agree that we shouldn't merge this change to 3.3/3.4 until the maintenance piece is backported, I'm working on the backport. Though I think we can leave this PR open for discussion, and I also wanted to raise this to see which tests would need to be updated in CI (and I wanted to get some clarity if 3.3/3.4 tests were different somehow or not). What I'd propose is we backport the sync maintenance to 3.3/3.4 since I think we'll need that anyways, and then if there's consensus here/mailing list to change the default table property then we could just update all the spark tests in one go. Alternatively like @aokolnychyi mentioned above we could just change the specific spark conf in the case we don't want to backport to older spark versions. I am leaning towards just doing the backport because I think we should just need to take a stance if we think the new default is just going to be generally better, rather than making it specific to a spark version. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org For additional commands, e-mail: issues-h...@iceberg.apache.org
