geruh opened a new pull request, #2947: URL: https://github.com/apache/iceberg-python/pull/2947
closes #2945 # Rationale for this change This PR removes the `SortedContainers` dependency. Looking at the behavior of sorted containers we can simplify the logic for merging manigests and collecting the results while maintaining identical behavior. **What the logic today was doing:** 1. Submit all manifest merge tasks to thread pool (pallelism starts with executor) 2. Collect futures as they complete using `as_completed()` which is out of order 3. Store completed futures in a SortedList to maintain order by submission 4. Extract all results from the sorted futures 5. Flatten and return **What we do now:** 1. Submit all manifest merge tasks to thread pool (pallelism starts with executor) 2. Iterate through futures in submission order, calling .result() on each 3. Flatten and return This shows we must collect the results before the next step. So we can iterate futures directly and call `.result()` in order. This blocks the main thread until each future completes, but doesn't block worker threads and they all continue running in parallel. ## Are these changes tested? All existing tests pass. ## Are there any user-facing changes? No -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
