JNSimba opened a new pull request, #380: URL: https://github.com/apache/doris-spark-connector/pull/380
## Problem Summary S3 TVF uploads copy each file again when creating the SDK request body, after the writer has already copied its buffer. Use a repeatable content provider to remove that extra copy while allowing retries to read the complete content again. Add upload size and duration logs, and INSERT object count and duration logs, for both successful and failed operations. These changes apply to the shared base module used by Spark 2/3/4. ## Validation - `S3ClientObjectStoreTest`: 1 test passed with the Spark 3.5 profile and JDK 17. Covers request metadata, absence of the extra copy, and independent streams for repeatable reads. The no-copy assertion fails on the original implementation. - `git diff --check` passed. - Real-cluster integration tests were not run. ## Checklist(Required) 1. Does it affect the original behavior: Yes 2. Has unit tests been added: Yes 3. Has document been added or modified: No Need 4. Does it need to update dependencies: No 5. Are there any changes that cannot be rolled back: No -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
