wary commented on code in PR #285:
URL:
https://github.com/apache/doris-spark-connector/pull/285#discussion_r2010207327
##########
spark-doris-connector/spark-doris-connector-base/src/main/java/org/apache/doris/spark/client/write/AbstractStreamLoadProcessor.java:
##########
@@ -85,19 +83,19 @@ public abstract class AbstractStreamLoadProcessor<R>
extends DorisWriter<R> impl
private final Map<String, String> properties;
- private final String format;
+ private final DataFormat format;
protected String columnSeparator;
- private String lineDelimiter;
+ private byte[] lineDelimiter;
private final boolean isGzipCompressionEnabled;
private String groupCommit;
private final boolean isPassThrough;
- private PipedOutputStream output;
+ private StreamLoadEntity output;
Review Comment:
For better performance, The test results in our actual scenario are as
follows
Before:
100.8 GiB, Time: 23 min
After:
100.8 GiB, Time: 12 min
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
