agturley commented on code in PR #10981:
URL: https://github.com/apache/nifi/pull/10981#discussion_r3036436814
##########
nifi-extension-bundles/nifi-elasticsearch-bundle/nifi-elasticsearch-restapi-processors/src/main/java/org/apache/nifi/processors/elasticsearch/PutElasticsearchJson.java:
##########
@@ -145,6 +165,55 @@ public class PutElasticsearchJson extends
AbstractPutElasticsearch {
.required(true)
.build();
+ static final PropertyDescriptor MAX_BATCH_SIZE = new
PropertyDescriptor.Builder()
+ .name("Max Batch Size")
+ .description("The maximum amount of data to send in a single
Elasticsearch _bulk API request. " +
+ "For NDJSON and JSON Array modes, FlowFiles are
accumulated until this threshold is reached, then flushed. " +
+ "For Single JSON mode, this acts as a size-based safety
limit: if the accumulated FlowFiles exceed this size " +
+ "before Max FlowFiles Per Batch is reached, the request is
flushed early. " +
+ "Elasticsearch recommends 5-15 MB per bulk request for
optimal performance.")
+ .defaultValue("10 MB")
+ .addValidator(StandardValidators.DATA_SIZE_VALIDATOR)
+ .expressionLanguageSupported(ExpressionLanguageScope.ENVIRONMENT)
+ .required(true)
+ .build();
Review Comment:
This is a good point, while I agree that it's probably fine... i want to
make sure there is no issues upgrading to this. I set the default value to "10
MB" however, it's no longer a required field. if a user upgrades from 2.8 or
below, it'll set the max batch size to "Set empty string" which now means
unlimited batch size like before. if someone creates a new PutElasticsearchJson
processor it'll default to "10 MB". This is also shown in the description for
"Max Batch Size".
Now old stuff keeps the unlimited batch size, while new stuff is defaulted
to 10 MB.
##########
nifi-extension-bundles/nifi-elasticsearch-bundle/nifi-elasticsearch-restapi-processors/src/main/java/org/apache/nifi/processors/elasticsearch/PutElasticsearchJson.java:
##########
@@ -145,6 +165,55 @@ public class PutElasticsearchJson extends
AbstractPutElasticsearch {
.required(true)
.build();
+ static final PropertyDescriptor MAX_BATCH_SIZE = new
PropertyDescriptor.Builder()
+ .name("Max Batch Size")
+ .description("The maximum amount of data to send in a single
Elasticsearch _bulk API request. " +
+ "For NDJSON and JSON Array modes, FlowFiles are
accumulated until this threshold is reached, then flushed. " +
+ "For Single JSON mode, this acts as a size-based safety
limit: if the accumulated FlowFiles exceed this size " +
+ "before Max FlowFiles Per Batch is reached, the request is
flushed early. " +
+ "Elasticsearch recommends 5-15 MB per bulk request for
optimal performance.")
Review Comment:
Fixed
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
