This is an automated email from the ASF dual-hosted git repository. acosentino pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/camel.git
commit 36489fad498bcef9481fd0d7f49eb819f73e26bb Author: Andrea Cosentino <anco...@gmail.com> AuthorDate: Fri Apr 2 08:24:07 2021 +0200 CAMEL-16185 - Added more docs --- .../src/main/docs/aws2-s3-component.adoc | 21 +++++++++++++++++++++ 1 file changed, 21 insertions(+) diff --git a/components/camel-aws/camel-aws2-s3/src/main/docs/aws2-s3-component.adoc b/components/camel-aws/camel-aws2-s3/src/main/docs/aws2-s3-component.adoc index b9f86e9..5d5ea99 100644 --- a/components/camel-aws/camel-aws2-s3/src/main/docs/aws2-s3-component.adoc +++ b/components/camel-aws/camel-aws2-s3/src/main/docs/aws2-s3-component.adoc @@ -553,6 +553,27 @@ This operation will return an S3Object instance related to the camelKey object i This operation will return a download link url for the file camel-key in the bucket mycamelbucket and region region +== Stream mode + +With the stream mode enabled users will be able to upload data to S3 without knowing ahead of time the dimension of the data, by leveraging multipart upload. +The upload will be completed when: the batchSize has been completed or the batchMessageNumber has been reached. +There are two possible naming strategy: progressive and random. With the progressive strategy each file will have the name composed by keyName option and a progressive counter, and eventually the file extension (if any), while with the random strategy a UUID will be added after keyName and eventually the file extension will appended. + +As an example: + +[source,java] +-------------------------------------------------------------------------------- +from(kafka("topic1").brokers("localhost:9092")) + .log("Kafka Message is: ${body}") + .to(aws2S3("camel-bucket").streamMode(true).batchMessageNumber(25).namingStrategy(AWS2S3EndpointBuilderFactory.AWSS3NamingStrategyEnum.progressive).keyName("{{kafkaTopic1}}/{{kafkaTopic1}}.txt")); + +from(kafka("topic2").brokers("localhost:9092")) + .log("Kafka Message is: ${body}") + .to(aws2S3("camel-bucket").streamMode(true).batchMessageNumber(25).namingStrategy(AWS2S3EndpointBuilderFactory.AWSS3NamingStrategyEnum.progressive).keyName("{{kafkaTopic2}}/{{kafkaTopic2}}.txt")); +-------------------------------------------------------------------------------- + +The default size for a batch is 1 Mb, but you can adjust it according to your requirements. + == Bucket Autocreation With the option `autoCreateBucket` users are able to avoid the autocreation of an S3 Bucket in case it doesn't exist. The default for this option is `true`.