This is an automated email from the ASF dual-hosted git repository.
acosentino pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/camel.git
commit 3b86ce99f22c2e06f0b772d7162c4bed7aa46421
Author: Raffaele Marcello <marcelloraffa...@gmail.com>
AuthorDate: Sat Feb 13 21:24:14 2021 +0100
CAMEL-15964 create camel-google-storage component
---
.../catalog/docs/google-storage-component.adoc | 340 ++++++++++++++++++++-
.../src/main/docs/google-storage-component.adoc | 340 ++++++++++++++++++++-
.../GoogleCloudStorageComponentConfiguration.java | 2 +-
.../google/storage/GoogleCloudStorageConsumer.java | 6 +-
.../google/storage/GoogleCloudStorageEndpoint.java | 23 +-
.../google/storage/GoogleCloudStorageProducer.java | 13 +
.../ROOT/pages/google-storage-component.adoc | 340 ++++++++++++++++++++-
7 files changed, 1038 insertions(+), 26 deletions(-)
diff --git
a/catalog/camel-catalog/src/generated/resources/org/apache/camel/catalog/docs/google-storage-component.adoc
b/catalog/camel-catalog/src/generated/resources/org/apache/camel/catalog/docs/google-storage-component.adoc
index 8e5250c..cc2010d 100644
---
a/catalog/camel-catalog/src/generated/resources/org/apache/camel/catalog/docs/google-storage-component.adoc
+++
b/catalog/camel-catalog/src/generated/resources/org/apache/camel/catalog/docs/google-storage-component.adoc
@@ -165,10 +165,342 @@ with the following path and query parameters:
// endpoint options: END
-== Message Headers
+== Usage
-TBD
+=== Message headers evaluated by the Google Storage Producer
-== Producer Endpoints
+[width="100%",cols="10%,10%,80%",options="header",]
+|=======================================================================
+|Header |Type |Description
+
+|`CamelGoogleCloudStorageBucketName` |`String` |The bucket Name which this
object will be stored or which will be used for the current operation
+
+|`CamelGoogleCloudStorageObjectName` |`String` |The object Name which will be
used for the current operation
+
+|`CamelGoogleCloudStorageDestinationBucketName` |`String` |The bucket
Destination Name which will be used for the current operation
+
+|`CamelGoogleCloudStorageDestinationObjectName` |`String` |The object
Destination Name which will be used for the current operation
+
+|`CamelGoogleCloudStorageContentLength` |`Long` |The content length of this
object.
+
+|`CamelGoogleCloudStorageContentType` |`String` |The content type of this
object.
+
+
+|`CamelGoogleCloudStorageContentDisposition` |`String` |The content
disposition of this object.
+
+|`CamelGoogleCloudStorageContentEncoding` |`String` |The content encoding of
this object.
+
+|`CamelGoogleCloudStorageContentMd5` |`String` |The md5 checksum of this
object.
+
+|`CamelGoogleCloudStorageOperation` |`String` |The operation to perform.
Permitted values are copyObject, listObjects, deleteObject, deleteBucket,
listBuckets, getObject, createDownloadLink
+
+|`CamelGoogleCloudStorageDownloadLinkExpirationTime` |`Long` |The time in
millisecond the download link will be valid.
+
+|=======================================================================
+
+=== Message headers set by the Google Storage Producer
+
+[width="100%",cols="10%,10%,80%",options="header",]
+|=======================================================================
+|Header |Type |Description
+
+|`CamelGoogleCloudStorageETag` |`String` |The ETag value for the newly
uploaded object.
+
+|=======================================================================
+
+=== Message headers set by the Google Storage Consumer
+
+[width="100%",cols="10%,10%,80%",options="header",]
+|=======================================================================
+|Header |Type |Description
+|`CamelGoogleCloudStorageBucketName` |`String` |The bucket Name which this
object will be stored or which will be used for the current operation
+|`CamelGoogleCloudStorageObjectName` |`String` |The object Name which will be
used for the current operation
+| `CamelGoogleCloudStorageCacheControl` |`String` | The Cache-Control metadata
can specify two different aspects of how data is served from Cloud Storage:
whether the data can be cached and whether the data can be transformed
+| `CamelGoogleCloudStorageComponentCount` |`String` | The component count of
this object
+| `CamelGoogleCloudStorageContentDisposition` |`String` |The content
disposition of this object.
+| `CamelGoogleCloudStorageContentEncoding` |`String` |The content encoding of
this object.
+| `CamelGoogleCloudStorageContentLanguage` |`String` | The Content-Language
metadata indicates the language(s) that the object is intended for.
+| `CamelGoogleCloudStorageContentType` |`String` |The content type of this
object.
+| `CamelGoogleCloudStorageCustomTime` |`String` | The Custom-Time metadata is
a user-specified date and time represented in the RFC 3339 format
YYYY-MM-DD'T'HH:MM:SS.SS'Z' or YYYY-MM-DD'T'HH:MM:SS'Z' when milliseconds are
zero. This metadata is typically set in order to use the DaysSinceCustomTime
condition in Object Lifecycle Management.
+| `CamelGoogleCloudStorageCrc32cHex` |`String` | The CRC32c of the object
+| `CamelGoogleCloudStorageETag` |`String` | The ETag for the Object.
+| `CamelGoogleCloudStorageGeneration` |`String` | Is the generation number of
the object for which you are retrieving information.
+| `CamelGoogleCloudStorageBlobId` |`String` | The blob id of the object
+| `CamelGoogleCloudStorageKmsKeyName` |`String` | The KMS key name
+| `CamelGoogleCloudStorageContentMd5` |`String` |The md5 checksum of this
object.
+| `CamelGoogleCloudStorageMediaLink` |`String` | The media link
+| `CamelGoogleCloudStorageMetageneration` | `String` | The metageneration of
the object
+| `CamelGoogleCloudStorageContentLength` |`Long` |The content length of this
object.
+| `CamelGoogleCloudStorageStorageClass` | `String` | The storage class of the
object
+| `CamelGoogleCloudStorageCreateTime` |`String` | The creation time of the
object
+| `CamelGoogleCloudStorageLastUpdate` |`String` | The last update of the object
+|=======================================================================
+
+=== Google Storage Producer operations
+
+Google Storage component provides the following operation on the producer side:
+
+- copyObject
+- listObjects
+- deleteObject
+- deleteBucket
+- listBuckets
+- getObject
+- createDownloadLink
+
+If you don't specify an operation explicitly the producer will a file upload.
+
+=== Advanced component configuration
+
+If you need to have more control over the `storageClient` instance
configuration, you can create your own instance and refer to it in your Camel
google-storage component configuration:
+
+[source,java]
+--------------------------------------------------------------------------------
+from("google-storage://myCamelBucket?storageClient=#client")
+.to("mock:result");
+--------------------------------------------------------------------------------
+
+=== Google Storage Producer Operation examples
+
+- File Upload: This operation will upload a file to the Google Storage based
on the body content
+
+[source,java]
+--------------------------------------------------------------------------------
+//upload a file
+byte[] payload = "Camel rocks!".getBytes();
+ByteArrayInputStream bais = new ByteArrayInputStream(payload);
+from("direct:start")
+.process( exchange -> {
+ exchange.getIn().setHeader(GoogleCloudStorageConstants.OBJECT_NAME,
"camel.txt");
+ exchange.getIn().setBody(bais);
+})
+.to("google-storage://myCamelBucket?serviceAccountKey=/home/user/Downloads/my-key.json")
+.log("uploaded file object:${header.CamelGoogleCloudStorageObjectName},
body:${body}");
+--------------------------------------------------------------------------------
+
+This operation will upload the file camel.txt with the content "Camel rocks!"
in the myCamelBucket bucket
+
+- CopyObject: this operation copy an object from one bucket to a different one
+
+[source,java]
+--------------------------------------------------------------------------------
+ from("direct:start").process(new Processor() {
+
+ @Override
+ public void process(Exchange exchange) throws Exception {
+ exchange.getIn().setHeader(S3Constants.BUCKET_DESTINATION_NAME,
"camelDestinationBucket");
+ exchange.getIn().setHeader(S3Constants.KEY, "camelKey");
+ exchange.getIn().setHeader(S3Constants.DESTINATION_KEY,
"camelDestinationKey");
+ }
+ })
+
.to("google-storage://mycamelbucket?amazonS3Client=#amazonS3Client&operation=copyObject")
+ .to("mock:result");
+--------------------------------------------------------------------------------
+
+This operation will copy the object with the name expressed in the header
camelDestinationKey to the camelDestinationBucket bucket, from the bucket
mycamelbucket.
+
+- DeleteObject: this operation deletes an object from a bucket
+
+[source,java]
+--------------------------------------------------------------------------------
+ from("direct:start").process(new Processor() {
+
+ @Override
+ public void process(Exchange exchange) throws Exception {
+ exchange.getIn().setHeader(S3Constants.KEY, "camelKey");
+ }
+ })
+
.to("google-storage://mycamelbucket?amazonS3Client=#amazonS3Client&operation=deleteObject")
+ .to("mock:result");
+--------------------------------------------------------------------------------
+
+This operation will delete the object camelKey from the bucket mycamelbucket.
+
+- ListBuckets: this operation list the buckets for this account in this region
+
+[source,java]
+--------------------------------------------------------------------------------
+ from("direct:start")
+
.to("google-storage://mycamelbucket?amazonS3Client=#amazonS3Client&operation=listBuckets")
+ .to("mock:result");
+--------------------------------------------------------------------------------
+
+This operation will list the buckets for this account
+
+- DeleteBucket: this operation delete the bucket specified as URI parameter or
header
+
+[source,java]
+--------------------------------------------------------------------------------
+ from("direct:start")
+
.to("google-storage://mycamelbucket?amazonS3Client=#amazonS3Client&operation=deleteBucket")
+ .to("mock:result");
+--------------------------------------------------------------------------------
+
+This operation will delete the bucket mycamelbucket
+
+- ListObjects: this operation list object in a specific bucket
+
+[source,java]
+--------------------------------------------------------------------------------
+ from("direct:start")
+
.to("google-storage://mycamelbucket?amazonS3Client=#amazonS3Client&operation=listObjects")
+ .to("mock:result");
+--------------------------------------------------------------------------------
+
+This operation will list the objects in the mycamelbucket bucket
+
+- GetObject: this operation get a single object in a specific bucket
+
+[source,java]
+--------------------------------------------------------------------------------
+ from("direct:start").process(new Processor() {
+
+ @Override
+ public void process(Exchange exchange) throws Exception {
+ exchange.getIn().setHeader(S3Constants.KEY, "camelKey");
+ }
+ })
+
.to("google-storage://mycamelbucket?amazonS3Client=#amazonS3Client&operation=getObject")
+ .to("mock:result");
+--------------------------------------------------------------------------------
+
+This operation will return an S3Object instance related to the camelKey object
in mycamelbucket bucket.
+
+- GetObjectRange: this operation get a single object range in a specific bucket
+
+[source,java]
+--------------------------------------------------------------------------------
+ from("direct:start").process(new Processor() {
+
+ @Override
+ public void process(Exchange exchange) throws Exception {
+ exchange.getIn().setHeader(S3Constants.KEY, "camelKey");
+ exchange.getIn().setHeader(S3Constants.RANGE_START, "0");
+ exchange.getIn().setHeader(S3Constants.RANGE_END, "9");
+ }
+ })
+
.to("google-storage://mycamelbucket?amazonS3Client=#amazonS3Client&operation=getObjectRange")
+ .to("mock:result");
+--------------------------------------------------------------------------------
+
+This operation will return an S3Object instance related to the camelKey object
in mycamelbucket bucket, containing a the bytes from 0 to 9.
+
+- CreateDownloadLink: this operation will return a download link through S3
Presigner
+
+[source,java]
+--------------------------------------------------------------------------------
+ from("direct:start").process(new Processor() {
+
+ @Override
+ public void process(Exchange exchange) throws Exception {
+ exchange.getIn().setHeader(S3Constants.KEY, "camelKey");
+ }
+ })
+
.to("google-storage://mycamelbucket?accessKey=xxx&secretKey=yyy®ion=region&operation=createDownloadLink")
+ .to("mock:result");
+--------------------------------------------------------------------------------
+
+This operation will return a download link url for the file camel-key in the
bucket mycamelbucket and region region
+
+== Bucket Autocreation
+
+With the option `autoCreateBucket` users are able to avoid the autocreation of
an S3 Bucket in case it doesn't exist. The default for this option is `true`.
+If set to false any operation on a not-existent bucket in AWS won't be
successful and an error will be returned.
+
+== Moving stuff between a bucket and another bucket
+
+Some users like to consume stuff from a bucket and move the content in a
different one without using the copyObject feature of this component.
+If this is case for you, don't forget to remove the bucketName header from the
incoming exchange of the consumer, otherwise the file will be always
overwritten on the same
+original bucket.
+
+== MoveAfterRead consumer option
+
+In addition to deleteAfterRead it has been added another option,
moveAfterRead. With this option enabled the consumed object will be moved to a
target destinationBucket instead of being only deleted.
+This will require specifying the destinationBucket option. As example:
+
+[source,java]
+--------------------------------------------------------------------------------
+
from("google-storage://mycamelbucket?amazonS3Client=#amazonS3Client&moveAfterRead=true&destinationBucket=myothercamelbucket")
+ .to("mock:result");
+--------------------------------------------------------------------------------
+
+In this case the objects consumed will be moved to myothercamelbucket bucket
and deleted from the original one (because of deleteAfterRead set to true as
default).
+
+You have also the possibility of using a key prefix/suffix while moving the
file to a different bucket. The options are destinationBucketPrefix and
destinationBucketSuffix.
+
+Taking the above example, you could do something like:
+
+[source,java]
+--------------------------------------------------------------------------------
+
from("google-storage://mycamelbucket?amazonS3Client=#amazonS3Client&moveAfterRead=true&destinationBucket=myothercamelbucket&destinationBucketPrefix=RAW(pre-)&destinationBucketSuffix=RAW(-suff)")
+ .to("mock:result");
+--------------------------------------------------------------------------------
+
+In this case the objects consumed will be moved to myothercamelbucket bucket
and deleted from the original one (because of deleteAfterRead set to true as
default).
+
+So if the file name is test, in the myothercamelbucket you should see a file
called pre-test-suff.
+
+== Using customer key as encryption
+
+We introduced also the customer key support (an alternative of using KMS). The
following code shows an example.
+
+[source,java]
+--------------------------------------------------------------------------------
+String key = UUID.randomUUID().toString();
+byte[] secretKey = generateSecretKey();
+String b64Key = Base64.getEncoder().encodeToString(secretKey);
+String b64KeyMd5 = Md5Utils.md5AsBase64(secretKey);
+
+String awsEndpoint =
"google-storage://mycamel?autoCreateBucket=false&useCustomerKey=true&customerKeyId=RAW("
+ b64Key + ")&customerKeyMD5=RAW(" + b64KeyMd5 + ")&customerAlgorithm=" +
AES256.name();
+
+from("direct:putObject")
+ .setHeader(AWS2S3Constants.KEY, constant("test.txt"))
+ .setBody(constant("Test"))
+ .to(awsEndpoint);
+--------------------------------------------------------------------------------
+
+== Using a POJO as body
+
+Sometimes build an AWS Request can be complex, because of multiple options. We
introduce the possibility to use a POJO as body.
+In AWS S3 there are multiple operations you can submit, as an example for List
brokers request, you can do something like:
+
+------------------------------------------------------------------------------------------------------
+from("direct:google-storage")
+ .setBody(ListObjectsRequest.builder().bucket(bucketName).build())
+
.to("google-storage://test?amazonS3Client=#amazonS3Client&operation=listObjects&pojoRequest=true")
+------------------------------------------------------------------------------------------------------
+
+In this way you'll pass the request directly without the need of passing
headers and options specifically related to this operation.
+
+== Create S3 client and add component to registry
+Sometimes you would want to perform some advanced configuration using
AWS2S3Configuration which also allows to set the S3 client.
+You can create and set the S3 client in the component configuration as shown
in the following example
+
+[source,java]
+--------------------------------------------------------------------------------
+String awsBucketAccessKey = "your_access_key";
+String awsBucketSecretKey = "your_secret_key";
+
+S3Client s3Client =
S3Client.builder().credentialsProvider(StaticCredentialsProvider.create(AwsBasicCredentials.create(awsBucketAccessKey,
awsBucketSecretKey)))
+ .region(Region.US_EAST_1).build();
+
+AWS2S3Configuration configuration = new AWS2S3Configuration();
+configuration.setAmazonS3Client(s3Client);
+configuration.setAutoDiscoverClient(true);
+configuration.setBucketName("s3bucket2020");
+configuration.setRegion("us-east-1");
+--------------------------------------------------------------------------------
+
+Now you can configure the S3 component (using the configuration object created
above) and add it to the registry in the
+configure method before initialization of routes.
+
+[source,java]
+--------------------------------------------------------------------------------
+AWS2S3Component s3Component = new AWS2S3Component(getContext());
+s3Component.setConfiguration(configuration);
+s3Component.setLazyStartProducer(true);
+camelContext.addComponent("google-storage", s3Component);
+--------------------------------------------------------------------------------
+
+Now your component will be used for all the operations implemented in camel
routes.
-TBD
diff --git
a/components/camel-google-storage/src/main/docs/google-storage-component.adoc
b/components/camel-google-storage/src/main/docs/google-storage-component.adoc
index 8e5250c..6ef1b8e 100644
---
a/components/camel-google-storage/src/main/docs/google-storage-component.adoc
+++
b/components/camel-google-storage/src/main/docs/google-storage-component.adoc
@@ -165,10 +165,342 @@ with the following path and query parameters:
// endpoint options: END
-== Message Headers
+== Usage
-TBD
+=== Message headers evaluated by the Google Storage Producer
-== Producer Endpoints
+[width="100%",cols="10%,10%,80%",options="header",]
+|=======================================================================
+|Header |Type |Description
+
+|`CamelGoogleCloudStorageBucketName` |`String` |The bucket Name which this
object will be stored or which will be used for the current operation
+
+|`CamelGoogleCloudStorageObjectName` |`String` |The object Name which will be
used for the current operation
+
+|`CamelGoogleCloudStorageDestinationBucketName` |`String` |The bucket
Destination Name which will be used for the current operation
+
+|`CamelGoogleCloudStorageDestinationObjectName` |`String` |The object
Destination Name which will be used for the current operation
+
+|`CamelGoogleCloudStorageContentLength` |`Long` |The content length of this
object.
+
+|`CamelGoogleCloudStorageContentType` |`String` |The content type of this
object.
+
+
+|`CamelGoogleCloudStorageContentDisposition` |`String` |The content
disposition of this object.
+
+|`CamelGoogleCloudStorageContentEncoding` |`String` |The content encoding of
this object.
+
+|`CamelGoogleCloudStorageContentMd5` |`String` |The md5 checksum of this
object.
+
+|`CamelGoogleCloudStorageOperation` |`String` |The operation to perform.
Permitted values are copyObject, listObjects, deleteObject, deleteBucket,
listBuckets, getObject, createDownloadLink
+
+|`CamelGoogleCloudStorageDownloadLinkExpirationTime` |`Long` |The time in
millisecond the download link will be valid.
+
+|=======================================================================
+
+=== Message headers set by the Google Storage Producer
+
+[width="100%",cols="10%,10%,80%",options="header",]
+|=======================================================================
+|Header |Type |Description
+
+|`CamelGoogleCloudStorageETag` |`String` |The ETag value for the newly
uploaded object.
+
+|=======================================================================
+
+=== Message headers set by the Google Storage Consumer
+
+[width="100%",cols="10%,10%,80%",options="header",]
+|=======================================================================
+|Header |Type |Description
+|`CamelGoogleCloudStorageBucketName` |`String` |The bucket Name which this
object will be stored or which will be used for the current operation
+|`CamelGoogleCloudStorageObjectName` |`String` |The object Name which will be
used for the current operation
+| `CamelGoogleCloudStorageCacheControl` |`String` | The Cache-Control metadata
can specify two different aspects of how data is served from Cloud Storage:
whether the data can be cached and whether the data can be transformed
+| `CamelGoogleCloudStorageComponentCount` |`String` | The component count of
this object
+| `CamelGoogleCloudStorageContentDisposition` |`String` |The content
disposition of this object.
+| `CamelGoogleCloudStorageContentEncoding` |`String` |The content encoding of
this object.
+| `CamelGoogleCloudStorageContentLanguage` |`String` | The Content-Language
metadata indicates the language(s) that the object is intended for.
+| `CamelGoogleCloudStorageContentType` |`String` |The content type of this
object.
+| `CamelGoogleCloudStorageCustomTime` |`String` | The Custom-Time metadata is
a user-specified date and time represented in the RFC 3339 format
YYYY-MM-DD'T'HH:MM:SS.SS'Z' or YYYY-MM-DD'T'HH:MM:SS'Z' when milliseconds are
zero. This metadata is typically set in order to use the DaysSinceCustomTime
condition in Object Lifecycle Management.
+| `CamelGoogleCloudStorageCrc32cHex` |`String` | The CRC32c of the object
+| `CamelGoogleCloudStorageETag` |`String` | The ETag for the Object.
+| `CamelGoogleCloudStorageGeneration` |`String` | Is the generation number of
the object for which you are retrieving information.
+| `CamelGoogleCloudStorageBlobId` |`String` | The blob id of the object
+| `CamelGoogleCloudStorageKmsKeyName` |`String` | The KMS key name
+| `CamelGoogleCloudStorageContentMd5` |`String` |The md5 checksum of this
object.
+| `CamelGoogleCloudStorageMediaLink` |`String` | The media link
+| `CamelGoogleCloudStorageMetageneration` | `String` | The metageneration of
the object
+| `CamelGoogleCloudStorageContentLength` |`Long` |The content length of this
object.
+| `CamelGoogleCloudStorageStorageClass` | `String` | The storage class of the
object
+| `CamelGoogleCloudStorageCreateTime` |`String` | The creation time of the
object
+| `CamelGoogleCloudStorageLastUpdate` |`String` | The last update of the object
+|=======================================================================
+
+=== Google Storage Producer operations
+
+Google Storage component provides the following operation on the producer side:
+
+- copyObject
+- listObjects
+- deleteObject
+- deleteBucket
+- listBuckets
+- getObject
+- createDownloadLink
+
+If you don't specify an operation explicitly the producer will a file upload.
+
+=== Advanced component configuration
+
+If you need to have more control over the `storageClient` instance
configuration, you can create your own instance and refer to it in your Camel
google-storage component configuration:
+
+[source,java]
+--------------------------------------------------------------------------------
+from("google-storage://myCamelBucket?storageClient=#client")
+.to("mock:result");
+--------------------------------------------------------------------------------
+
+=== Google Storage Producer Operation examples
+
+- File Upload: This operation will upload a file to the Google Storage based
on the body content
+
+[source,java]
+--------------------------------------------------------------------------------
+//upload a file
+byte[] payload = "Camel rocks!".getBytes();
+ByteArrayInputStream bais = new ByteArrayInputStream(payload);
+from("direct:start")
+.process( exchange -> {
+ exchange.getIn().setHeader(GoogleCloudStorageConstants.OBJECT_NAME,
"camel.txt");
+ exchange.getIn().setBody(bais);
+})
+.to("google-storage://myCamelBucket?serviceAccountKey=/home/user/Downloads/my-key.json")
+.log("uploaded file object:${header.CamelGoogleCloudStorageObjectName},
body:${body}");
+--------------------------------------------------------------------------------
+
+This operation will upload the file camel.txt with the content "Camel rocks!"
in the myCamelBucket bucket
+
+- CopyObject: this operation copy an object from one bucket to a different one
+
+[source,java]
+--------------------------------------------------------------------------------
+ from("direct:start").process( exchange -> {
+ exchange.getIn().setHeader(GoogleCloudStorageConstants.OPERATION,
GoogleCloudStorageComponentOperations.copyObject);
+ exchange.getIn().setHeader(GoogleCloudStorageConstants.OBJECT_NAME,
"camel.txt" );
+
exchange.getIn().setHeader(GoogleCloudStorageConstants.DESTINATION_BUCKET_NAME,
"myCamelBucket_dest");
+
exchange.getIn().setHeader(GoogleCloudStorageConstants.DESTINATION_OBJECT_NAME,
"camel_copy.txt");
+ })
+
.to("google-storage://myCamelBucket?serviceAccountKey=/home/user/Downloads/my-key.json")
+ .to("mock:result");
+--------------------------------------------------------------------------------
+
+This operation will copy the object with the name expressed in the header
DESTINATION_OBJECT_NAME to the DESTINATION_BUCKET_NAME bucket, from the bucket
myCamelBucket.
+
+- DeleteObject: this operation deletes an object from a bucket
+
+[source,java]
+--------------------------------------------------------------------------------
+ from("direct:start").process( exchange -> {
+ exchange.getIn().setHeader(GoogleCloudStorageConstants.OPERATION,
GoogleCloudStorageComponentOperations.deleteObject);
+ exchange.getIn().setHeader(GoogleCloudStorageConstants.OBJECT_NAME,
"camel.txt" );
+ })
+
.to("google-storage://myCamelBucket?serviceAccountKey=/home/user/Downloads/my-key.json")
+ .to("mock:result");
+--------------------------------------------------------------------------------
+
+This operation will delete the object from the bucket myCamelBucket.
+
+- ListBuckets: this operation list the buckets for this account in this region
+
+[source,java]
+--------------------------------------------------------------------------------
+from("direct:start").process( exchange -> {
+ exchange.getIn().setHeader(GoogleCloudStorageConstants.OPERATION,
GoogleCloudStorageComponentOperations.listBuckets);
+})
+.to("google-storage://myCamelBucket?serviceAccountKey=/home/user/Downloads/my-key.json")
+.to("mock:result");
+--------------------------------------------------------------------------------
+
+This operation will list the buckets for this account
+
+- DeleteBucket: this operation delete the bucket specified as URI parameter or
header
+
+[source,java]
+--------------------------------------------------------------------------------
+from("direct:start").process( exchange -> {
+ exchange.getIn().setHeader(GoogleCloudStorageConstants.OPERATION,
GoogleCloudStorageComponentOperations.listObjects);
+})
+.to("google-storage://myCamelBucket?serviceAccountKey=/home/user/Downloads/my-key.json")
+.to("mock:result");
+--------------------------------------------------------------------------------
+
+This operation will delete the bucket mycamelbucket
+
+- ListObjects: this operation list object in a specific bucket
+
+[source,java]
+--------------------------------------------------------------------------------
+from("direct:start").process( exchange -> {
+ exchange.getIn().setHeader(GoogleCloudStorageConstants.OPERATION,
GoogleCloudStorageComponentOperations.listObjects);
+})
+.to("google-storage://myCamelBucket?serviceAccountKey=/home/user/Downloads/my-key.json")
+.to("mock:result");
+--------------------------------------------------------------------------------
+
+This operation will list the objects in the mycamelbucket bucket
+
+- GetObject: this operation get a single object in a specific bucket
+
+[source,java]
+--------------------------------------------------------------------------------
+ from("direct:start").process(new Processor() {
+
+ @Override
+ public void process(Exchange exchange) throws Exception {
+ exchange.getIn().setHeader(S3Constants.KEY, "camelKey");
+ }
+ })
+
.to("google-storage://mycamelbucket?amazonS3Client=#amazonS3Client&operation=getObject")
+ .to("mock:result");
+--------------------------------------------------------------------------------
+
+This operation will return an S3Object instance related to the camelKey object
in mycamelbucket bucket.
+
+- GetObjectRange: this operation get a single object range in a specific bucket
+
+[source,java]
+--------------------------------------------------------------------------------
+ from("direct:start").process(new Processor() {
+
+ @Override
+ public void process(Exchange exchange) throws Exception {
+ exchange.getIn().setHeader(S3Constants.KEY, "camelKey");
+ exchange.getIn().setHeader(S3Constants.RANGE_START, "0");
+ exchange.getIn().setHeader(S3Constants.RANGE_END, "9");
+ }
+ })
+
.to("google-storage://mycamelbucket?amazonS3Client=#amazonS3Client&operation=getObjectRange")
+ .to("mock:result");
+--------------------------------------------------------------------------------
+
+This operation will return an S3Object instance related to the camelKey object
in mycamelbucket bucket, containing a the bytes from 0 to 9.
+
+- CreateDownloadLink: this operation will return a download link through S3
Presigner
+
+[source,java]
+--------------------------------------------------------------------------------
+ from("direct:start").process(new Processor() {
+
+ @Override
+ public void process(Exchange exchange) throws Exception {
+ exchange.getIn().setHeader(S3Constants.KEY, "camelKey");
+ }
+ })
+
.to("google-storage://mycamelbucket?accessKey=xxx&secretKey=yyy®ion=region&operation=createDownloadLink")
+ .to("mock:result");
+--------------------------------------------------------------------------------
+
+This operation will return a download link url for the file camel-key in the
bucket mycamelbucket and region region
+
+== Bucket Autocreation
+
+With the option `autoCreateBucket` users are able to avoid the autocreation of
an S3 Bucket in case it doesn't exist. The default for this option is `true`.
+If set to false any operation on a not-existent bucket in AWS won't be
successful and an error will be returned.
+
+== Moving stuff between a bucket and another bucket
+
+Some users like to consume stuff from a bucket and move the content in a
different one without using the copyObject feature of this component.
+If this is case for you, don't forget to remove the bucketName header from the
incoming exchange of the consumer, otherwise the file will be always
overwritten on the same
+original bucket.
+
+== MoveAfterRead consumer option
+
+In addition to deleteAfterRead it has been added another option,
moveAfterRead. With this option enabled the consumed object will be moved to a
target destinationBucket instead of being only deleted.
+This will require specifying the destinationBucket option. As example:
+
+[source,java]
+--------------------------------------------------------------------------------
+
from("google-storage://mycamelbucket?amazonS3Client=#amazonS3Client&moveAfterRead=true&destinationBucket=myothercamelbucket")
+ .to("mock:result");
+--------------------------------------------------------------------------------
+
+In this case the objects consumed will be moved to myothercamelbucket bucket
and deleted from the original one (because of deleteAfterRead set to true as
default).
+
+You have also the possibility of using a key prefix/suffix while moving the
file to a different bucket. The options are destinationBucketPrefix and
destinationBucketSuffix.
+
+Taking the above example, you could do something like:
+
+[source,java]
+--------------------------------------------------------------------------------
+
from("google-storage://mycamelbucket?amazonS3Client=#amazonS3Client&moveAfterRead=true&destinationBucket=myothercamelbucket&destinationBucketPrefix=RAW(pre-)&destinationBucketSuffix=RAW(-suff)")
+ .to("mock:result");
+--------------------------------------------------------------------------------
+
+In this case the objects consumed will be moved to myothercamelbucket bucket
and deleted from the original one (because of deleteAfterRead set to true as
default).
+
+So if the file name is test, in the myothercamelbucket you should see a file
called pre-test-suff.
+
+== Using customer key as encryption
+
+We introduced also the customer key support (an alternative of using KMS). The
following code shows an example.
+
+[source,java]
+--------------------------------------------------------------------------------
+String key = UUID.randomUUID().toString();
+byte[] secretKey = generateSecretKey();
+String b64Key = Base64.getEncoder().encodeToString(secretKey);
+String b64KeyMd5 = Md5Utils.md5AsBase64(secretKey);
+
+String awsEndpoint =
"google-storage://mycamel?autoCreateBucket=false&useCustomerKey=true&customerKeyId=RAW("
+ b64Key + ")&customerKeyMD5=RAW(" + b64KeyMd5 + ")&customerAlgorithm=" +
AES256.name();
+
+from("direct:putObject")
+ .setHeader(AWS2S3Constants.KEY, constant("test.txt"))
+ .setBody(constant("Test"))
+ .to(awsEndpoint);
+--------------------------------------------------------------------------------
+
+== Using a POJO as body
+
+Sometimes build an AWS Request can be complex, because of multiple options. We
introduce the possibility to use a POJO as body.
+In AWS S3 there are multiple operations you can submit, as an example for List
brokers request, you can do something like:
+
+------------------------------------------------------------------------------------------------------
+from("direct:google-storage")
+ .setBody(ListObjectsRequest.builder().bucket(bucketName).build())
+
.to("google-storage://test?amazonS3Client=#amazonS3Client&operation=listObjects&pojoRequest=true")
+------------------------------------------------------------------------------------------------------
+
+In this way you'll pass the request directly without the need of passing
headers and options specifically related to this operation.
+
+== Create S3 client and add component to registry
+Sometimes you would want to perform some advanced configuration using
AWS2S3Configuration which also allows to set the S3 client.
+You can create and set the S3 client in the component configuration as shown
in the following example
+
+[source,java]
+--------------------------------------------------------------------------------
+String awsBucketAccessKey = "your_access_key";
+String awsBucketSecretKey = "your_secret_key";
+
+S3Client s3Client =
S3Client.builder().credentialsProvider(StaticCredentialsProvider.create(AwsBasicCredentials.create(awsBucketAccessKey,
awsBucketSecretKey)))
+ .region(Region.US_EAST_1).build();
+
+AWS2S3Configuration configuration = new AWS2S3Configuration();
+configuration.setAmazonS3Client(s3Client);
+configuration.setAutoDiscoverClient(true);
+configuration.setBucketName("s3bucket2020");
+configuration.setRegion("us-east-1");
+--------------------------------------------------------------------------------
+
+Now you can configure the S3 component (using the configuration object created
above) and add it to the registry in the
+configure method before initialization of routes.
+
+[source,java]
+--------------------------------------------------------------------------------
+AWS2S3Component s3Component = new AWS2S3Component(getContext());
+s3Component.setConfiguration(configuration);
+s3Component.setLazyStartProducer(true);
+camelContext.addComponent("google-storage", s3Component);
+--------------------------------------------------------------------------------
+
+Now your component will be used for all the operations implemented in camel
routes.
-TBD
diff --git
a/components/camel-google-storage/src/main/java/org/apache/camel/component/google/storage/GoogleCloudStorageComponentConfiguration.java
b/components/camel-google-storage/src/main/java/org/apache/camel/component/google/storage/GoogleCloudStorageComponentConfiguration.java
index 737aeab..7129bdb 100644
---
a/components/camel-google-storage/src/main/java/org/apache/camel/component/google/storage/GoogleCloudStorageComponentConfiguration.java
+++
b/components/camel-google-storage/src/main/java/org/apache/camel/component/google/storage/GoogleCloudStorageComponentConfiguration.java
@@ -43,7 +43,7 @@ public class GoogleCloudStorageComponentConfiguration
implements Cloneable {
@UriParam(label = "common", defaultValue = "US-EAST1",
description = "The Cloud Storage location to use when creating
the new buckets")
- private String storageLocation;
+ private String storageLocation = "US-EAST1";
@UriParam(label = "common", defaultValue = "STANDARD",
description = "The Cloud Storage class to use when creating the
new buckets")
diff --git
a/components/camel-google-storage/src/main/java/org/apache/camel/component/google/storage/GoogleCloudStorageConsumer.java
b/components/camel-google-storage/src/main/java/org/apache/camel/component/google/storage/GoogleCloudStorageConsumer.java
index 42e7035..6fa173e 100644
---
a/components/camel-google-storage/src/main/java/org/apache/camel/component/google/storage/GoogleCloudStorageConsumer.java
+++
b/components/camel-google-storage/src/main/java/org/apache/camel/component/google/storage/GoogleCloudStorageConsumer.java
@@ -23,7 +23,6 @@ import java.util.Queue;
import com.google.cloud.storage.Blob;
import com.google.cloud.storage.BlobId;
import com.google.cloud.storage.Bucket;
-import com.google.cloud.storage.BucketInfo;
import com.google.cloud.storage.CopyWriter;
import com.google.cloud.storage.Storage;
import com.google.cloud.storage.Storage.CopyRequest;
@@ -63,9 +62,8 @@ public class GoogleCloudStorageConsumer extends
ScheduledBatchPollingConsumer {
LOG.trace("Destination Bucket [{}] doesn't exist yet",
getConfiguration().getDestinationBucket());
if (getConfiguration().isAutoCreateBucket()) {
// creates the new bucket because it doesn't exist yet
- BucketInfo bucketInfo =
BucketInfo.newBuilder(getConfiguration().getDestinationBucket()).build();
- bucket = getStorageClient().create(bucketInfo);
- LOG.trace("Destination Bucket created", bucket.getName());
+
GoogleCloudStorageEndpoint.createNewBucket(getConfiguration().getDestinationBucket(),
getConfiguration(),
+ getStorageClient());
}
}
diff --git
a/components/camel-google-storage/src/main/java/org/apache/camel/component/google/storage/GoogleCloudStorageEndpoint.java
b/components/camel-google-storage/src/main/java/org/apache/camel/component/google/storage/GoogleCloudStorageEndpoint.java
index 677cd75..043b414 100644
---
a/components/camel-google-storage/src/main/java/org/apache/camel/component/google/storage/GoogleCloudStorageEndpoint.java
+++
b/components/camel-google-storage/src/main/java/org/apache/camel/component/google/storage/GoogleCloudStorageEndpoint.java
@@ -95,15 +95,7 @@ public class GoogleCloudStorageEndpoint extends
ScheduledPollEndpoint {
return;
} else {
// creates the new bucket because it doesn't exist yet
- final String location = configuration.getStorageLocation();
- final StorageClass storageClass =
configuration.getStorageClass();
-
- Builder bucketBuilder =
BucketInfo.newBuilder(configuration.getBucketName())
- .setStorageClass(storageClass)
- .setLocation(location);
- BucketInfo bucketInfo = bucketBuilder.build();
- bucket = storageClient.create(bucketInfo);
- LOG.trace("Bucket [{}] has been created",
bucket.getName());
+ createNewBucket(configuration.getBucketName(),
configuration, this.storageClient);
}
} catch (Exception e) {
LOG.error("Error - autocreatebucket", e);
@@ -112,6 +104,19 @@ public class GoogleCloudStorageEndpoint extends
ScheduledPollEndpoint {
}
}
+ public static Bucket createNewBucket(String bucketName,
GoogleCloudStorageComponentConfiguration conf, Storage storage) {
+ final String location = conf.getStorageLocation();
+ final StorageClass storageClass = conf.getStorageClass();
+
+ Builder bucketBuilder = BucketInfo.newBuilder(bucketName)
+ .setStorageClass(storageClass)
+ .setLocation(location);
+ BucketInfo bucketInfo = bucketBuilder.build();
+ Bucket bucket = storage.create(bucketInfo);
+ LOG.trace("Bucket [{}] has been created", bucket.getName());
+ return bucket;
+ }
+
public GoogleCloudStorageComponentConfiguration getConfiguration() {
return configuration;
}
diff --git
a/components/camel-google-storage/src/main/java/org/apache/camel/component/google/storage/GoogleCloudStorageProducer.java
b/components/camel-google-storage/src/main/java/org/apache/camel/component/google/storage/GoogleCloudStorageProducer.java
index 9cf73b5..b4a3a51 100644
---
a/components/camel-google-storage/src/main/java/org/apache/camel/component/google/storage/GoogleCloudStorageProducer.java
+++
b/components/camel-google-storage/src/main/java/org/apache/camel/component/google/storage/GoogleCloudStorageProducer.java
@@ -241,6 +241,19 @@ public class GoogleCloudStorageProducer extends
DefaultProducer {
throw new IllegalArgumentException("Destination Key must be
specified for copyObject Operation");
}
+ //if bucket does not exsist, create it
+ Bucket destinationBucketCheck = storage.get(bucketNameDestination);
+ if (destinationBucketCheck != null) {
+ LOG.trace("destinationBucketCheck [{}] already exists",
destinationBucketCheck.getName());
+ } else {
+ LOG.trace("Destination Bucket [{}] doesn't exist yet",
bucketNameDestination);
+ if (getConfiguration().isAutoCreateBucket()) {
+ // creates the new bucket because it doesn't exist yet
+ destinationBucketCheck
+ =
GoogleCloudStorageEndpoint.createNewBucket(bucketNameDestination,
getConfiguration(), storage);
+ }
+ }
+
BlobId sourceBlobId = BlobId.of(bucketName, objectName);
BlobId targetBlobId = BlobId.of(bucketNameDestination,
destinationObjectName);
CopyRequest request = CopyRequest.of(sourceBlobId, targetBlobId);
diff --git a/docs/components/modules/ROOT/pages/google-storage-component.adoc
b/docs/components/modules/ROOT/pages/google-storage-component.adoc
index 9b83c19..c078d71 100644
--- a/docs/components/modules/ROOT/pages/google-storage-component.adoc
+++ b/docs/components/modules/ROOT/pages/google-storage-component.adoc
@@ -167,10 +167,342 @@ with the following path and query parameters:
// endpoint options: END
-== Message Headers
+== Usage
-TBD
+=== Message headers evaluated by the Google Storage Producer
-== Producer Endpoints
+[width="100%",cols="10%,10%,80%",options="header",]
+|=======================================================================
+|Header |Type |Description
+
+|`CamelGoogleCloudStorageBucketName` |`String` |The bucket Name which this
object will be stored or which will be used for the current operation
+
+|`CamelGoogleCloudStorageObjectName` |`String` |The object Name which will be
used for the current operation
+
+|`CamelGoogleCloudStorageDestinationBucketName` |`String` |The bucket
Destination Name which will be used for the current operation
+
+|`CamelGoogleCloudStorageDestinationObjectName` |`String` |The object
Destination Name which will be used for the current operation
+
+|`CamelGoogleCloudStorageContentLength` |`Long` |The content length of this
object.
+
+|`CamelGoogleCloudStorageContentType` |`String` |The content type of this
object.
+
+
+|`CamelGoogleCloudStorageContentDisposition` |`String` |The content
disposition of this object.
+
+|`CamelGoogleCloudStorageContentEncoding` |`String` |The content encoding of
this object.
+
+|`CamelGoogleCloudStorageContentMd5` |`String` |The md5 checksum of this
object.
+
+|`CamelGoogleCloudStorageOperation` |`String` |The operation to perform.
Permitted values are copyObject, listObjects, deleteObject, deleteBucket,
listBuckets, getObject, createDownloadLink
+
+|`CamelGoogleCloudStorageDownloadLinkExpirationTime` |`Long` |The time in
millisecond the download link will be valid.
+
+|=======================================================================
+
+=== Message headers set by the Google Storage Producer
+
+[width="100%",cols="10%,10%,80%",options="header",]
+|=======================================================================
+|Header |Type |Description
+
+|`CamelGoogleCloudStorageETag` |`String` |The ETag value for the newly
uploaded object.
+
+|=======================================================================
+
+=== Message headers set by the Google Storage Consumer
+
+[width="100%",cols="10%,10%,80%",options="header",]
+|=======================================================================
+|Header |Type |Description
+|`CamelGoogleCloudStorageBucketName` |`String` |The bucket Name which this
object will be stored or which will be used for the current operation
+|`CamelGoogleCloudStorageObjectName` |`String` |The object Name which will be
used for the current operation
+| `CamelGoogleCloudStorageCacheControl` |`String` | The Cache-Control metadata
can specify two different aspects of how data is served from Cloud Storage:
whether the data can be cached and whether the data can be transformed
+| `CamelGoogleCloudStorageComponentCount` |`String` | The component count of
this object
+| `CamelGoogleCloudStorageContentDisposition` |`String` |The content
disposition of this object.
+| `CamelGoogleCloudStorageContentEncoding` |`String` |The content encoding of
this object.
+| `CamelGoogleCloudStorageContentLanguage` |`String` | The Content-Language
metadata indicates the language(s) that the object is intended for.
+| `CamelGoogleCloudStorageContentType` |`String` |The content type of this
object.
+| `CamelGoogleCloudStorageCustomTime` |`String` | The Custom-Time metadata is
a user-specified date and time represented in the RFC 3339 format
YYYY-MM-DD'T'HH:MM:SS.SS'Z' or YYYY-MM-DD'T'HH:MM:SS'Z' when milliseconds are
zero. This metadata is typically set in order to use the DaysSinceCustomTime
condition in Object Lifecycle Management.
+| `CamelGoogleCloudStorageCrc32cHex` |`String` | The CRC32c of the object
+| `CamelGoogleCloudStorageETag` |`String` | The ETag for the Object.
+| `CamelGoogleCloudStorageGeneration` |`String` | Is the generation number of
the object for which you are retrieving information.
+| `CamelGoogleCloudStorageBlobId` |`String` | The blob id of the object
+| `CamelGoogleCloudStorageKmsKeyName` |`String` | The KMS key name
+| `CamelGoogleCloudStorageContentMd5` |`String` |The md5 checksum of this
object.
+| `CamelGoogleCloudStorageMediaLink` |`String` | The media link
+| `CamelGoogleCloudStorageMetageneration` | `String` | The metageneration of
the object
+| `CamelGoogleCloudStorageContentLength` |`Long` |The content length of this
object.
+| `CamelGoogleCloudStorageStorageClass` | `String` | The storage class of the
object
+| `CamelGoogleCloudStorageCreateTime` |`String` | The creation time of the
object
+| `CamelGoogleCloudStorageLastUpdate` |`String` | The last update of the object
+|=======================================================================
+
+=== Google Storage Producer operations
+
+Google Storage component provides the following operation on the producer side:
+
+- copyObject
+- listObjects
+- deleteObject
+- deleteBucket
+- listBuckets
+- getObject
+- createDownloadLink
+
+If you don't specify an operation explicitly the producer will a file upload.
+
+=== Advanced component configuration
+
+If you need to have more control over the `storageClient` instance
configuration, you can create your own instance and refer to it in your Camel
google-storage component configuration:
+
+[source,java]
+--------------------------------------------------------------------------------
+from("google-storage://myCamelBucket?storageClient=#client")
+.to("mock:result");
+--------------------------------------------------------------------------------
+
+=== Google Storage Producer Operation examples
+
+- File Upload: This operation will upload a file to the Google Storage based
on the body content
+
+[source,java]
+--------------------------------------------------------------------------------
+//upload a file
+byte[] payload = "Camel rocks!".getBytes();
+ByteArrayInputStream bais = new ByteArrayInputStream(payload);
+from("direct:start")
+.process( exchange -> {
+ exchange.getIn().setHeader(GoogleCloudStorageConstants.OBJECT_NAME,
"camel.txt");
+ exchange.getIn().setBody(bais);
+})
+.to("google-storage://myCamelBucket?serviceAccountKey=/home/user/Downloads/my-key.json")
+.log("uploaded file object:${header.CamelGoogleCloudStorageObjectName},
body:${body}");
+--------------------------------------------------------------------------------
+
+This operation will upload the file camel.txt with the content "Camel rocks!"
in the myCamelBucket bucket
+
+- CopyObject: this operation copy an object from one bucket to a different one
+
+[source,java]
+--------------------------------------------------------------------------------
+ from("direct:start").process(new Processor() {
+
+ @Override
+ public void process(Exchange exchange) throws Exception {
+ exchange.getIn().setHeader(S3Constants.BUCKET_DESTINATION_NAME,
"camelDestinationBucket");
+ exchange.getIn().setHeader(S3Constants.KEY, "camelKey");
+ exchange.getIn().setHeader(S3Constants.DESTINATION_KEY,
"camelDestinationKey");
+ }
+ })
+
.to("google-storage://mycamelbucket?amazonS3Client=#amazonS3Client&operation=copyObject")
+ .to("mock:result");
+--------------------------------------------------------------------------------
+
+This operation will copy the object with the name expressed in the header
camelDestinationKey to the camelDestinationBucket bucket, from the bucket
mycamelbucket.
+
+- DeleteObject: this operation deletes an object from a bucket
+
+[source,java]
+--------------------------------------------------------------------------------
+ from("direct:start").process(new Processor() {
+
+ @Override
+ public void process(Exchange exchange) throws Exception {
+ exchange.getIn().setHeader(S3Constants.KEY, "camelKey");
+ }
+ })
+
.to("google-storage://mycamelbucket?amazonS3Client=#amazonS3Client&operation=deleteObject")
+ .to("mock:result");
+--------------------------------------------------------------------------------
+
+This operation will delete the object camelKey from the bucket mycamelbucket.
+
+- ListBuckets: this operation list the buckets for this account in this region
+
+[source,java]
+--------------------------------------------------------------------------------
+ from("direct:start")
+
.to("google-storage://mycamelbucket?amazonS3Client=#amazonS3Client&operation=listBuckets")
+ .to("mock:result");
+--------------------------------------------------------------------------------
+
+This operation will list the buckets for this account
+
+- DeleteBucket: this operation delete the bucket specified as URI parameter or
header
+
+[source,java]
+--------------------------------------------------------------------------------
+ from("direct:start")
+
.to("google-storage://mycamelbucket?amazonS3Client=#amazonS3Client&operation=deleteBucket")
+ .to("mock:result");
+--------------------------------------------------------------------------------
+
+This operation will delete the bucket mycamelbucket
+
+- ListObjects: this operation list object in a specific bucket
+
+[source,java]
+--------------------------------------------------------------------------------
+ from("direct:start")
+
.to("google-storage://mycamelbucket?amazonS3Client=#amazonS3Client&operation=listObjects")
+ .to("mock:result");
+--------------------------------------------------------------------------------
+
+This operation will list the objects in the mycamelbucket bucket
+
+- GetObject: this operation get a single object in a specific bucket
+
+[source,java]
+--------------------------------------------------------------------------------
+ from("direct:start").process(new Processor() {
+
+ @Override
+ public void process(Exchange exchange) throws Exception {
+ exchange.getIn().setHeader(S3Constants.KEY, "camelKey");
+ }
+ })
+
.to("google-storage://mycamelbucket?amazonS3Client=#amazonS3Client&operation=getObject")
+ .to("mock:result");
+--------------------------------------------------------------------------------
+
+This operation will return an S3Object instance related to the camelKey object
in mycamelbucket bucket.
+
+- GetObjectRange: this operation get a single object range in a specific bucket
+
+[source,java]
+--------------------------------------------------------------------------------
+ from("direct:start").process(new Processor() {
+
+ @Override
+ public void process(Exchange exchange) throws Exception {
+ exchange.getIn().setHeader(S3Constants.KEY, "camelKey");
+ exchange.getIn().setHeader(S3Constants.RANGE_START, "0");
+ exchange.getIn().setHeader(S3Constants.RANGE_END, "9");
+ }
+ })
+
.to("google-storage://mycamelbucket?amazonS3Client=#amazonS3Client&operation=getObjectRange")
+ .to("mock:result");
+--------------------------------------------------------------------------------
+
+This operation will return an S3Object instance related to the camelKey object
in mycamelbucket bucket, containing a the bytes from 0 to 9.
+
+- CreateDownloadLink: this operation will return a download link through S3
Presigner
+
+[source,java]
+--------------------------------------------------------------------------------
+ from("direct:start").process(new Processor() {
+
+ @Override
+ public void process(Exchange exchange) throws Exception {
+ exchange.getIn().setHeader(S3Constants.KEY, "camelKey");
+ }
+ })
+
.to("google-storage://mycamelbucket?accessKey=xxx&secretKey=yyy®ion=region&operation=createDownloadLink")
+ .to("mock:result");
+--------------------------------------------------------------------------------
+
+This operation will return a download link url for the file camel-key in the
bucket mycamelbucket and region region
+
+== Bucket Autocreation
+
+With the option `autoCreateBucket` users are able to avoid the autocreation of
an S3 Bucket in case it doesn't exist. The default for this option is `true`.
+If set to false any operation on a not-existent bucket in AWS won't be
successful and an error will be returned.
+
+== Moving stuff between a bucket and another bucket
+
+Some users like to consume stuff from a bucket and move the content in a
different one without using the copyObject feature of this component.
+If this is case for you, don't forget to remove the bucketName header from the
incoming exchange of the consumer, otherwise the file will be always
overwritten on the same
+original bucket.
+
+== MoveAfterRead consumer option
+
+In addition to deleteAfterRead it has been added another option,
moveAfterRead. With this option enabled the consumed object will be moved to a
target destinationBucket instead of being only deleted.
+This will require specifying the destinationBucket option. As example:
+
+[source,java]
+--------------------------------------------------------------------------------
+
from("google-storage://mycamelbucket?amazonS3Client=#amazonS3Client&moveAfterRead=true&destinationBucket=myothercamelbucket")
+ .to("mock:result");
+--------------------------------------------------------------------------------
+
+In this case the objects consumed will be moved to myothercamelbucket bucket
and deleted from the original one (because of deleteAfterRead set to true as
default).
+
+You have also the possibility of using a key prefix/suffix while moving the
file to a different bucket. The options are destinationBucketPrefix and
destinationBucketSuffix.
+
+Taking the above example, you could do something like:
+
+[source,java]
+--------------------------------------------------------------------------------
+
from("google-storage://mycamelbucket?amazonS3Client=#amazonS3Client&moveAfterRead=true&destinationBucket=myothercamelbucket&destinationBucketPrefix=RAW(pre-)&destinationBucketSuffix=RAW(-suff)")
+ .to("mock:result");
+--------------------------------------------------------------------------------
+
+In this case the objects consumed will be moved to myothercamelbucket bucket
and deleted from the original one (because of deleteAfterRead set to true as
default).
+
+So if the file name is test, in the myothercamelbucket you should see a file
called pre-test-suff.
+
+== Using customer key as encryption
+
+We introduced also the customer key support (an alternative of using KMS). The
following code shows an example.
+
+[source,java]
+--------------------------------------------------------------------------------
+String key = UUID.randomUUID().toString();
+byte[] secretKey = generateSecretKey();
+String b64Key = Base64.getEncoder().encodeToString(secretKey);
+String b64KeyMd5 = Md5Utils.md5AsBase64(secretKey);
+
+String awsEndpoint =
"google-storage://mycamel?autoCreateBucket=false&useCustomerKey=true&customerKeyId=RAW("
+ b64Key + ")&customerKeyMD5=RAW(" + b64KeyMd5 + ")&customerAlgorithm=" +
AES256.name();
+
+from("direct:putObject")
+ .setHeader(AWS2S3Constants.KEY, constant("test.txt"))
+ .setBody(constant("Test"))
+ .to(awsEndpoint);
+--------------------------------------------------------------------------------
+
+== Using a POJO as body
+
+Sometimes build an AWS Request can be complex, because of multiple options. We
introduce the possibility to use a POJO as body.
+In AWS S3 there are multiple operations you can submit, as an example for List
brokers request, you can do something like:
+
+------------------------------------------------------------------------------------------------------
+from("direct:google-storage")
+ .setBody(ListObjectsRequest.builder().bucket(bucketName).build())
+
.to("google-storage://test?amazonS3Client=#amazonS3Client&operation=listObjects&pojoRequest=true")
+------------------------------------------------------------------------------------------------------
+
+In this way you'll pass the request directly without the need of passing
headers and options specifically related to this operation.
+
+== Create S3 client and add component to registry
+Sometimes you would want to perform some advanced configuration using
AWS2S3Configuration which also allows to set the S3 client.
+You can create and set the S3 client in the component configuration as shown
in the following example
+
+[source,java]
+--------------------------------------------------------------------------------
+String awsBucketAccessKey = "your_access_key";
+String awsBucketSecretKey = "your_secret_key";
+
+S3Client s3Client =
S3Client.builder().credentialsProvider(StaticCredentialsProvider.create(AwsBasicCredentials.create(awsBucketAccessKey,
awsBucketSecretKey)))
+ .region(Region.US_EAST_1).build();
+
+AWS2S3Configuration configuration = new AWS2S3Configuration();
+configuration.setAmazonS3Client(s3Client);
+configuration.setAutoDiscoverClient(true);
+configuration.setBucketName("s3bucket2020");
+configuration.setRegion("us-east-1");
+--------------------------------------------------------------------------------
+
+Now you can configure the S3 component (using the configuration object created
above) and add it to the registry in the
+configure method before initialization of routes.
+
+[source,java]
+--------------------------------------------------------------------------------
+AWS2S3Component s3Component = new AWS2S3Component(getContext());
+s3Component.setConfiguration(configuration);
+s3Component.setLazyStartProducer(true);
+camelContext.addComponent("google-storage", s3Component);
+--------------------------------------------------------------------------------
+
+Now your component will be used for all the operations implemented in camel
routes.
-TBD
