This is an automated email from the ASF dual-hosted git repository.
acosentino pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/camel.git
commit 72777e50566e7ae6863c39f8e52f6ebd6f724998
Author: Raffaele Marcello <marcelloraffa...@gmail.com>
AuthorDate: Sat Feb 13 18:30:20 2021 +0100
CAMEL-15964 create camel-google-storage component
---
.../catalog/docs/google-storage-component.adoc | 35 ++++--
.../GoogleCloudStorageComponentConfigurer.java | 6 +
.../GoogleCloudStorageEndpointConfigurer.java | 6 +
.../GoogleCloudStorageEndpointUriFactory.java | 3 +-
.../component/google/storage/google-storage.json | 2 +
.../src/main/docs/google-storage-component.adoc | 35 ++++--
.../storage/GoogleCloudStorageComponent.java | 16 +++
.../GoogleCloudStorageComponentConfiguration.java | 67 +++++-------
.../GoogleCloudStorageComponentOperations.java | 16 +++
.../GoogleCloudStorageConnectionFactory.java | 31 +++++-
.../storage/GoogleCloudStorageConstants.java | 55 +++++-----
.../google/storage/GoogleCloudStorageConsumer.java | 121 +++++----------------
.../google/storage/GoogleCloudStorageEndpoint.java | 74 ++++++-------
.../google/storage/GoogleCloudStorageProducer.java | 32 +++---
.../dsl/GoogleStorageComponentBuilderFactory.java | 19 ++++
.../GoogleCloudStorageEndpointBuilderFactory.java | 37 +++++++
.../ROOT/pages/google-storage-component.adoc | 35 ++++--
17 files changed, 349 insertions(+), 241 deletions(-)
diff --git
a/catalog/camel-catalog/src/generated/resources/org/apache/camel/catalog/docs/google-storage-component.adoc
b/catalog/camel-catalog/src/generated/resources/org/apache/camel/catalog/docs/google-storage-component.adoc
index 7754e9a..a473b5e 100644
---
a/catalog/camel-catalog/src/generated/resources/org/apache/camel/catalog/docs/google-storage-component.adoc
+++
b/catalog/camel-catalog/src/generated/resources/org/apache/camel/catalog/docs/google-storage-component.adoc
@@ -14,7 +14,7 @@
include::{cq-version}@camel-quarkus:ROOT:partial$reference/components/google-sto
*{component-header}*
-The Google Storage component provides access to
https://cloud.google.com/storage/[Cloud Storage] via
+The Google Storage component provides access to
https://cloud.google.com/storage/[Google Cloud Storage] via
the https://github.com/googleapis/java-storage[google java storage library].
Maven users will need to add the following dependency to their pom.xml
@@ -46,26 +46,43 @@ Google security credentials can be set explicitly by
providing the path to the G
export GOOGLE_APPLICATION_CREDENTIALS="/home/user/Downloads/my-key.json"
--------------------------------------------------------
-or directly through the component endpoint
+or for windows:
+
[source,text]
--------------------------------------------------------
-from("google-storage://myCamelBucket?serviceAccountKey=/home/user/Downloads/my-key.json")
+$Env:GOOGLE_APPLICATION_CREDENTIALS = "/home/user/Downloads/my-key.json"
--------------------------------------------------------
-
+or directly through the component endpoint
+[source,text]
+--------------------------------------------------------
+String endpoint =
"google-storage://myCamelBucket?serviceAccountKey=/home/user/Downloads/my-key.json";
+--------------------------------------------------------
== URI Format
[source,text]
--------------------------------------------------------
- google-storage://bucketName?[options]
+google-storage://bucketNameOrArn?[options]
--------------------------------------------------------
+The bucket will be created if it don't already exists. +
+ You can append query options to the URI in the following format,
+?options=value&option2=value&...
+
+For example in order to read file `hello.txt` from bucket `helloBucket`, use
the following snippet:
+
+[source,java]
+--------------------------------------------------------------------------------
+from("google-storage://myCamelBucket?serviceAccountKey=/home/user/Downloads/my-key.json&prefix=hello.txt")
+ .to("file:/var/downloaded");
+--------------------------------------------------------------------------------
+
-== Options
+== URI Options
// component options: START
-The Google Storage component supports 13 options, which are listed below.
+The Google Storage component supports 14 options, which are listed below.
@@ -80,6 +97,7 @@ The Google Storage component supports 13 options, which are
listed below.
| *deleteAfterRead* (consumer) | Delete objects from the bucket after they
have been retrieved. The delete is only performed if the Exchange is committed.
If a rollback occurs, the object is not deleted. If this option is false, then
the same objects will be retrieve over and over again on the polls. | true |
boolean
| *destinationBucket* (consumer) | Define the destination bucket where an
object must be moved when moveAfterRead is set to true. | | String
| *includeBody* (consumer) | If it is true, the Object exchange will be
consumed and put into the body and closed. If false the Object stream will be
put raw into the body and the headers will be set with the object metadata. |
true | boolean
+| *includeFolders* (consumer) | If it is true, the folders/directories will be
consumed. If it is false, they will be ignored, and Exchanges will not be
created for those | true | boolean
| *moveAfterRead* (consumer) | Move objects from the origin bucket to a
different bucket after they have been retrieved. To accomplish the operation
the destinationBucket option must be set. The copy bucket operation is only
performed if the Exchange is committed. If a rollback occurs, the object is not
moved. | false | boolean
| *lazyStartProducer* (producer) | Whether the producer should be started lazy
(on the first message). By starting lazy you can use this to allow CamelContext
and routes to startup in situations where a producer may otherwise fail during
starting and cause the route to fail being started. By deferring this startup
to be lazy then the startup failure can be handled during routing messages via
Camel's routing error handlers. Beware that when the first message is processed
then creating and [...]
| *objectName* (producer) | Object name | | String
@@ -107,7 +125,7 @@ with the following path and query parameters:
|===
-=== Query Parameters (29 parameters):
+=== Query Parameters (30 parameters):
[width="100%",cols="2,5,^1,2",options="header"]
@@ -120,6 +138,7 @@ with the following path and query parameters:
| *deleteAfterRead* (consumer) | Delete objects from the bucket after they
have been retrieved. The delete is only performed if the Exchange is committed.
If a rollback occurs, the object is not deleted. If this option is false, then
the same objects will be retrieve over and over again on the polls. | true |
boolean
| *destinationBucket* (consumer) | Define the destination bucket where an
object must be moved when moveAfterRead is set to true. | | String
| *includeBody* (consumer) | If it is true, the Object exchange will be
consumed and put into the body and closed. If false the Object stream will be
put raw into the body and the headers will be set with the object metadata. |
true | boolean
+| *includeFolders* (consumer) | If it is true, the folders/directories will be
consumed. If it is false, they will be ignored, and Exchanges will not be
created for those | true | boolean
| *moveAfterRead* (consumer) | Move objects from the origin bucket to a
different bucket after they have been retrieved. To accomplish the operation
the destinationBucket option must be set. The copy bucket operation is only
performed if the Exchange is committed. If a rollback occurs, the object is not
moved. | false | boolean
| *sendEmptyMessageWhenIdle* (consumer) | If the polling consumer did not poll
any files, you can enable this option to send an empty message (no body)
instead. | false | boolean
| *exceptionHandler* (consumer) | To let the consumer use a custom
ExceptionHandler. Notice if the option bridgeErrorHandler is enabled then this
option is not in use. By default the consumer will deal with exceptions, that
will be logged at WARN or ERROR level and ignored. | | ExceptionHandler
diff --git
a/components/camel-google-storage/src/generated/java/org/apache/camel/component/google/storage/GoogleCloudStorageComponentConfigurer.java
b/components/camel-google-storage/src/generated/java/org/apache/camel/component/google/storage/GoogleCloudStorageComponentConfigurer.java
index 559d418..f2aa873 100644
---
a/components/camel-google-storage/src/generated/java/org/apache/camel/component/google/storage/GoogleCloudStorageComponentConfigurer.java
+++
b/components/camel-google-storage/src/generated/java/org/apache/camel/component/google/storage/GoogleCloudStorageComponentConfigurer.java
@@ -41,6 +41,8 @@ public class GoogleCloudStorageComponentConfigurer extends
PropertyConfigurerSup
case "destinationBucket":
getOrCreateConfiguration(target).setDestinationBucket(property(camelContext,
java.lang.String.class, value)); return true;
case "includebody":
case "includeBody":
getOrCreateConfiguration(target).setIncludeBody(property(camelContext,
boolean.class, value)); return true;
+ case "includefolders":
+ case "includeFolders":
getOrCreateConfiguration(target).setIncludeFolders(property(camelContext,
boolean.class, value)); return true;
case "lazystartproducer":
case "lazyStartProducer":
target.setLazyStartProducer(property(camelContext, boolean.class, value));
return true;
case "moveafterread":
@@ -72,6 +74,8 @@ public class GoogleCloudStorageComponentConfigurer extends
PropertyConfigurerSup
case "destinationBucket": return java.lang.String.class;
case "includebody":
case "includeBody": return boolean.class;
+ case "includefolders":
+ case "includeFolders": return boolean.class;
case "lazystartproducer":
case "lazyStartProducer": return boolean.class;
case "moveafterread":
@@ -104,6 +108,8 @@ public class GoogleCloudStorageComponentConfigurer extends
PropertyConfigurerSup
case "destinationBucket": return
getOrCreateConfiguration(target).getDestinationBucket();
case "includebody":
case "includeBody": return
getOrCreateConfiguration(target).isIncludeBody();
+ case "includefolders":
+ case "includeFolders": return
getOrCreateConfiguration(target).isIncludeFolders();
case "lazystartproducer":
case "lazyStartProducer": return target.isLazyStartProducer();
case "moveafterread":
diff --git
a/components/camel-google-storage/src/generated/java/org/apache/camel/component/google/storage/GoogleCloudStorageEndpointConfigurer.java
b/components/camel-google-storage/src/generated/java/org/apache/camel/component/google/storage/GoogleCloudStorageEndpointConfigurer.java
index fe3ab5f..1e8d17a 100644
---
a/components/camel-google-storage/src/generated/java/org/apache/camel/component/google/storage/GoogleCloudStorageEndpointConfigurer.java
+++
b/components/camel-google-storage/src/generated/java/org/apache/camel/component/google/storage/GoogleCloudStorageEndpointConfigurer.java
@@ -43,6 +43,8 @@ public class GoogleCloudStorageEndpointConfigurer extends
PropertyConfigurerSupp
case "greedy": target.setGreedy(property(camelContext, boolean.class,
value)); return true;
case "includebody":
case "includeBody":
target.getConfiguration().setIncludeBody(property(camelContext, boolean.class,
value)); return true;
+ case "includefolders":
+ case "includeFolders":
target.getConfiguration().setIncludeFolders(property(camelContext,
boolean.class, value)); return true;
case "initialdelay":
case "initialDelay": target.setInitialDelay(property(camelContext,
long.class, value)); return true;
case "lazystartproducer":
@@ -104,6 +106,8 @@ public class GoogleCloudStorageEndpointConfigurer extends
PropertyConfigurerSupp
case "greedy": return boolean.class;
case "includebody":
case "includeBody": return boolean.class;
+ case "includefolders":
+ case "includeFolders": return boolean.class;
case "initialdelay":
case "initialDelay": return long.class;
case "lazystartproducer":
@@ -166,6 +170,8 @@ public class GoogleCloudStorageEndpointConfigurer extends
PropertyConfigurerSupp
case "greedy": return target.isGreedy();
case "includebody":
case "includeBody": return target.getConfiguration().isIncludeBody();
+ case "includefolders":
+ case "includeFolders": return
target.getConfiguration().isIncludeFolders();
case "initialdelay":
case "initialDelay": return target.getInitialDelay();
case "lazystartproducer":
diff --git
a/components/camel-google-storage/src/generated/java/org/apache/camel/component/google/storage/GoogleCloudStorageEndpointUriFactory.java
b/components/camel-google-storage/src/generated/java/org/apache/camel/component/google/storage/GoogleCloudStorageEndpointUriFactory.java
index dbd0a68..86fe7f9 100644
---
a/components/camel-google-storage/src/generated/java/org/apache/camel/component/google/storage/GoogleCloudStorageEndpointUriFactory.java
+++
b/components/camel-google-storage/src/generated/java/org/apache/camel/component/google/storage/GoogleCloudStorageEndpointUriFactory.java
@@ -20,7 +20,7 @@ public class GoogleCloudStorageEndpointUriFactory extends
org.apache.camel.suppo
private static final Set<String> PROPERTY_NAMES;
private static final Set<String> SECRET_PROPERTY_NAMES;
static {
- Set<String> props = new HashSet<>(30);
+ Set<String> props = new HashSet<>(31);
props.add("backoffMultiplier");
props.add("bucketName");
props.add("destinationBucket");
@@ -40,6 +40,7 @@ public class GoogleCloudStorageEndpointUriFactory extends
org.apache.camel.suppo
props.add("sendEmptyMessageWhenIdle");
props.add("schedulerProperties");
props.add("exchangePattern");
+ props.add("includeFolders");
props.add("storageClient");
props.add("backoffIdleThreshold");
props.add("lazyStartProducer");
diff --git
a/components/camel-google-storage/src/generated/resources/org/apache/camel/component/google/storage/google-storage.json
b/components/camel-google-storage/src/generated/resources/org/apache/camel/component/google/storage/google-storage.json
index a139f6c..7e946ab 100644
---
a/components/camel-google-storage/src/generated/resources/org/apache/camel/component/google/storage/google-storage.json
+++
b/components/camel-google-storage/src/generated/resources/org/apache/camel/component/google/storage/google-storage.json
@@ -30,6 +30,7 @@
"deleteAfterRead": { "kind": "property", "displayName": "Delete After
Read", "group": "consumer", "label": "consumer", "required": false, "type":
"boolean", "javaType": "boolean", "deprecated": false, "autowired": false,
"secret": false, "defaultValue": true, "configurationClass":
"org.apache.camel.component.google.storage.GoogleCloudStorageComponentConfiguration",
"configurationField": "configuration", "description": "Delete objects from the
bucket after they have been retrieved. Th [...]
"destinationBucket": { "kind": "property", "displayName": "Destination
Bucket", "group": "consumer", "label": "consumer", "required": false, "type":
"string", "javaType": "java.lang.String", "deprecated": false, "autowired":
false, "secret": false, "configurationClass":
"org.apache.camel.component.google.storage.GoogleCloudStorageComponentConfiguration",
"configurationField": "configuration", "description": "Define the destination
bucket where an object must be moved when moveAfterRe [...]
"includeBody": { "kind": "property", "displayName": "Include Body",
"group": "consumer", "label": "consumer", "required": false, "type": "boolean",
"javaType": "boolean", "deprecated": false, "autowired": false, "secret":
false, "defaultValue": true, "configurationClass":
"org.apache.camel.component.google.storage.GoogleCloudStorageComponentConfiguration",
"configurationField": "configuration", "description": "If it is true, the
Object exchange will be consumed and put into the body [...]
+ "includeFolders": { "kind": "property", "displayName": "Include Folders",
"group": "consumer", "label": "consumer", "required": false, "type": "boolean",
"javaType": "boolean", "deprecated": false, "autowired": false, "secret":
false, "defaultValue": true, "configurationClass":
"org.apache.camel.component.google.storage.GoogleCloudStorageComponentConfiguration",
"configurationField": "configuration", "description": "If it is true, the
folders\/directories will be consumed. If it is f [...]
"moveAfterRead": { "kind": "property", "displayName": "Move After Read",
"group": "consumer", "label": "consumer", "required": false, "type": "boolean",
"javaType": "boolean", "deprecated": false, "autowired": false, "secret":
false, "defaultValue": false, "configurationClass":
"org.apache.camel.component.google.storage.GoogleCloudStorageComponentConfiguration",
"configurationField": "configuration", "description": "Move objects from the
origin bucket to a different bucket after they [...]
"lazyStartProducer": { "kind": "property", "displayName": "Lazy Start
Producer", "group": "producer", "label": "producer", "required": false, "type":
"boolean", "javaType": "boolean", "deprecated": false, "autowired": false,
"secret": false, "defaultValue": false, "description": "Whether the producer
should be started lazy (on the first message). By starting lazy you can use
this to allow CamelContext and routes to startup in situations where a producer
may otherwise fail during star [...]
"objectName": { "kind": "property", "displayName": "Object Name", "group":
"producer", "label": "producer", "required": false, "type": "string",
"javaType": "java.lang.String", "deprecated": false, "autowired": false,
"secret": false, "configurationClass":
"org.apache.camel.component.google.storage.GoogleCloudStorageComponentConfiguration",
"configurationField": "configuration", "description": "Object name" },
@@ -45,6 +46,7 @@
"deleteAfterRead": { "kind": "parameter", "displayName": "Delete After
Read", "group": "consumer", "label": "consumer", "required": false, "type":
"boolean", "javaType": "boolean", "deprecated": false, "autowired": false,
"secret": false, "defaultValue": true, "configurationClass":
"org.apache.camel.component.google.storage.GoogleCloudStorageComponentConfiguration",
"configurationField": "configuration", "description": "Delete objects from the
bucket after they have been retrieved. T [...]
"destinationBucket": { "kind": "parameter", "displayName": "Destination
Bucket", "group": "consumer", "label": "consumer", "required": false, "type":
"string", "javaType": "java.lang.String", "deprecated": false, "autowired":
false, "secret": false, "configurationClass":
"org.apache.camel.component.google.storage.GoogleCloudStorageComponentConfiguration",
"configurationField": "configuration", "description": "Define the destination
bucket where an object must be moved when moveAfterR [...]
"includeBody": { "kind": "parameter", "displayName": "Include Body",
"group": "consumer", "label": "consumer", "required": false, "type": "boolean",
"javaType": "boolean", "deprecated": false, "autowired": false, "secret":
false, "defaultValue": true, "configurationClass":
"org.apache.camel.component.google.storage.GoogleCloudStorageComponentConfiguration",
"configurationField": "configuration", "description": "If it is true, the
Object exchange will be consumed and put into the body [...]
+ "includeFolders": { "kind": "parameter", "displayName": "Include Folders",
"group": "consumer", "label": "consumer", "required": false, "type": "boolean",
"javaType": "boolean", "deprecated": false, "autowired": false, "secret":
false, "defaultValue": true, "configurationClass":
"org.apache.camel.component.google.storage.GoogleCloudStorageComponentConfiguration",
"configurationField": "configuration", "description": "If it is true, the
folders\/directories will be consumed. If it is [...]
"moveAfterRead": { "kind": "parameter", "displayName": "Move After Read",
"group": "consumer", "label": "consumer", "required": false, "type": "boolean",
"javaType": "boolean", "deprecated": false, "autowired": false, "secret":
false, "defaultValue": false, "configurationClass":
"org.apache.camel.component.google.storage.GoogleCloudStorageComponentConfiguration",
"configurationField": "configuration", "description": "Move objects from the
origin bucket to a different bucket after the [...]
"sendEmptyMessageWhenIdle": { "kind": "parameter", "displayName": "Send
Empty Message When Idle", "group": "consumer", "label": "consumer", "required":
false, "type": "boolean", "javaType": "boolean", "deprecated": false,
"autowired": false, "secret": false, "defaultValue": false, "description": "If
the polling consumer did not poll any files, you can enable this option to send
an empty message (no body) instead." },
"exceptionHandler": { "kind": "parameter", "displayName": "Exception
Handler", "group": "consumer (advanced)", "label": "consumer,advanced",
"required": false, "type": "object", "javaType":
"org.apache.camel.spi.ExceptionHandler", "optionalPrefix": "consumer.",
"deprecated": false, "autowired": false, "secret": false, "description": "To
let the consumer use a custom ExceptionHandler. Notice if the option
bridgeErrorHandler is enabled then this option is not in use. By default the
con [...]
diff --git
a/components/camel-google-storage/src/main/docs/google-storage-component.adoc
b/components/camel-google-storage/src/main/docs/google-storage-component.adoc
index 7754e9a..a473b5e 100644
---
a/components/camel-google-storage/src/main/docs/google-storage-component.adoc
+++
b/components/camel-google-storage/src/main/docs/google-storage-component.adoc
@@ -14,7 +14,7 @@
include::{cq-version}@camel-quarkus:ROOT:partial$reference/components/google-sto
*{component-header}*
-The Google Storage component provides access to
https://cloud.google.com/storage/[Cloud Storage] via
+The Google Storage component provides access to
https://cloud.google.com/storage/[Google Cloud Storage] via
the https://github.com/googleapis/java-storage[google java storage library].
Maven users will need to add the following dependency to their pom.xml
@@ -46,26 +46,43 @@ Google security credentials can be set explicitly by
providing the path to the G
export GOOGLE_APPLICATION_CREDENTIALS="/home/user/Downloads/my-key.json"
--------------------------------------------------------
-or directly through the component endpoint
+or for windows:
+
[source,text]
--------------------------------------------------------
-from("google-storage://myCamelBucket?serviceAccountKey=/home/user/Downloads/my-key.json")
+$Env:GOOGLE_APPLICATION_CREDENTIALS = "/home/user/Downloads/my-key.json"
--------------------------------------------------------
-
+or directly through the component endpoint
+[source,text]
+--------------------------------------------------------
+String endpoint =
"google-storage://myCamelBucket?serviceAccountKey=/home/user/Downloads/my-key.json";
+--------------------------------------------------------
== URI Format
[source,text]
--------------------------------------------------------
- google-storage://bucketName?[options]
+google-storage://bucketNameOrArn?[options]
--------------------------------------------------------
+The bucket will be created if it don't already exists. +
+ You can append query options to the URI in the following format,
+?options=value&option2=value&...
+
+For example in order to read file `hello.txt` from bucket `helloBucket`, use
the following snippet:
+
+[source,java]
+--------------------------------------------------------------------------------
+from("google-storage://myCamelBucket?serviceAccountKey=/home/user/Downloads/my-key.json&prefix=hello.txt")
+ .to("file:/var/downloaded");
+--------------------------------------------------------------------------------
+
-== Options
+== URI Options
// component options: START
-The Google Storage component supports 13 options, which are listed below.
+The Google Storage component supports 14 options, which are listed below.
@@ -80,6 +97,7 @@ The Google Storage component supports 13 options, which are
listed below.
| *deleteAfterRead* (consumer) | Delete objects from the bucket after they
have been retrieved. The delete is only performed if the Exchange is committed.
If a rollback occurs, the object is not deleted. If this option is false, then
the same objects will be retrieve over and over again on the polls. | true |
boolean
| *destinationBucket* (consumer) | Define the destination bucket where an
object must be moved when moveAfterRead is set to true. | | String
| *includeBody* (consumer) | If it is true, the Object exchange will be
consumed and put into the body and closed. If false the Object stream will be
put raw into the body and the headers will be set with the object metadata. |
true | boolean
+| *includeFolders* (consumer) | If it is true, the folders/directories will be
consumed. If it is false, they will be ignored, and Exchanges will not be
created for those | true | boolean
| *moveAfterRead* (consumer) | Move objects from the origin bucket to a
different bucket after they have been retrieved. To accomplish the operation
the destinationBucket option must be set. The copy bucket operation is only
performed if the Exchange is committed. If a rollback occurs, the object is not
moved. | false | boolean
| *lazyStartProducer* (producer) | Whether the producer should be started lazy
(on the first message). By starting lazy you can use this to allow CamelContext
and routes to startup in situations where a producer may otherwise fail during
starting and cause the route to fail being started. By deferring this startup
to be lazy then the startup failure can be handled during routing messages via
Camel's routing error handlers. Beware that when the first message is processed
then creating and [...]
| *objectName* (producer) | Object name | | String
@@ -107,7 +125,7 @@ with the following path and query parameters:
|===
-=== Query Parameters (29 parameters):
+=== Query Parameters (30 parameters):
[width="100%",cols="2,5,^1,2",options="header"]
@@ -120,6 +138,7 @@ with the following path and query parameters:
| *deleteAfterRead* (consumer) | Delete objects from the bucket after they
have been retrieved. The delete is only performed if the Exchange is committed.
If a rollback occurs, the object is not deleted. If this option is false, then
the same objects will be retrieve over and over again on the polls. | true |
boolean
| *destinationBucket* (consumer) | Define the destination bucket where an
object must be moved when moveAfterRead is set to true. | | String
| *includeBody* (consumer) | If it is true, the Object exchange will be
consumed and put into the body and closed. If false the Object stream will be
put raw into the body and the headers will be set with the object metadata. |
true | boolean
+| *includeFolders* (consumer) | If it is true, the folders/directories will be
consumed. If it is false, they will be ignored, and Exchanges will not be
created for those | true | boolean
| *moveAfterRead* (consumer) | Move objects from the origin bucket to a
different bucket after they have been retrieved. To accomplish the operation
the destinationBucket option must be set. The copy bucket operation is only
performed if the Exchange is committed. If a rollback occurs, the object is not
moved. | false | boolean
| *sendEmptyMessageWhenIdle* (consumer) | If the polling consumer did not poll
any files, you can enable this option to send an empty message (no body)
instead. | false | boolean
| *exceptionHandler* (consumer) | To let the consumer use a custom
ExceptionHandler. Notice if the option bridgeErrorHandler is enabled then this
option is not in use. By default the consumer will deal with exceptions, that
will be logged at WARN or ERROR level and ignored. | | ExceptionHandler
diff --git
a/components/camel-google-storage/src/main/java/org/apache/camel/component/google/storage/GoogleCloudStorageComponent.java
b/components/camel-google-storage/src/main/java/org/apache/camel/component/google/storage/GoogleCloudStorageComponent.java
index 8ff9591..f53a171 100644
---
a/components/camel-google-storage/src/main/java/org/apache/camel/component/google/storage/GoogleCloudStorageComponent.java
+++
b/components/camel-google-storage/src/main/java/org/apache/camel/component/google/storage/GoogleCloudStorageComponent.java
@@ -1,3 +1,19 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
package org.apache.camel.component.google.storage;
import java.util.Map;
diff --git
a/components/camel-google-storage/src/main/java/org/apache/camel/component/google/storage/GoogleCloudStorageComponentConfiguration.java
b/components/camel-google-storage/src/main/java/org/apache/camel/component/google/storage/GoogleCloudStorageComponentConfiguration.java
index c77e5b3..5355ec8 100644
---
a/components/camel-google-storage/src/main/java/org/apache/camel/component/google/storage/GoogleCloudStorageComponentConfiguration.java
+++
b/components/camel-google-storage/src/main/java/org/apache/camel/component/google/storage/GoogleCloudStorageComponentConfiguration.java
@@ -1,3 +1,19 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
package org.apache.camel.component.google.storage;
import com.google.cloud.storage.Storage;
@@ -29,28 +45,18 @@ public class GoogleCloudStorageComponentConfiguration
implements Cloneable {
@UriParam(label = "consumer")
private boolean moveAfterRead;
+
@UriParam(label = "consumer")
private String destinationBucket;
- // @UriParam(label = "consumer")
- // private String destinationBucketPrefix;
- // @UriParam(label = "consumer")
- // private String destinationBucketSuffix;
+
@UriParam(label = "consumer", defaultValue = "true")
private boolean deleteAfterRead = true;
- /*
- @UriParam(label = "consumer")
- private String fileName;
- @UriParam(label = "consumer")
- private String prefix;
- @UriParam(label = "consumer")
- private String delimiter;
- @UriParam(label = "consumer")
- private String doneFileName;
- */
+
@UriParam(label = "consumer", defaultValue = "true")
private boolean includeBody = true;
- //@UriParam(label = "consumer", defaultValue = "true")
- //private boolean includeFolders = true;
+
+ @UriParam(label = "consumer", defaultValue = "true")
+ private boolean includeFolders = true;
@UriParam
private Storage storageClient;
@@ -151,41 +157,18 @@ public class GoogleCloudStorageComponentConfiguration
implements Cloneable {
this.destinationBucket = destinationBucket;
}
- /*
- public String getDestinationBucketPrefix() {
- return destinationBucketPrefix;
- }
-
- /**
- * Define the destination bucket prefix to use when an object must be
moved and moveAfterRead is set to true.
- *
- public void setDestinationBucketPrefix(String destinationBucketPrefix) {
- this.destinationBucketPrefix = destinationBucketPrefix;
- }
-
- public String getDestinationBucketSuffix() {
- return destinationBucketSuffix;
- }
-
- /**
- * Define the destination bucket suffix to use when an object must be
moved and moveAfterRead is set to true.
- *
- public void setDestinationBucketSuffix(String destinationBucketSuffix) {
- this.destinationBucketSuffix = destinationBucketSuffix;
- }
-
/**
* If it is true, the folders/directories will be consumed. If it is
false, they will be ignored, and Exchanges will
* not be created for those
- *
+ */
public void setIncludeFolders(boolean includeFolders) {
this.includeFolders = includeFolders;
}
-
+
public boolean isIncludeFolders() {
return includeFolders;
}
- */
+
public boolean isDeleteAfterRead() {
return deleteAfterRead;
}
diff --git
a/components/camel-google-storage/src/main/java/org/apache/camel/component/google/storage/GoogleCloudStorageComponentOperations.java
b/components/camel-google-storage/src/main/java/org/apache/camel/component/google/storage/GoogleCloudStorageComponentOperations.java
index f81597e..2415b6b 100644
---
a/components/camel-google-storage/src/main/java/org/apache/camel/component/google/storage/GoogleCloudStorageComponentOperations.java
+++
b/components/camel-google-storage/src/main/java/org/apache/camel/component/google/storage/GoogleCloudStorageComponentOperations.java
@@ -1,3 +1,19 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
package org.apache.camel.component.google.storage;
public enum GoogleCloudStorageComponentOperations {
diff --git
a/components/camel-google-storage/src/main/java/org/apache/camel/component/google/storage/GoogleCloudStorageConnectionFactory.java
b/components/camel-google-storage/src/main/java/org/apache/camel/component/google/storage/GoogleCloudStorageConnectionFactory.java
index 04ede16..e27c763 100644
---
a/components/camel-google-storage/src/main/java/org/apache/camel/component/google/storage/GoogleCloudStorageConnectionFactory.java
+++
b/components/camel-google-storage/src/main/java/org/apache/camel/component/google/storage/GoogleCloudStorageConnectionFactory.java
@@ -1,7 +1,24 @@
package org.apache.camel.component.google.storage;
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
import java.io.FileInputStream;
+import com.google.api.client.util.Strings;
import com.google.auth.oauth2.ServiceAccountCredentials;
import com.google.cloud.storage.Storage;
import com.google.cloud.storage.StorageOptions;
@@ -9,10 +26,16 @@ import com.google.cloud.storage.StorageOptions;
public class GoogleCloudStorageConnectionFactory {
public static Storage create(GoogleCloudStorageComponentConfiguration
configuration) throws Exception {
- Storage storage = StorageOptions.newBuilder()
- .setCredentials(ServiceAccountCredentials.fromStream(new
FileInputStream(configuration.getServiceAccountKey())))
- .build().getService();
- return storage;
+ if (!Strings.isNullOrEmpty(configuration.getServiceAccountKey())) {
+ Storage storage = StorageOptions.newBuilder()
+ .setCredentials(
+ ServiceAccountCredentials.fromStream(new
FileInputStream(configuration.getServiceAccountKey())))
+ .build().getService();
+ return storage;
+ } else {
+ Storage storage = StorageOptions.getDefaultInstance().getService();
+ return storage;
+ }
}
}
diff --git
a/components/camel-google-storage/src/main/java/org/apache/camel/component/google/storage/GoogleCloudStorageConstants.java
b/components/camel-google-storage/src/main/java/org/apache/camel/component/google/storage/GoogleCloudStorageConstants.java
index 5fda9ed..b16fc9d 100644
---
a/components/camel-google-storage/src/main/java/org/apache/camel/component/google/storage/GoogleCloudStorageConstants.java
+++
b/components/camel-google-storage/src/main/java/org/apache/camel/component/google/storage/GoogleCloudStorageConstants.java
@@ -1,3 +1,19 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
package org.apache.camel.component.google.storage;
public class GoogleCloudStorageConstants {
@@ -15,31 +31,18 @@ public class GoogleCloudStorageConstants {
public static final String CONTENT_ENCODING =
"CamelGoogleCloudStorageContentEncoding";
public static final String CONTENT_MD5 =
"CamelGoogleCloudStorageContentMd5";
- /*
- public static final String BUCKET = "BUCKET";
- public static final String CACHE_CONTROL = "CACHE_CONTROL";
-
- public static final String COMPONENT_COUNT = "COMPONENT_COUNT";
- public static final String CONTENT_DISPOSITION = "CONTENT_DISPOSITION";
- public static final String CONTENT_ENCODING = "CONTENT_ENCODING";
- public static final String CONTENT_LANGUAGE = "CONTENT_LANGUAGE";
- public static final String CONTENT_TYPE = "CONTENT_TYPE";
- public static final String CUSTOM_TIME = "CUSTOM_TIME";
- public static final String CRC32C = "CRC32C";
- public static final String CRC32C_HEX_STRING = "CRC32C_HEX_STRING";
- public static final String ETAG = "ETAG";
- public static final String GENERATION = "GENERATION";
- public static final String ID = "ID";
- public static final String KMS_KEY_NAME = "KMS_KEY_NAME";
- public static final String MD5_HASH = "MD5_HASH";
- public static final String MD5_HEX_STRING = "MD5_HEX_STRING";
- public static final String MEDIA_LINK ="MEDIA_LINK";
- public static final String METAGENERATION = "METAGENERATION";
- public static final String NAME = "NAME";
- public static final String SIZE = "SIZE";
- public static final String SORAGE_CLASS = "STORAGE_CLASS";
- public static final String TIME_CREATED = "TIME_CREATED";
- public static final String LAST_METADATA_UPDATE = "LAST_METADATA_UPDATE";
- */
+ public static final String METADATA_COMPONENT_COUNT =
"CamelGoogleCloudStorageComponentCount";
+ public static final String METADATA_CONTENT_LANGUAGE =
"CamelGoogleCloudStorageContentLanguage";
+ public static final String METADATA_CUSTOM_TIME =
"CamelGoogleCloudStorageCustomTime";
+ public static final String METADATA_CRC32C_hex =
"CamelGoogleCloudStorageCrc32cHex";
+ public static final String METADATA_ETAG = "CamelGoogleCloudStorageETag";
+ public static final String METADATA_GENERATION =
"CamelGoogleCloudStorageGeneration";
+ public static final String METADATA_BLOB_ID =
"CamelGoogleCloudStorageBlobId";
+ public static final String METADATA_KMS_KEY_NAME =
"CamelGoogleCloudStorageKmsKeyName";
+ public static final String METADATA_MEDIA_LINK =
"CamelGoogleCloudStorageMediaLink";
+ public static final String METADATA_METAGENERATION =
"CamelGoogleCloudStorageMetageneration";
+ public static final String METADATA_STORAGE_CLASS =
"CamelGoogleCloudStorageStorageClass";
+ public static final String METADATA_CREATE_TIME =
"CamelGoogleCloudStorageCreateTime";
+ public static final String METADATA_LAST_UPDATE =
"CamelGoogleCloudStorageLastUpdate";
}
diff --git
a/components/camel-google-storage/src/main/java/org/apache/camel/component/google/storage/GoogleCloudStorageConsumer.java
b/components/camel-google-storage/src/main/java/org/apache/camel/component/google/storage/GoogleCloudStorageConsumer.java
index 92cc6d4..42e7035 100644
---
a/components/camel-google-storage/src/main/java/org/apache/camel/component/google/storage/GoogleCloudStorageConsumer.java
+++
b/components/camel-google-storage/src/main/java/org/apache/camel/component/google/storage/GoogleCloudStorageConsumer.java
@@ -1,3 +1,19 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
package org.apache.camel.component.google.storage;
import java.util.LinkedList;
@@ -64,13 +80,8 @@ public class GoogleCloudStorageConsumer extends
ScheduledBatchPollingConsumer {
String fileName = getConfiguration().getObjectName();
String bucketName = getConfiguration().getBucketName();
- //String doneFileName = getConfiguration().getDoneFileName();
Queue<Exchange> exchanges = new LinkedList<>();
- //TODO ripristinare
- //if (!doneFileCheckPasses(bucketName, doneFileName)) {
- // exchanges = new LinkedList<>();
- //} else
if (fileName != null) {
LOG.trace("Getting object in bucket [{}] with file name [{}]...",
bucketName, fileName);
@@ -85,32 +96,6 @@ public class GoogleCloudStorageConsumer extends
ScheduledBatchPollingConsumer {
bloblist.add(blob);
}
- /*
- ListObjectsRequest.Builder listObjectsRequest =
ListObjectsRequest.builder();
- listObjectsRequest.bucket(bucketName);
- listObjectsRequest.prefix(getConfiguration().getPrefix());
- listObjectsRequest.delimiter(getConfiguration().getDelimiter());
-
- if (maxMessagesPerPoll > 0) {
- listObjectsRequest.maxKeys(maxMessagesPerPoll);
- }
- // if there was a marker from previous poll then use that to
- // continue from where we left last time
- if (marker != null) {
- LOG.trace("Resuming from marker: {}", marker);
- listObjectsRequest.marker(marker);
- }
-
- ListObjectsResponse listObjects =
getAmazonS3Client().listObjects(listObjectsRequest.build());
-
- if (listObjects.isTruncated()) {
- marker = listObjects.nextMarker();
- LOG.trace("Returned list is truncated, so setting next marker:
{}", marker);
- } else {
- // no more data so clear marker
- marker = null;
- }
- */
if (LOG.isTraceEnabled()) {
LOG.trace("Found {} objects in bucket [{}]...",
bloblist.size(), bucketName);
}
@@ -121,30 +106,6 @@ public class GoogleCloudStorageConsumer extends
ScheduledBatchPollingConsumer {
return processBatch(CastUtils.cast(exchanges));
}
- /*
- private boolean doneFileCheckPasses(String bucketName, String
doneFileName) {
- if (doneFileName == null) {
- return true;
- } else {
- return checkFileExists(bucketName, doneFileName);
- }
- }
- */
-
- /*
- private boolean checkFileExists(String bucketName, String doneFileName) {
- HeadObjectRequest.Builder headObjectsRequest =
HeadObjectRequest.builder();
- headObjectsRequest.bucket(bucketName);
- headObjectsRequest.key(doneFileName);
- try {
- getAmazonS3Client().headObject(headObjectsRequest.build());
- return true;
- } catch (NoSuchKeyException e) {
- return false;
- }
- }
- */
-
protected Queue<Exchange> createExchanges(Blob blob, String key) {
Queue<Exchange> answer = new LinkedList<>();
Exchange exchange = getEndpoint().createExchange(blob, key);
@@ -157,41 +118,18 @@ public class GoogleCloudStorageConsumer extends
ScheduledBatchPollingConsumer {
LOG.trace("Received {} messages in this poll", blobList.size());
}
- //Collection<Blob> blobs = new ArrayList<>();
Queue<Exchange> answer = new LinkedList<>();
try {
for (Blob blob : blobList) {
- /*
- Builder getRequest
- =
GetObjectRequest.builder().bucket(getConfiguration().getBucketName()).key(s3ObjectSummary.key());
- if (getConfiguration().isUseCustomerKey()) {
- if
(ObjectHelper.isNotEmpty(getConfiguration().getCustomerKeyId())) {
-
getRequest.sseCustomerKey(getConfiguration().getCustomerKeyId());
- }
- if
(ObjectHelper.isNotEmpty(getConfiguration().getCustomerKeyMD5())) {
-
getRequest.sseCustomerKeyMD5(getConfiguration().getCustomerKeyMD5());
- }
- if
(ObjectHelper.isNotEmpty(getConfiguration().getCustomerAlgorithm())) {
-
getRequest.sseCustomerAlgorithm(getConfiguration().getCustomerAlgorithm());
- }
- }*/
- //ResponseInputStream<GetObjectResponse> s3Object =
getAmazonS3Client().getObject(getRequest.build(),
ResponseTransformer.toInputStream());
-
- //if (includeS3Object(s3Object)) {
- // blobs.add(s3Object);
- Exchange exchange = getEndpoint().createExchange(blob,
blob.getBlobId().getName());
- answer.add(exchange);
- //} else {
- // If includeFolders != true and the object is not included,
it is safe to close the object here.
- // If includeFolders == true, the exchange will close the
object.
- // IOHelper.close(s3Object);
- //}
+
+ if (includeObject(blob)) {
+ Exchange exchange = getEndpoint().createExchange(blob,
blob.getBlobId().getName());
+ answer.add(exchange);
+ }
+
}
} catch (Exception e) {
LOG.warn("Error getting object due: {}", e.getMessage(), e);
- // ensure all previous gathered s3 objects are closed
- // if there was an exception creating the exchanges in this batch
- //s3Objects.forEach(IOHelper::close);
throw e;
}
@@ -206,16 +144,11 @@ public class GoogleCloudStorageConsumer extends
ScheduledBatchPollingConsumer {
*/
protected boolean includeObject(Blob blob) {
- // if (getConfiguration().isIncludeFolders()) {
- // return true;
- // } else {
- //TODO understand if the object is a directory
-
+ if (getConfiguration().isIncludeFolders()) {
+ return true;
+ }
// Config says to ignore folders/directories
- //return !Optional.of(((GetObjectResponse)
s3Object.response()).contentType()).orElse("")
- // .toLowerCase().startsWith("application/x-directory");
- return true;
- //}
+ return blob.getName().endsWith("/");
}
@Override
@@ -245,7 +178,7 @@ public class GoogleCloudStorageConsumer extends
ScheduledBatchPollingConsumer {
@Override
public String toString() {
- return "S3ConsumerOnCompletion";
+ return "ConsumerOnCompletion";
}
});
diff --git
a/components/camel-google-storage/src/main/java/org/apache/camel/component/google/storage/GoogleCloudStorageEndpoint.java
b/components/camel-google-storage/src/main/java/org/apache/camel/component/google/storage/GoogleCloudStorageEndpoint.java
index a5a3f83..028642a 100644
---
a/components/camel-google-storage/src/main/java/org/apache/camel/component/google/storage/GoogleCloudStorageEndpoint.java
+++
b/components/camel-google-storage/src/main/java/org/apache/camel/component/google/storage/GoogleCloudStorageEndpoint.java
@@ -1,6 +1,23 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
package org.apache.camel.component.google.storage;
import java.io.ByteArrayOutputStream;
+import java.util.Date;
import com.google.cloud.storage.Blob;
import com.google.cloud.storage.Bucket;
@@ -120,7 +137,6 @@ public class GoogleCloudStorageEndpoint extends
ScheduledPollEndpoint {
ByteArrayOutputStream baos = new ByteArrayOutputStream();
blob.downloadTo(baos);
message.setBody(baos.toByteArray());
-
} catch (Exception e) {
throw new RuntimeCamelException(e);
}
@@ -130,42 +146,26 @@ public class GoogleCloudStorageEndpoint extends
ScheduledPollEndpoint {
message.setHeader(GoogleCloudStorageConstants.OBJECT_NAME, key);
message.setHeader(GoogleCloudStorageConstants.BUCKET_NAME,
getConfiguration().getBucketName());
- /*
- TODO
- message.setHeader(AWS2S3Constants.E_TAG, s3Object.response().eTag());
- message.setHeader(AWS2S3Constants.LAST_MODIFIED,
s3Object.response().lastModified());
- message.setHeader(AWS2S3Constants.VERSION_ID,
s3Object.response().versionId());
- message.setHeader(AWS2S3Constants.CONTENT_TYPE,
s3Object.response().contentType());
- message.setHeader(AWS2S3Constants.CONTENT_LENGTH,
s3Object.response().contentLength());
- message.setHeader(AWS2S3Constants.CONTENT_ENCODING,
s3Object.response().contentEncoding());
- message.setHeader(AWS2S3Constants.CONTENT_DISPOSITION,
s3Object.response().contentDisposition());
- message.setHeader(AWS2S3Constants.CACHE_CONTROL,
s3Object.response().cacheControl());
- message.setHeader(AWS2S3Constants.SERVER_SIDE_ENCRYPTION,
s3Object.response().serverSideEncryption());
- message.setHeader(AWS2S3Constants.EXPIRATION_TIME,
s3Object.response().expiration());
- message.setHeader(AWS2S3Constants.REPLICATION_STATUS,
s3Object.response().replicationStatus());
- message.setHeader(AWS2S3Constants.STORAGE_CLASS,
s3Object.response().storageClass());
- message.setHeader(AWS2S3Constants.METADATA,
s3Object.response().metadata());
- */
- /*
- * If includeBody == true, it is safe to close the object here because
the S3Object
- * was consumed already. If includeBody != true, the caller is
responsible for
- * closing the stream once the body has been fully consumed or use the
autoCloseBody
- * configuration to automatically schedule the body closing at the end
of exchange.
- */
- /*
- if (configuration.isIncludeBody()) {
- IOHelper.close(s3Object);
- } else {
- if (configuration.isAutocloseBody()) {
- exchange.adapt(ExtendedExchange.class).addOnCompletion(new
SynchronizationAdapter() {
- @Override
- public void onDone(Exchange exchange) {
- IOHelper.close(s3Object);
- }
- });
- }
- }
- */
+ //OTHER METADATA
+ message.setHeader(GoogleCloudStorageConstants.CACHE_CONTROL,
blob.getCacheControl());
+
message.setHeader(GoogleCloudStorageConstants.METADATA_COMPONENT_COUNT,
blob.getComponentCount());
+ message.setHeader(GoogleCloudStorageConstants.CONTENT_DISPOSITION,
blob.getContentDisposition());
+ message.setHeader(GoogleCloudStorageConstants.CONTENT_ENCODING,
blob.getContentEncoding());
+
message.setHeader(GoogleCloudStorageConstants.METADATA_CONTENT_LANGUAGE,
blob.getContentLanguage());
+ message.setHeader(GoogleCloudStorageConstants.CONTENT_TYPE,
blob.getContentType());
+ message.setHeader(GoogleCloudStorageConstants.METADATA_CUSTOM_TIME,
blob.getCustomTime());
+ message.setHeader(GoogleCloudStorageConstants.METADATA_CRC32C_hex,
blob.getCrc32cToHexString());
+ message.setHeader(GoogleCloudStorageConstants.METADATA_ETAG,
blob.getEtag());
+ message.setHeader(GoogleCloudStorageConstants.METADATA_GENERATION,
blob.getGeneration());
+ message.setHeader(GoogleCloudStorageConstants.METADATA_BLOB_ID,
blob.getBlobId());
+ message.setHeader(GoogleCloudStorageConstants.METADATA_KMS_KEY_NAME,
blob.getKmsKeyName());
+ message.setHeader(GoogleCloudStorageConstants.CONTENT_MD5,
blob.getMd5ToHexString());
+ message.setHeader(GoogleCloudStorageConstants.METADATA_MEDIA_LINK,
blob.getMediaLink());
+ message.setHeader(GoogleCloudStorageConstants.METADATA_METAGENERATION,
blob.getMetageneration());
+ message.setHeader(GoogleCloudStorageConstants.CONTENT_LENGTH,
blob.getSize());
+ message.setHeader(GoogleCloudStorageConstants.METADATA_STORAGE_CLASS,
blob.getStorageClass());
+ message.setHeader(GoogleCloudStorageConstants.METADATA_CREATE_TIME,
blob.getCreateTime());
+ message.setHeader(GoogleCloudStorageConstants.METADATA_LAST_UPDATE,
new Date(blob.getUpdateTime()));
return exchange;
}
diff --git
a/components/camel-google-storage/src/main/java/org/apache/camel/component/google/storage/GoogleCloudStorageProducer.java
b/components/camel-google-storage/src/main/java/org/apache/camel/component/google/storage/GoogleCloudStorageProducer.java
index 37f48e2..9cf73b5 100644
---
a/components/camel-google-storage/src/main/java/org/apache/camel/component/google/storage/GoogleCloudStorageProducer.java
+++
b/components/camel-google-storage/src/main/java/org/apache/camel/component/google/storage/GoogleCloudStorageProducer.java
@@ -1,3 +1,19 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
package org.apache.camel.component.google.storage;
import java.io.ByteArrayInputStream;
@@ -100,7 +116,7 @@ public class GoogleCloudStorageProducer extends
DefaultProducer {
} else {
is = exchange.getIn().getMandatoryBody(InputStream.class);
baos = determineLengthInputStream(is);
- //TODO improve here
+
if (objectMetadata.containsKey(Exchange.CONTENT_LENGTH)) {
if (objectMetadata.get("Content-Length").equals("0")
&&
ObjectHelper.isEmpty(exchange.getProperty(Exchange.CONTENT_LENGTH))) {
@@ -137,12 +153,9 @@ public class GoogleCloudStorageProducer extends
DefaultProducer {
Message message = getMessageForResponse(exchange);
message.setBody(createdBlob);
+ IOHelper.close(baos);
IOHelper.close(is);
- /*
- //TODO improve here
- * if (getConfiguration().isDeleteAfterWrite() && filePayload != null)
{
- * FileUtil.deleteFile(filePayload); }
- */
+
}
private ByteArrayOutputStream determineLengthInputStream(InputStream is)
throws IOException {
@@ -307,17 +320,10 @@ public class GoogleCloudStorageProducer extends
DefaultProducer {
}
private String determineBucketName(Exchange exchange) {
- //String bucketName =
exchange.getIn().getHeader(GoogleCloudStorageConstants.BUCKET_NAME,
String.class);
-
- //if (ObjectHelper.isEmpty(bucketName)) {
String bucketName = getConfiguration().getBucketName();
- // LOG.trace("Google Cloud Storage Bucket name header is missing,
using default one [{}]", bucketName);
- //}
-
if (bucketName == null) {
throw new IllegalArgumentException("Bucket name is missing or not
configured.");
}
-
return bucketName;
}
diff --git
a/core/camel-componentdsl/src/generated/java/org/apache/camel/builder/component/dsl/GoogleStorageComponentBuilderFactory.java
b/core/camel-componentdsl/src/generated/java/org/apache/camel/builder/component/dsl/GoogleStorageComponentBuilderFactory.java
index 4bb5f79..4c8a69d 100644
---
a/core/camel-componentdsl/src/generated/java/org/apache/camel/builder/component/dsl/GoogleStorageComponentBuilderFactory.java
+++
b/core/camel-componentdsl/src/generated/java/org/apache/camel/builder/component/dsl/GoogleStorageComponentBuilderFactory.java
@@ -190,6 +190,24 @@ public interface GoogleStorageComponentBuilderFactory {
return this;
}
/**
+ * If it is true, the folders/directories will be consumed. If it is
+ * false, they will be ignored, and Exchanges will not be created for
+ * those.
+ *
+ * The option is a: <code>boolean</code> type.
+ *
+ * Default: true
+ * Group: consumer
+ *
+ * @param includeFolders the value to set
+ * @return the dsl builder
+ */
+ default GoogleStorageComponentBuilder includeFolders(
+ boolean includeFolders) {
+ doSetProperty("includeFolders", includeFolders);
+ return this;
+ }
+ /**
* Move objects from the origin bucket to a different bucket after they
* have been retrieved. To accomplish the operation the
* destinationBucket option must be set. The copy bucket operation is
@@ -317,6 +335,7 @@ public interface GoogleStorageComponentBuilderFactory {
case "deleteAfterRead":
getOrCreateConfiguration((GoogleCloudStorageComponent)
component).setDeleteAfterRead((boolean) value); return true;
case "destinationBucket":
getOrCreateConfiguration((GoogleCloudStorageComponent)
component).setDestinationBucket((java.lang.String) value); return true;
case "includeBody":
getOrCreateConfiguration((GoogleCloudStorageComponent)
component).setIncludeBody((boolean) value); return true;
+ case "includeFolders":
getOrCreateConfiguration((GoogleCloudStorageComponent)
component).setIncludeFolders((boolean) value); return true;
case "moveAfterRead":
getOrCreateConfiguration((GoogleCloudStorageComponent)
component).setMoveAfterRead((boolean) value); return true;
case "lazyStartProducer": ((GoogleCloudStorageComponent)
component).setLazyStartProducer((boolean) value); return true;
case "objectName":
getOrCreateConfiguration((GoogleCloudStorageComponent)
component).setObjectName((java.lang.String) value); return true;
diff --git
a/core/camel-endpointdsl/src/generated/java/org/apache/camel/builder/endpoint/dsl/GoogleCloudStorageEndpointBuilderFactory.java
b/core/camel-endpointdsl/src/generated/java/org/apache/camel/builder/endpoint/dsl/GoogleCloudStorageEndpointBuilderFactory.java
index 87fe200d..804ccf7 100644
---
a/core/camel-endpointdsl/src/generated/java/org/apache/camel/builder/endpoint/dsl/GoogleCloudStorageEndpointBuilderFactory.java
+++
b/core/camel-endpointdsl/src/generated/java/org/apache/camel/builder/endpoint/dsl/GoogleCloudStorageEndpointBuilderFactory.java
@@ -265,6 +265,43 @@ public interface GoogleCloudStorageEndpointBuilderFactory {
return this;
}
/**
+ * If it is true, the folders/directories will be consumed. If it is
+ * false, they will be ignored, and Exchanges will not be created for
+ * those.
+ *
+ * The option is a: <code>boolean</code> type.
+ *
+ * Default: true
+ * Group: consumer
+ *
+ * @param includeFolders the value to set
+ * @return the dsl builder
+ */
+ default GoogleCloudStorageEndpointConsumerBuilder includeFolders(
+ boolean includeFolders) {
+ doSetProperty("includeFolders", includeFolders);
+ return this;
+ }
+ /**
+ * If it is true, the folders/directories will be consumed. If it is
+ * false, they will be ignored, and Exchanges will not be created for
+ * those.
+ *
+ * The option will be converted to a <code>boolean</code>
+ * type.
+ *
+ * Default: true
+ * Group: consumer
+ *
+ * @param includeFolders the value to set
+ * @return the dsl builder
+ */
+ default GoogleCloudStorageEndpointConsumerBuilder includeFolders(
+ String includeFolders) {
+ doSetProperty("includeFolders", includeFolders);
+ return this;
+ }
+ /**
* Move objects from the origin bucket to a different bucket after they
* have been retrieved. To accomplish the operation the
* destinationBucket option must be set. The copy bucket operation is
diff --git a/docs/components/modules/ROOT/pages/google-storage-component.adoc
b/docs/components/modules/ROOT/pages/google-storage-component.adoc
index 9bc1d1b..447289a 100644
--- a/docs/components/modules/ROOT/pages/google-storage-component.adoc
+++ b/docs/components/modules/ROOT/pages/google-storage-component.adoc
@@ -16,7 +16,7 @@
include::{cq-version}@camel-quarkus:ROOT:partial$reference/components/google-sto
*{component-header}*
-The Google Storage component provides access to
https://cloud.google.com/storage/[Cloud Storage] via
+The Google Storage component provides access to
https://cloud.google.com/storage/[Google Cloud Storage] via
the https://github.com/googleapis/java-storage[google java storage library].
Maven users will need to add the following dependency to their pom.xml
@@ -48,26 +48,43 @@ Google security credentials can be set explicitly by
providing the path to the G
export GOOGLE_APPLICATION_CREDENTIALS="/home/user/Downloads/my-key.json"
--------------------------------------------------------
-or directly through the component endpoint
+or for windows:
+
[source,text]
--------------------------------------------------------
-from("google-storage://myCamelBucket?serviceAccountKey=/home/user/Downloads/my-key.json")
+$Env:GOOGLE_APPLICATION_CREDENTIALS = "/home/user/Downloads/my-key.json"
--------------------------------------------------------
-
+or directly through the component endpoint
+[source,text]
+--------------------------------------------------------
+String endpoint =
"google-storage://myCamelBucket?serviceAccountKey=/home/user/Downloads/my-key.json";
+--------------------------------------------------------
== URI Format
[source,text]
--------------------------------------------------------
- google-storage://bucketName?[options]
+google-storage://bucketNameOrArn?[options]
--------------------------------------------------------
+The bucket will be created if it don't already exists. +
+ You can append query options to the URI in the following format,
+?options=value&option2=value&...
+
+For example in order to read file `hello.txt` from bucket `helloBucket`, use
the following snippet:
+
+[source,java]
+--------------------------------------------------------------------------------
+from("google-storage://myCamelBucket?serviceAccountKey=/home/user/Downloads/my-key.json&prefix=hello.txt")
+ .to("file:/var/downloaded");
+--------------------------------------------------------------------------------
+
-== Options
+== URI Options
// component options: START
-The Google Storage component supports 13 options, which are listed below.
+The Google Storage component supports 14 options, which are listed below.
@@ -82,6 +99,7 @@ The Google Storage component supports 13 options, which are
listed below.
| *deleteAfterRead* (consumer) | Delete objects from the bucket after they
have been retrieved. The delete is only performed if the Exchange is committed.
If a rollback occurs, the object is not deleted. If this option is false, then
the same objects will be retrieve over and over again on the polls. | true |
boolean
| *destinationBucket* (consumer) | Define the destination bucket where an
object must be moved when moveAfterRead is set to true. | | String
| *includeBody* (consumer) | If it is true, the Object exchange will be
consumed and put into the body and closed. If false the Object stream will be
put raw into the body and the headers will be set with the object metadata. |
true | boolean
+| *includeFolders* (consumer) | If it is true, the folders/directories will be
consumed. If it is false, they will be ignored, and Exchanges will not be
created for those | true | boolean
| *moveAfterRead* (consumer) | Move objects from the origin bucket to a
different bucket after they have been retrieved. To accomplish the operation
the destinationBucket option must be set. The copy bucket operation is only
performed if the Exchange is committed. If a rollback occurs, the object is not
moved. | false | boolean
| *lazyStartProducer* (producer) | Whether the producer should be started lazy
(on the first message). By starting lazy you can use this to allow CamelContext
and routes to startup in situations where a producer may otherwise fail during
starting and cause the route to fail being started. By deferring this startup
to be lazy then the startup failure can be handled during routing messages via
Camel's routing error handlers. Beware that when the first message is processed
then creating and [...]
| *objectName* (producer) | Object name | | String
@@ -109,7 +127,7 @@ with the following path and query parameters:
|===
-=== Query Parameters (29 parameters):
+=== Query Parameters (30 parameters):
[width="100%",cols="2,5,^1,2",options="header"]
@@ -122,6 +140,7 @@ with the following path and query parameters:
| *deleteAfterRead* (consumer) | Delete objects from the bucket after they
have been retrieved. The delete is only performed if the Exchange is committed.
If a rollback occurs, the object is not deleted. If this option is false, then
the same objects will be retrieve over and over again on the polls. | true |
boolean
| *destinationBucket* (consumer) | Define the destination bucket where an
object must be moved when moveAfterRead is set to true. | | String
| *includeBody* (consumer) | If it is true, the Object exchange will be
consumed and put into the body and closed. If false the Object stream will be
put raw into the body and the headers will be set with the object metadata. |
true | boolean
+| *includeFolders* (consumer) | If it is true, the folders/directories will be
consumed. If it is false, they will be ignored, and Exchanges will not be
created for those | true | boolean
| *moveAfterRead* (consumer) | Move objects from the origin bucket to a
different bucket after they have been retrieved. To accomplish the operation
the destinationBucket option must be set. The copy bucket operation is only
performed if the Exchange is committed. If a rollback occurs, the object is not
moved. | false | boolean
| *sendEmptyMessageWhenIdle* (consumer) | If the polling consumer did not poll
any files, you can enable this option to send an empty message (no body)
instead. | false | boolean
| *exceptionHandler* (consumer) | To let the consumer use a custom
ExceptionHandler. Notice if the option bridgeErrorHandler is enabled then this
option is not in use. By default the consumer will deal with exceptions, that
will be logged at WARN or ERROR level and ignored. | | ExceptionHandler
