This is an automated email from the ASF dual-hosted git repository. acosentino pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/camel.git
commit 07c12d6ee478f2f0198ace2c2c514a8016b43d63 Author: Raffaele Marcello <marcelloraffa...@gmail.com> AuthorDate: Sat Mar 13 00:12:17 2021 +0100 CAMEL-15963 Create a Google Cloud Functions component --- .../catalog/docs/google-functions-component.adoc | 211 ++++++--------------- .../src/main/docs/google-functions-component.adoc | 211 ++++++--------------- .../functions/integration/IntegrationTest.java | 3 +- .../ROOT/pages/google-functions-component.adoc | 211 ++++++--------------- 4 files changed, 190 insertions(+), 446 deletions(-) diff --git a/catalog/camel-catalog/src/generated/resources/org/apache/camel/catalog/docs/google-functions-component.adoc b/catalog/camel-catalog/src/generated/resources/org/apache/camel/catalog/docs/google-functions-component.adoc index 956c8b9..cad2ef2 100644 --- a/catalog/camel-catalog/src/generated/resources/org/apache/camel/catalog/docs/google-functions-component.adoc +++ b/catalog/camel-catalog/src/generated/resources/org/apache/camel/catalog/docs/google-functions-component.adoc @@ -57,11 +57,11 @@ export GOOGLE_APPLICATION_CREDENTIALS="/home/user/Downloads/my-key.json" [source,text] -------------------------------------------------------- -google-functions://functionName?[options] +google-functions://functionName[?options] -------------------------------------------------------- You can append query options to the URI in the following format, -?options=value&option2=value&... +`?options=value&option2=value&...` For example in order to call the function `myCamelFunction` from the project `myProject` and location `us-central1`, use the following snippet: @@ -130,29 +130,10 @@ with the following path and query parameters: [width="100%",cols="10%,10%,80%",options="header",] |======================================================================= |Header |Type |Description - -|`CamelGoogleCloudStorageBucketName` |`String` |The bucket Name which this object will be stored or which will be used for the current operation - -|`CamelGoogleCloudStorageObjectName` |`String` |The object Name which will be used for the current operation - -|`CamelGoogleCloudStorageDestinationBucketName` |`String` |The bucket Destination Name which will be used for the current operation - -|`CamelGoogleCloudStorageDestinationObjectName` |`String` |The object Destination Name which will be used for the current operation - -|`CamelGoogleCloudStorageContentLength` |`Long` |The content length of this object. - -|`CamelGoogleCloudStorageContentType` |`String` |The content type of this object. - - -|`CamelGoogleCloudStorageContentDisposition` |`String` |The content disposition of this object. - -|`CamelGoogleCloudStorageContentEncoding` |`String` |The content encoding of this object. - -|`CamelGoogleCloudStorageContentMd5` |`String` |The md5 checksum of this object. - -|`CamelGoogleCloudStorageOperation` |`String` |The operation to perform. Permitted values are copyObject, listObjects, deleteObject, deleteBucket, listBuckets, getObject, createDownloadLink - -|`CamelGoogleCloudStorageDownloadLinkExpirationTime` |`Long` |The time in millisecond the download link will be valid. +|`GoogleCloudFunctionsOperation` |`String` |The operation to perform. Permitted values are listFunctions, getFunction, callFunction, generateDownloadUrl, generateUploadUrl, createFunction, updateFunction, deleteFunction +|`GoogleCloudFunctionsEntryPoint` |`String` |The name of the function (as defined in source code) that will be executed. Used for createFunction operation. +|`GoogleCloudFunctionsRuntime` |`String` |The runtime in which to run the function. Possible values are `nodejs10`, `nodejs12`, `nodejs14`, `python37`, `python38`, `python39`, `go111`, `go113`, `java11`, `dotnet3`, `ruby26`, `nodejs6`, `nodejs8`. Used for createFunction operation. +|`GoogleCloudFunctionsSourceArchiveUrl` |`String` |The Google Cloud Storage URL, starting with `gs://`, pointing to the zip archive which contains the function. Used for createFunction operation. |======================================================================= @@ -161,195 +142,129 @@ with the following path and query parameters: [width="100%",cols="10%,10%,80%",options="header",] |======================================================================= |Header |Type |Description - -|`CamelGoogleCloudStorageETag` |`String` |The ETag value for the newly uploaded object. - -|======================================================================= - -=== Message headers set by the Google Storage Consumer - -[width="100%",cols="10%,10%,80%",options="header",] -|======================================================================= -|Header |Type |Description -|`CamelGoogleCloudStorageBucketName` |`String` |The bucket Name which this object will be stored or which will be used for the current operation -|`CamelGoogleCloudStorageObjectName` |`String` |The object Name which will be used for the current operation -| `CamelGoogleCloudStorageCacheControl` |`String` | The Cache-Control metadata can specify two different aspects of how data is served from Cloud Storage: whether the data can be cached and whether the data can be transformed -| `CamelGoogleCloudStorageComponentCount` |`String` | The component count of this object -| `CamelGoogleCloudStorageContentDisposition` |`String` |The content disposition of this object. -| `CamelGoogleCloudStorageContentEncoding` |`String` |The content encoding of this object. -| `CamelGoogleCloudStorageContentLanguage` |`String` | The Content-Language metadata indicates the language(s) that the object is intended for. -| `CamelGoogleCloudStorageContentType` |`String` |The content type of this object. -| `CamelGoogleCloudStorageCustomTime` |`String` | The Custom-Time metadata is a user-specified date and time represented in the RFC 3339 format YYYY-MM-DD'T'HH:MM:SS.SS'Z' or YYYY-MM-DD'T'HH:MM:SS'Z' when milliseconds are zero. This metadata is typically set in order to use the DaysSinceCustomTime condition in Object Lifecycle Management. -| `CamelGoogleCloudStorageCrc32cHex` |`String` | The CRC32c of the object -| `CamelGoogleCloudStorageETag` |`String` | The ETag for the Object. -| `CamelGoogleCloudStorageGeneration` |`String` | Is the generation number of the object for which you are retrieving information. -| `CamelGoogleCloudStorageBlobId` |`String` | The blob id of the object -| `CamelGoogleCloudStorageKmsKeyName` |`String` | The KMS key name -| `CamelGoogleCloudStorageContentMd5` |`String` |The md5 checksum of this object. -| `CamelGoogleCloudStorageMediaLink` |`String` | The media link -| `CamelGoogleCloudStorageMetageneration` | `String` | The metageneration of the object -| `CamelGoogleCloudStorageContentLength` |`Long` |The content length of this object. -| `CamelGoogleCloudStorageStorageClass` | `String` | The functions class of the object -| `CamelGoogleCloudStorageCreateTime` |`String` | The creation time of the object -| `CamelGoogleCloudStorageLastUpdate` |`String` | The last update of the object +|`GoogleCloudFunctionsResponseObject` |`Object` |The response object resulting from the Google Functions Client invocation |======================================================================= === Google Functions Producer operations Google Functions component provides the following operation on the producer side: -- copyObject -- listObjects -- deleteObject -- deleteBucket -- listBuckets -- getObject -- createDownloadLink +- listFunctions +- getFunction +- callFunction +- generateDownloadUrl +- generateUploadUrl +- createFunction +- updateFunction +- deleteFunction -If you don't specify an operation explicitly the producer will a file upload. +If you don't specify an operation by default the producer will use the `callFunction` operation. === Advanced component configuration -If you need to have more control over the `functionsClient` instance configuration, you can create your own instance and refer to it in your Camel google-functions component configuration: +If you need to have more control over the `client` instance configuration, you can create your own instance and refer to it in your Camel google-functions component configuration: [source,java] -------------------------------------------------------------------------------- -from("google-functions://myCamelFunction?functionsClient=#client") +from("google-functions://myCamelFunction?client=#myClient") .to("mock:result"); -------------------------------------------------------------------------------- === Google Functions Producer Operation examples -- File Upload: This operation will upload a file to the Google Functions based on the body content +- ListFunctions: This operation invoke the Google Functions client and get the list of cloud Functions [source,java] -------------------------------------------------------------------------------- -//upload a file -byte[] payload = "Camel rocks!".getBytes(); -ByteArrayInputStream bais = new ByteArrayInputStream(payload); -from("direct:start") -.process( exchange -> { - exchange.getIn().setHeader(GoogleCloudStorageConstants.OBJECT_NAME, "camel.txt"); - exchange.getIn().setBody(bais); -}) -.to("google-functions://myCamelFunction?serviceAccountKey=/home/user/Downloads/my-key.json") -.log("uploaded file object:${header.CamelGoogleCloudStorageObjectName}, body:${body}"); --------------------------------------------------------------------------------- - -This operation will upload the file camel.txt with the content "Camel rocks!" in the myCamelBucket bucket -- CopyObject: this operation copy an object from one bucket to a different one +//list functions +from("direct:start") +.to("google-functions://myCamelFunction?serviceAccountKey=/home/user/Downloads/my-key.json&project=myProject&location=us-central1&operation=listFunctions") +.log("body:${body}") -[source,java] --------------------------------------------------------------------------------- - from("direct:start").process( exchange -> { - exchange.getIn().setHeader(GoogleCloudStorageConstants.OPERATION, GoogleCloudStorageOperations.copyObject); - exchange.getIn().setHeader(GoogleCloudStorageConstants.OBJECT_NAME, "camel.txt" ); - exchange.getIn().setHeader(GoogleCloudStorageConstants.DESTINATION_BUCKET_NAME, "myCamelBucket_dest"); - exchange.getIn().setHeader(GoogleCloudStorageConstants.DESTINATION_OBJECT_NAME, "camel_copy.txt"); - }) - .to("google-functions://myCamelFunction?serviceAccountKey=/home/user/Downloads/my-key.json") - .to("mock:result"); -------------------------------------------------------------------------------- -This operation will copy the object with the name expressed in the header DESTINATION_OBJECT_NAME to the DESTINATION_BUCKET_NAME bucket, from the bucket myCamelBucket. +This operation will get the list of cloud functions for the project `myProject` and location `us-central1`. -- DeleteObject: this operation deletes an object from a bucket +- GetFunction: this operation get the Cloud Functions object [source,java] -------------------------------------------------------------------------------- - from("direct:start").process( exchange -> { - exchange.getIn().setHeader(GoogleCloudStorageConstants.OPERATION, GoogleCloudStorageOperations.deleteObject); - exchange.getIn().setHeader(GoogleCloudStorageConstants.OBJECT_NAME, "camel.txt" ); - }) - .to("google-functions://myCamelFunction?serviceAccountKey=/home/user/Downloads/my-key.json") - .to("mock:result"); +//get function +from("direct:start") +.to("google-functions://myCamelFunction?serviceAccountKey=/home/user/Downloads/my-key.json&project=myProject&location=us-central1&operation=getFunction") +.log("body:${body}") +.to("mock:result"); -------------------------------------------------------------------------------- -This operation will delete the object from the bucket myCamelBucket. +This operation will get the `CloudFunction` object for the project `myProject`, location `us-central1` and functionName `myCamelFunction`. -- ListBuckets: this operation list the buckets for this account in this region +- CallFunction: this operation call the function using an HTTP request [source,java] -------------------------------------------------------------------------------- +//call function from("direct:start") -.to("google-functions://myCamelFunction?serviceAccountKey=/home/user/Downloads/my-key.json&operation=listBuckets") +.to("google-functions://myCamelFunction?serviceAccountKey=/home/user/Downloads/my-key.json&project=myProject&location=us-central1&operation=callFunction") +.log("body:${body}") .to("mock:result"); -------------------------------------------------------------------------------- -This operation will list the buckets for this account. - -- DeleteBucket: this operation delete the bucket specified as URI parameter or header +- GenerateDownloadUrl: this operation generate the signed URL for downloading deployed function source code. [source,java] -------------------------------------------------------------------------------- -from("direct:start") -.to("google-functions://myCamelFunction?serviceAccountKey=/home/user/Downloads/my-key.json&operation=deleteBucket") +//generate download url +from("direct:start").to("google-functions://myCamelFunction?serviceAccountKey=/home/user/Downloads/my-key.json&project=myProject&location=us-central1&operation=generateDownloadUrl") +.log("body:${body}") .to("mock:result"); -------------------------------------------------------------------------------- -This operation will delete the bucket myCamelBucket. - -- ListObjects: this operation list object in a specific bucket +- GenerateUploadUrl: this operation generate a signed URL for uploading a function source code. [source,java] -------------------------------------------------------------------------------- -from("direct:start") -.to("google-functions://myCamelFunction?serviceAccountKey=/home/user/Downloads/my-key.json&operation=listObjects") +from("direct:start").to("google-functions://myCamelFunction?serviceAccountKey=/home/user/Downloads/my-key.json&project=myProject&location=us-central1&operation=generateUploadUrl") +.log("body:${body}") .to("mock:result"); -------------------------------------------------------------------------------- -This operation will list the objects in the myCamelBucket bucket. - -- GetObject: this operation get a single object in a specific bucket +- createFunction: this operation creates a new function. [source,java] -------------------------------------------------------------------------------- + from("direct:start") -.process( exchange -> { - exchange.getIn().setHeader(GoogleCloudStorageConstants.OBJECT_NAME, "camel.txt"); -}) -.to("google-functions://myCamelFunction?serviceAccountKey=/home/user/Downloads/my-key.json&operation=getObject") +.process(exchange -> { + exchange.getIn().setHeader(GoogleCloudFunctionsConstants.ENTRY_POINT, "com.example.Example"); + exchange.getIn().setHeader(GoogleCloudFunctionsConstants.RUNTIME, "java11"); + exchange.getIn().setHeader(GoogleCloudFunctionsConstants.SOURCE_ARCHIVE_URL, "gs://myBucket/source.zip"); +}).to("google-functions://myCamelFunction?serviceAccountKey=/home/user/Downloads/my-key.json&project=myProject&location=us-central1&operation=createFunction") +.log("body:${body}") .to("mock:result"); -------------------------------------------------------------------------------- -This operation will return an Blob objct instance related to the OBJECT_NAME object in myCamelBucket bucket. - -- CreateDownloadLink: this operation will return a download link +- updateFunction: this operation updates existing function. [source,java] -------------------------------------------------------------------------------- from("direct:start") -.process( exchange -> { - exchange.getIn().setHeader(GoogleCloudStorageConstants.OBJECT_NAME, "camel.txt" ); - exchange.getIn().setHeader(GoogleCloudStorageConstants.DOWNLOAD_LINK_EXPIRATION_TIME, 86400000L); //1 day -}) -.to("google-functions://myCamelFunction?serviceAccountKey=/home/user/Downloads/my-key.json&operation=createDownloadLink") +.process(exchange -> { + UpdateFunctionRequest request = UpdateFunctionRequest.newBuilder() + .setFunction(CloudFunction.newBuilder().build()) + .setUpdateMask(FieldMask.newBuilder().build()).build(); + exchange.getIn().setBody(request); +}).to("google-functions://myCamelFunction?serviceAccountKey=/home/user/Downloads/my-key.json&project=myProject&location=us-central1&operation=updateFunction&pojoRequest=true") +.log("body:${body}") .to("mock:result"); -------------------------------------------------------------------------------- -This operation will return a download link url for the file OBJECT_NAME in the bucket myCamelBucket. It's possible to specify the expiration time for the created link through the header DOWNLOAD_LINK_EXPIRATION_TIME. If not specified, by default it is 5 minutes. - -== Bucket Autocreation - -With the option `autoCreateBucket` users are able to avoid the autocreation of a Bucket in case it doesn't exist. The default for this option is `true`. -If set to false any operation on a not-existent bucket won't be successful and an error will be returned. - -== MoveAfterRead consumer option - -In addition to deleteAfterRead it has been added another option, moveAfterRead. With this option enabled the consumed object will be moved to a target destinationBucket instead of being only deleted. -This will require specifying the destinationBucket option. As example: +- deleteFunction: this operation Deletes a function with the given name from the specified project. [source,java] -------------------------------------------------------------------------------- - from("google-functions://myCamelFunction?serviceAccountKey=/home/user/Downloads/my-key.json" - + "&autoCreateBucket=true" - + "&destinationBucket=myCamelProcessedBucket" - + "&moveAfterRead=true" - + "&deleteAfterRead=true" - + "&includeBody=true" - ) - .to("mock:result"); +from("direct:start") +.to("google-functions://myCamelFunction?serviceAccountKey=/home/user/Downloads/my-key.json&project=myProject&location=us-central1&operation=deleteFunction") +.log("body:${body}") +.to("mock:result"); -------------------------------------------------------------------------------- - -In this case the objects consumed will be moved to myCamelProcessedBucket bucket and deleted from the original one (because of deleteAfterRead). diff --git a/components/camel-google-functions/src/main/docs/google-functions-component.adoc b/components/camel-google-functions/src/main/docs/google-functions-component.adoc index 956c8b9..cad2ef2 100644 --- a/components/camel-google-functions/src/main/docs/google-functions-component.adoc +++ b/components/camel-google-functions/src/main/docs/google-functions-component.adoc @@ -57,11 +57,11 @@ export GOOGLE_APPLICATION_CREDENTIALS="/home/user/Downloads/my-key.json" [source,text] -------------------------------------------------------- -google-functions://functionName?[options] +google-functions://functionName[?options] -------------------------------------------------------- You can append query options to the URI in the following format, -?options=value&option2=value&... +`?options=value&option2=value&...` For example in order to call the function `myCamelFunction` from the project `myProject` and location `us-central1`, use the following snippet: @@ -130,29 +130,10 @@ with the following path and query parameters: [width="100%",cols="10%,10%,80%",options="header",] |======================================================================= |Header |Type |Description - -|`CamelGoogleCloudStorageBucketName` |`String` |The bucket Name which this object will be stored or which will be used for the current operation - -|`CamelGoogleCloudStorageObjectName` |`String` |The object Name which will be used for the current operation - -|`CamelGoogleCloudStorageDestinationBucketName` |`String` |The bucket Destination Name which will be used for the current operation - -|`CamelGoogleCloudStorageDestinationObjectName` |`String` |The object Destination Name which will be used for the current operation - -|`CamelGoogleCloudStorageContentLength` |`Long` |The content length of this object. - -|`CamelGoogleCloudStorageContentType` |`String` |The content type of this object. - - -|`CamelGoogleCloudStorageContentDisposition` |`String` |The content disposition of this object. - -|`CamelGoogleCloudStorageContentEncoding` |`String` |The content encoding of this object. - -|`CamelGoogleCloudStorageContentMd5` |`String` |The md5 checksum of this object. - -|`CamelGoogleCloudStorageOperation` |`String` |The operation to perform. Permitted values are copyObject, listObjects, deleteObject, deleteBucket, listBuckets, getObject, createDownloadLink - -|`CamelGoogleCloudStorageDownloadLinkExpirationTime` |`Long` |The time in millisecond the download link will be valid. +|`GoogleCloudFunctionsOperation` |`String` |The operation to perform. Permitted values are listFunctions, getFunction, callFunction, generateDownloadUrl, generateUploadUrl, createFunction, updateFunction, deleteFunction +|`GoogleCloudFunctionsEntryPoint` |`String` |The name of the function (as defined in source code) that will be executed. Used for createFunction operation. +|`GoogleCloudFunctionsRuntime` |`String` |The runtime in which to run the function. Possible values are `nodejs10`, `nodejs12`, `nodejs14`, `python37`, `python38`, `python39`, `go111`, `go113`, `java11`, `dotnet3`, `ruby26`, `nodejs6`, `nodejs8`. Used for createFunction operation. +|`GoogleCloudFunctionsSourceArchiveUrl` |`String` |The Google Cloud Storage URL, starting with `gs://`, pointing to the zip archive which contains the function. Used for createFunction operation. |======================================================================= @@ -161,195 +142,129 @@ with the following path and query parameters: [width="100%",cols="10%,10%,80%",options="header",] |======================================================================= |Header |Type |Description - -|`CamelGoogleCloudStorageETag` |`String` |The ETag value for the newly uploaded object. - -|======================================================================= - -=== Message headers set by the Google Storage Consumer - -[width="100%",cols="10%,10%,80%",options="header",] -|======================================================================= -|Header |Type |Description -|`CamelGoogleCloudStorageBucketName` |`String` |The bucket Name which this object will be stored or which will be used for the current operation -|`CamelGoogleCloudStorageObjectName` |`String` |The object Name which will be used for the current operation -| `CamelGoogleCloudStorageCacheControl` |`String` | The Cache-Control metadata can specify two different aspects of how data is served from Cloud Storage: whether the data can be cached and whether the data can be transformed -| `CamelGoogleCloudStorageComponentCount` |`String` | The component count of this object -| `CamelGoogleCloudStorageContentDisposition` |`String` |The content disposition of this object. -| `CamelGoogleCloudStorageContentEncoding` |`String` |The content encoding of this object. -| `CamelGoogleCloudStorageContentLanguage` |`String` | The Content-Language metadata indicates the language(s) that the object is intended for. -| `CamelGoogleCloudStorageContentType` |`String` |The content type of this object. -| `CamelGoogleCloudStorageCustomTime` |`String` | The Custom-Time metadata is a user-specified date and time represented in the RFC 3339 format YYYY-MM-DD'T'HH:MM:SS.SS'Z' or YYYY-MM-DD'T'HH:MM:SS'Z' when milliseconds are zero. This metadata is typically set in order to use the DaysSinceCustomTime condition in Object Lifecycle Management. -| `CamelGoogleCloudStorageCrc32cHex` |`String` | The CRC32c of the object -| `CamelGoogleCloudStorageETag` |`String` | The ETag for the Object. -| `CamelGoogleCloudStorageGeneration` |`String` | Is the generation number of the object for which you are retrieving information. -| `CamelGoogleCloudStorageBlobId` |`String` | The blob id of the object -| `CamelGoogleCloudStorageKmsKeyName` |`String` | The KMS key name -| `CamelGoogleCloudStorageContentMd5` |`String` |The md5 checksum of this object. -| `CamelGoogleCloudStorageMediaLink` |`String` | The media link -| `CamelGoogleCloudStorageMetageneration` | `String` | The metageneration of the object -| `CamelGoogleCloudStorageContentLength` |`Long` |The content length of this object. -| `CamelGoogleCloudStorageStorageClass` | `String` | The functions class of the object -| `CamelGoogleCloudStorageCreateTime` |`String` | The creation time of the object -| `CamelGoogleCloudStorageLastUpdate` |`String` | The last update of the object +|`GoogleCloudFunctionsResponseObject` |`Object` |The response object resulting from the Google Functions Client invocation |======================================================================= === Google Functions Producer operations Google Functions component provides the following operation on the producer side: -- copyObject -- listObjects -- deleteObject -- deleteBucket -- listBuckets -- getObject -- createDownloadLink +- listFunctions +- getFunction +- callFunction +- generateDownloadUrl +- generateUploadUrl +- createFunction +- updateFunction +- deleteFunction -If you don't specify an operation explicitly the producer will a file upload. +If you don't specify an operation by default the producer will use the `callFunction` operation. === Advanced component configuration -If you need to have more control over the `functionsClient` instance configuration, you can create your own instance and refer to it in your Camel google-functions component configuration: +If you need to have more control over the `client` instance configuration, you can create your own instance and refer to it in your Camel google-functions component configuration: [source,java] -------------------------------------------------------------------------------- -from("google-functions://myCamelFunction?functionsClient=#client") +from("google-functions://myCamelFunction?client=#myClient") .to("mock:result"); -------------------------------------------------------------------------------- === Google Functions Producer Operation examples -- File Upload: This operation will upload a file to the Google Functions based on the body content +- ListFunctions: This operation invoke the Google Functions client and get the list of cloud Functions [source,java] -------------------------------------------------------------------------------- -//upload a file -byte[] payload = "Camel rocks!".getBytes(); -ByteArrayInputStream bais = new ByteArrayInputStream(payload); -from("direct:start") -.process( exchange -> { - exchange.getIn().setHeader(GoogleCloudStorageConstants.OBJECT_NAME, "camel.txt"); - exchange.getIn().setBody(bais); -}) -.to("google-functions://myCamelFunction?serviceAccountKey=/home/user/Downloads/my-key.json") -.log("uploaded file object:${header.CamelGoogleCloudStorageObjectName}, body:${body}"); --------------------------------------------------------------------------------- - -This operation will upload the file camel.txt with the content "Camel rocks!" in the myCamelBucket bucket -- CopyObject: this operation copy an object from one bucket to a different one +//list functions +from("direct:start") +.to("google-functions://myCamelFunction?serviceAccountKey=/home/user/Downloads/my-key.json&project=myProject&location=us-central1&operation=listFunctions") +.log("body:${body}") -[source,java] --------------------------------------------------------------------------------- - from("direct:start").process( exchange -> { - exchange.getIn().setHeader(GoogleCloudStorageConstants.OPERATION, GoogleCloudStorageOperations.copyObject); - exchange.getIn().setHeader(GoogleCloudStorageConstants.OBJECT_NAME, "camel.txt" ); - exchange.getIn().setHeader(GoogleCloudStorageConstants.DESTINATION_BUCKET_NAME, "myCamelBucket_dest"); - exchange.getIn().setHeader(GoogleCloudStorageConstants.DESTINATION_OBJECT_NAME, "camel_copy.txt"); - }) - .to("google-functions://myCamelFunction?serviceAccountKey=/home/user/Downloads/my-key.json") - .to("mock:result"); -------------------------------------------------------------------------------- -This operation will copy the object with the name expressed in the header DESTINATION_OBJECT_NAME to the DESTINATION_BUCKET_NAME bucket, from the bucket myCamelBucket. +This operation will get the list of cloud functions for the project `myProject` and location `us-central1`. -- DeleteObject: this operation deletes an object from a bucket +- GetFunction: this operation get the Cloud Functions object [source,java] -------------------------------------------------------------------------------- - from("direct:start").process( exchange -> { - exchange.getIn().setHeader(GoogleCloudStorageConstants.OPERATION, GoogleCloudStorageOperations.deleteObject); - exchange.getIn().setHeader(GoogleCloudStorageConstants.OBJECT_NAME, "camel.txt" ); - }) - .to("google-functions://myCamelFunction?serviceAccountKey=/home/user/Downloads/my-key.json") - .to("mock:result"); +//get function +from("direct:start") +.to("google-functions://myCamelFunction?serviceAccountKey=/home/user/Downloads/my-key.json&project=myProject&location=us-central1&operation=getFunction") +.log("body:${body}") +.to("mock:result"); -------------------------------------------------------------------------------- -This operation will delete the object from the bucket myCamelBucket. +This operation will get the `CloudFunction` object for the project `myProject`, location `us-central1` and functionName `myCamelFunction`. -- ListBuckets: this operation list the buckets for this account in this region +- CallFunction: this operation call the function using an HTTP request [source,java] -------------------------------------------------------------------------------- +//call function from("direct:start") -.to("google-functions://myCamelFunction?serviceAccountKey=/home/user/Downloads/my-key.json&operation=listBuckets") +.to("google-functions://myCamelFunction?serviceAccountKey=/home/user/Downloads/my-key.json&project=myProject&location=us-central1&operation=callFunction") +.log("body:${body}") .to("mock:result"); -------------------------------------------------------------------------------- -This operation will list the buckets for this account. - -- DeleteBucket: this operation delete the bucket specified as URI parameter or header +- GenerateDownloadUrl: this operation generate the signed URL for downloading deployed function source code. [source,java] -------------------------------------------------------------------------------- -from("direct:start") -.to("google-functions://myCamelFunction?serviceAccountKey=/home/user/Downloads/my-key.json&operation=deleteBucket") +//generate download url +from("direct:start").to("google-functions://myCamelFunction?serviceAccountKey=/home/user/Downloads/my-key.json&project=myProject&location=us-central1&operation=generateDownloadUrl") +.log("body:${body}") .to("mock:result"); -------------------------------------------------------------------------------- -This operation will delete the bucket myCamelBucket. - -- ListObjects: this operation list object in a specific bucket +- GenerateUploadUrl: this operation generate a signed URL for uploading a function source code. [source,java] -------------------------------------------------------------------------------- -from("direct:start") -.to("google-functions://myCamelFunction?serviceAccountKey=/home/user/Downloads/my-key.json&operation=listObjects") +from("direct:start").to("google-functions://myCamelFunction?serviceAccountKey=/home/user/Downloads/my-key.json&project=myProject&location=us-central1&operation=generateUploadUrl") +.log("body:${body}") .to("mock:result"); -------------------------------------------------------------------------------- -This operation will list the objects in the myCamelBucket bucket. - -- GetObject: this operation get a single object in a specific bucket +- createFunction: this operation creates a new function. [source,java] -------------------------------------------------------------------------------- + from("direct:start") -.process( exchange -> { - exchange.getIn().setHeader(GoogleCloudStorageConstants.OBJECT_NAME, "camel.txt"); -}) -.to("google-functions://myCamelFunction?serviceAccountKey=/home/user/Downloads/my-key.json&operation=getObject") +.process(exchange -> { + exchange.getIn().setHeader(GoogleCloudFunctionsConstants.ENTRY_POINT, "com.example.Example"); + exchange.getIn().setHeader(GoogleCloudFunctionsConstants.RUNTIME, "java11"); + exchange.getIn().setHeader(GoogleCloudFunctionsConstants.SOURCE_ARCHIVE_URL, "gs://myBucket/source.zip"); +}).to("google-functions://myCamelFunction?serviceAccountKey=/home/user/Downloads/my-key.json&project=myProject&location=us-central1&operation=createFunction") +.log("body:${body}") .to("mock:result"); -------------------------------------------------------------------------------- -This operation will return an Blob objct instance related to the OBJECT_NAME object in myCamelBucket bucket. - -- CreateDownloadLink: this operation will return a download link +- updateFunction: this operation updates existing function. [source,java] -------------------------------------------------------------------------------- from("direct:start") -.process( exchange -> { - exchange.getIn().setHeader(GoogleCloudStorageConstants.OBJECT_NAME, "camel.txt" ); - exchange.getIn().setHeader(GoogleCloudStorageConstants.DOWNLOAD_LINK_EXPIRATION_TIME, 86400000L); //1 day -}) -.to("google-functions://myCamelFunction?serviceAccountKey=/home/user/Downloads/my-key.json&operation=createDownloadLink") +.process(exchange -> { + UpdateFunctionRequest request = UpdateFunctionRequest.newBuilder() + .setFunction(CloudFunction.newBuilder().build()) + .setUpdateMask(FieldMask.newBuilder().build()).build(); + exchange.getIn().setBody(request); +}).to("google-functions://myCamelFunction?serviceAccountKey=/home/user/Downloads/my-key.json&project=myProject&location=us-central1&operation=updateFunction&pojoRequest=true") +.log("body:${body}") .to("mock:result"); -------------------------------------------------------------------------------- -This operation will return a download link url for the file OBJECT_NAME in the bucket myCamelBucket. It's possible to specify the expiration time for the created link through the header DOWNLOAD_LINK_EXPIRATION_TIME. If not specified, by default it is 5 minutes. - -== Bucket Autocreation - -With the option `autoCreateBucket` users are able to avoid the autocreation of a Bucket in case it doesn't exist. The default for this option is `true`. -If set to false any operation on a not-existent bucket won't be successful and an error will be returned. - -== MoveAfterRead consumer option - -In addition to deleteAfterRead it has been added another option, moveAfterRead. With this option enabled the consumed object will be moved to a target destinationBucket instead of being only deleted. -This will require specifying the destinationBucket option. As example: +- deleteFunction: this operation Deletes a function with the given name from the specified project. [source,java] -------------------------------------------------------------------------------- - from("google-functions://myCamelFunction?serviceAccountKey=/home/user/Downloads/my-key.json" - + "&autoCreateBucket=true" - + "&destinationBucket=myCamelProcessedBucket" - + "&moveAfterRead=true" - + "&deleteAfterRead=true" - + "&includeBody=true" - ) - .to("mock:result"); +from("direct:start") +.to("google-functions://myCamelFunction?serviceAccountKey=/home/user/Downloads/my-key.json&project=myProject&location=us-central1&operation=deleteFunction") +.log("body:${body}") +.to("mock:result"); -------------------------------------------------------------------------------- - -In this case the objects consumed will be moved to myCamelProcessedBucket bucket and deleted from the original one (because of deleteAfterRead). diff --git a/components/camel-google-functions/src/test/java/org/apache/camel/component/google/functions/integration/IntegrationTest.java b/components/camel-google-functions/src/test/java/org/apache/camel/component/google/functions/integration/IntegrationTest.java index 560d154..92f989b 100644 --- a/components/camel-google-functions/src/test/java/org/apache/camel/component/google/functions/integration/IntegrationTest.java +++ b/components/camel-google-functions/src/test/java/org/apache/camel/component/google/functions/integration/IntegrationTest.java @@ -92,8 +92,7 @@ public class IntegrationTest extends CamelTestSupport { functionCreationSourceCode); }).to("google-functions://" + randomFunctionName + "?serviceAccountKey=" + serviceAccountKeyFile + "&project=" + project + "&location=" + location + "&operation=createFunction") - .process(exchange -> { - }).log("body:${body}").to("mock:createFunction"); + .log("body:${body}").to("mock:createFunction"); // delete function from("timer:timer1?repeatCount=1") diff --git a/docs/components/modules/ROOT/pages/google-functions-component.adoc b/docs/components/modules/ROOT/pages/google-functions-component.adoc index 2ed67bd..a808237 100644 --- a/docs/components/modules/ROOT/pages/google-functions-component.adoc +++ b/docs/components/modules/ROOT/pages/google-functions-component.adoc @@ -59,11 +59,11 @@ export GOOGLE_APPLICATION_CREDENTIALS="/home/user/Downloads/my-key.json" [source,text] -------------------------------------------------------- -google-functions://functionName?[options] +google-functions://functionName[?options] -------------------------------------------------------- You can append query options to the URI in the following format, -?options=value&option2=value&... +`?options=value&option2=value&...` For example in order to call the function `myCamelFunction` from the project `myProject` and location `us-central1`, use the following snippet: @@ -132,29 +132,10 @@ with the following path and query parameters: [width="100%",cols="10%,10%,80%",options="header",] |======================================================================= |Header |Type |Description - -|`CamelGoogleCloudStorageBucketName` |`String` |The bucket Name which this object will be stored or which will be used for the current operation - -|`CamelGoogleCloudStorageObjectName` |`String` |The object Name which will be used for the current operation - -|`CamelGoogleCloudStorageDestinationBucketName` |`String` |The bucket Destination Name which will be used for the current operation - -|`CamelGoogleCloudStorageDestinationObjectName` |`String` |The object Destination Name which will be used for the current operation - -|`CamelGoogleCloudStorageContentLength` |`Long` |The content length of this object. - -|`CamelGoogleCloudStorageContentType` |`String` |The content type of this object. - - -|`CamelGoogleCloudStorageContentDisposition` |`String` |The content disposition of this object. - -|`CamelGoogleCloudStorageContentEncoding` |`String` |The content encoding of this object. - -|`CamelGoogleCloudStorageContentMd5` |`String` |The md5 checksum of this object. - -|`CamelGoogleCloudStorageOperation` |`String` |The operation to perform. Permitted values are copyObject, listObjects, deleteObject, deleteBucket, listBuckets, getObject, createDownloadLink - -|`CamelGoogleCloudStorageDownloadLinkExpirationTime` |`Long` |The time in millisecond the download link will be valid. +|`GoogleCloudFunctionsOperation` |`String` |The operation to perform. Permitted values are listFunctions, getFunction, callFunction, generateDownloadUrl, generateUploadUrl, createFunction, updateFunction, deleteFunction +|`GoogleCloudFunctionsEntryPoint` |`String` |The name of the function (as defined in source code) that will be executed. Used for createFunction operation. +|`GoogleCloudFunctionsRuntime` |`String` |The runtime in which to run the function. Possible values are `nodejs10`, `nodejs12`, `nodejs14`, `python37`, `python38`, `python39`, `go111`, `go113`, `java11`, `dotnet3`, `ruby26`, `nodejs6`, `nodejs8`. Used for createFunction operation. +|`GoogleCloudFunctionsSourceArchiveUrl` |`String` |The Google Cloud Storage URL, starting with `gs://`, pointing to the zip archive which contains the function. Used for createFunction operation. |======================================================================= @@ -163,195 +144,129 @@ with the following path and query parameters: [width="100%",cols="10%,10%,80%",options="header",] |======================================================================= |Header |Type |Description - -|`CamelGoogleCloudStorageETag` |`String` |The ETag value for the newly uploaded object. - -|======================================================================= - -=== Message headers set by the Google Storage Consumer - -[width="100%",cols="10%,10%,80%",options="header",] -|======================================================================= -|Header |Type |Description -|`CamelGoogleCloudStorageBucketName` |`String` |The bucket Name which this object will be stored or which will be used for the current operation -|`CamelGoogleCloudStorageObjectName` |`String` |The object Name which will be used for the current operation -| `CamelGoogleCloudStorageCacheControl` |`String` | The Cache-Control metadata can specify two different aspects of how data is served from Cloud Storage: whether the data can be cached and whether the data can be transformed -| `CamelGoogleCloudStorageComponentCount` |`String` | The component count of this object -| `CamelGoogleCloudStorageContentDisposition` |`String` |The content disposition of this object. -| `CamelGoogleCloudStorageContentEncoding` |`String` |The content encoding of this object. -| `CamelGoogleCloudStorageContentLanguage` |`String` | The Content-Language metadata indicates the language(s) that the object is intended for. -| `CamelGoogleCloudStorageContentType` |`String` |The content type of this object. -| `CamelGoogleCloudStorageCustomTime` |`String` | The Custom-Time metadata is a user-specified date and time represented in the RFC 3339 format YYYY-MM-DD'T'HH:MM:SS.SS'Z' or YYYY-MM-DD'T'HH:MM:SS'Z' when milliseconds are zero. This metadata is typically set in order to use the DaysSinceCustomTime condition in Object Lifecycle Management. -| `CamelGoogleCloudStorageCrc32cHex` |`String` | The CRC32c of the object -| `CamelGoogleCloudStorageETag` |`String` | The ETag for the Object. -| `CamelGoogleCloudStorageGeneration` |`String` | Is the generation number of the object for which you are retrieving information. -| `CamelGoogleCloudStorageBlobId` |`String` | The blob id of the object -| `CamelGoogleCloudStorageKmsKeyName` |`String` | The KMS key name -| `CamelGoogleCloudStorageContentMd5` |`String` |The md5 checksum of this object. -| `CamelGoogleCloudStorageMediaLink` |`String` | The media link -| `CamelGoogleCloudStorageMetageneration` | `String` | The metageneration of the object -| `CamelGoogleCloudStorageContentLength` |`Long` |The content length of this object. -| `CamelGoogleCloudStorageStorageClass` | `String` | The functions class of the object -| `CamelGoogleCloudStorageCreateTime` |`String` | The creation time of the object -| `CamelGoogleCloudStorageLastUpdate` |`String` | The last update of the object +|`GoogleCloudFunctionsResponseObject` |`Object` |The response object resulting from the Google Functions Client invocation |======================================================================= === Google Functions Producer operations Google Functions component provides the following operation on the producer side: -- copyObject -- listObjects -- deleteObject -- deleteBucket -- listBuckets -- getObject -- createDownloadLink +- listFunctions +- getFunction +- callFunction +- generateDownloadUrl +- generateUploadUrl +- createFunction +- updateFunction +- deleteFunction -If you don't specify an operation explicitly the producer will a file upload. +If you don't specify an operation by default the producer will use the `callFunction` operation. === Advanced component configuration -If you need to have more control over the `functionsClient` instance configuration, you can create your own instance and refer to it in your Camel google-functions component configuration: +If you need to have more control over the `client` instance configuration, you can create your own instance and refer to it in your Camel google-functions component configuration: [source,java] -------------------------------------------------------------------------------- -from("google-functions://myCamelFunction?functionsClient=#client") +from("google-functions://myCamelFunction?client=#myClient") .to("mock:result"); -------------------------------------------------------------------------------- === Google Functions Producer Operation examples -- File Upload: This operation will upload a file to the Google Functions based on the body content +- ListFunctions: This operation invoke the Google Functions client and get the list of cloud Functions [source,java] -------------------------------------------------------------------------------- -//upload a file -byte[] payload = "Camel rocks!".getBytes(); -ByteArrayInputStream bais = new ByteArrayInputStream(payload); -from("direct:start") -.process( exchange -> { - exchange.getIn().setHeader(GoogleCloudStorageConstants.OBJECT_NAME, "camel.txt"); - exchange.getIn().setBody(bais); -}) -.to("google-functions://myCamelFunction?serviceAccountKey=/home/user/Downloads/my-key.json") -.log("uploaded file object:${header.CamelGoogleCloudStorageObjectName}, body:${body}"); --------------------------------------------------------------------------------- - -This operation will upload the file camel.txt with the content "Camel rocks!" in the myCamelBucket bucket -- CopyObject: this operation copy an object from one bucket to a different one +//list functions +from("direct:start") +.to("google-functions://myCamelFunction?serviceAccountKey=/home/user/Downloads/my-key.json&project=myProject&location=us-central1&operation=listFunctions") +.log("body:${body}") -[source,java] --------------------------------------------------------------------------------- - from("direct:start").process( exchange -> { - exchange.getIn().setHeader(GoogleCloudStorageConstants.OPERATION, GoogleCloudStorageOperations.copyObject); - exchange.getIn().setHeader(GoogleCloudStorageConstants.OBJECT_NAME, "camel.txt" ); - exchange.getIn().setHeader(GoogleCloudStorageConstants.DESTINATION_BUCKET_NAME, "myCamelBucket_dest"); - exchange.getIn().setHeader(GoogleCloudStorageConstants.DESTINATION_OBJECT_NAME, "camel_copy.txt"); - }) - .to("google-functions://myCamelFunction?serviceAccountKey=/home/user/Downloads/my-key.json") - .to("mock:result"); -------------------------------------------------------------------------------- -This operation will copy the object with the name expressed in the header DESTINATION_OBJECT_NAME to the DESTINATION_BUCKET_NAME bucket, from the bucket myCamelBucket. +This operation will get the list of cloud functions for the project `myProject` and location `us-central1`. -- DeleteObject: this operation deletes an object from a bucket +- GetFunction: this operation get the Cloud Functions object [source,java] -------------------------------------------------------------------------------- - from("direct:start").process( exchange -> { - exchange.getIn().setHeader(GoogleCloudStorageConstants.OPERATION, GoogleCloudStorageOperations.deleteObject); - exchange.getIn().setHeader(GoogleCloudStorageConstants.OBJECT_NAME, "camel.txt" ); - }) - .to("google-functions://myCamelFunction?serviceAccountKey=/home/user/Downloads/my-key.json") - .to("mock:result"); +//get function +from("direct:start") +.to("google-functions://myCamelFunction?serviceAccountKey=/home/user/Downloads/my-key.json&project=myProject&location=us-central1&operation=getFunction") +.log("body:${body}") +.to("mock:result"); -------------------------------------------------------------------------------- -This operation will delete the object from the bucket myCamelBucket. +This operation will get the `CloudFunction` object for the project `myProject`, location `us-central1` and functionName `myCamelFunction`. -- ListBuckets: this operation list the buckets for this account in this region +- CallFunction: this operation call the function using an HTTP request [source,java] -------------------------------------------------------------------------------- +//call function from("direct:start") -.to("google-functions://myCamelFunction?serviceAccountKey=/home/user/Downloads/my-key.json&operation=listBuckets") +.to("google-functions://myCamelFunction?serviceAccountKey=/home/user/Downloads/my-key.json&project=myProject&location=us-central1&operation=callFunction") +.log("body:${body}") .to("mock:result"); -------------------------------------------------------------------------------- -This operation will list the buckets for this account. - -- DeleteBucket: this operation delete the bucket specified as URI parameter or header +- GenerateDownloadUrl: this operation generate the signed URL for downloading deployed function source code. [source,java] -------------------------------------------------------------------------------- -from("direct:start") -.to("google-functions://myCamelFunction?serviceAccountKey=/home/user/Downloads/my-key.json&operation=deleteBucket") +//generate download url +from("direct:start").to("google-functions://myCamelFunction?serviceAccountKey=/home/user/Downloads/my-key.json&project=myProject&location=us-central1&operation=generateDownloadUrl") +.log("body:${body}") .to("mock:result"); -------------------------------------------------------------------------------- -This operation will delete the bucket myCamelBucket. - -- ListObjects: this operation list object in a specific bucket +- GenerateUploadUrl: this operation generate a signed URL for uploading a function source code. [source,java] -------------------------------------------------------------------------------- -from("direct:start") -.to("google-functions://myCamelFunction?serviceAccountKey=/home/user/Downloads/my-key.json&operation=listObjects") +from("direct:start").to("google-functions://myCamelFunction?serviceAccountKey=/home/user/Downloads/my-key.json&project=myProject&location=us-central1&operation=generateUploadUrl") +.log("body:${body}") .to("mock:result"); -------------------------------------------------------------------------------- -This operation will list the objects in the myCamelBucket bucket. - -- GetObject: this operation get a single object in a specific bucket +- createFunction: this operation creates a new function. [source,java] -------------------------------------------------------------------------------- + from("direct:start") -.process( exchange -> { - exchange.getIn().setHeader(GoogleCloudStorageConstants.OBJECT_NAME, "camel.txt"); -}) -.to("google-functions://myCamelFunction?serviceAccountKey=/home/user/Downloads/my-key.json&operation=getObject") +.process(exchange -> { + exchange.getIn().setHeader(GoogleCloudFunctionsConstants.ENTRY_POINT, "com.example.Example"); + exchange.getIn().setHeader(GoogleCloudFunctionsConstants.RUNTIME, "java11"); + exchange.getIn().setHeader(GoogleCloudFunctionsConstants.SOURCE_ARCHIVE_URL, "gs://myBucket/source.zip"); +}).to("google-functions://myCamelFunction?serviceAccountKey=/home/user/Downloads/my-key.json&project=myProject&location=us-central1&operation=createFunction") +.log("body:${body}") .to("mock:result"); -------------------------------------------------------------------------------- -This operation will return an Blob objct instance related to the OBJECT_NAME object in myCamelBucket bucket. - -- CreateDownloadLink: this operation will return a download link +- updateFunction: this operation updates existing function. [source,java] -------------------------------------------------------------------------------- from("direct:start") -.process( exchange -> { - exchange.getIn().setHeader(GoogleCloudStorageConstants.OBJECT_NAME, "camel.txt" ); - exchange.getIn().setHeader(GoogleCloudStorageConstants.DOWNLOAD_LINK_EXPIRATION_TIME, 86400000L); //1 day -}) -.to("google-functions://myCamelFunction?serviceAccountKey=/home/user/Downloads/my-key.json&operation=createDownloadLink") +.process(exchange -> { + UpdateFunctionRequest request = UpdateFunctionRequest.newBuilder() + .setFunction(CloudFunction.newBuilder().build()) + .setUpdateMask(FieldMask.newBuilder().build()).build(); + exchange.getIn().setBody(request); +}).to("google-functions://myCamelFunction?serviceAccountKey=/home/user/Downloads/my-key.json&project=myProject&location=us-central1&operation=updateFunction&pojoRequest=true") +.log("body:${body}") .to("mock:result"); -------------------------------------------------------------------------------- -This operation will return a download link url for the file OBJECT_NAME in the bucket myCamelBucket. It's possible to specify the expiration time for the created link through the header DOWNLOAD_LINK_EXPIRATION_TIME. If not specified, by default it is 5 minutes. - -== Bucket Autocreation - -With the option `autoCreateBucket` users are able to avoid the autocreation of a Bucket in case it doesn't exist. The default for this option is `true`. -If set to false any operation on a not-existent bucket won't be successful and an error will be returned. - -== MoveAfterRead consumer option - -In addition to deleteAfterRead it has been added another option, moveAfterRead. With this option enabled the consumed object will be moved to a target destinationBucket instead of being only deleted. -This will require specifying the destinationBucket option. As example: +- deleteFunction: this operation Deletes a function with the given name from the specified project. [source,java] -------------------------------------------------------------------------------- - from("google-functions://myCamelFunction?serviceAccountKey=/home/user/Downloads/my-key.json" - + "&autoCreateBucket=true" - + "&destinationBucket=myCamelProcessedBucket" - + "&moveAfterRead=true" - + "&deleteAfterRead=true" - + "&includeBody=true" - ) - .to("mock:result"); +from("direct:start") +.to("google-functions://myCamelFunction?serviceAccountKey=/home/user/Downloads/my-key.json&project=myProject&location=us-central1&operation=deleteFunction") +.log("body:${body}") +.to("mock:result"); -------------------------------------------------------------------------------- - -In this case the objects consumed will be moved to myCamelProcessedBucket bucket and deleted from the original one (because of deleteAfterRead).