christophd opened a new pull request, #8694: URL: https://github.com/apache/camel/pull/8694
WIP do not merge! This PR is supposed to be a work in progress seeking for guidance and help on implementing multiple data types on components feature as described in https://issues.apache.org/jira/browse/CAMEL-18698 The changes made in this PR should be seen as a POC starting some discussions and refinements of the feature. The idea is to let the user choose a specific data type that the component is producing as an output. The component itself may offer specific data transformation logic that gets automatically triggered when this data type is explicitly used on an endpoint URI: ``` from("kafka:topic?format=avro").to("aws2-s3:bucketName?format=string") ``` The `format` option tells the component to apply the specific data type as an input or output. The data type may be directly supported by the component or it may be a generic data type that makes use of the Camel TypeConverter logic to transform from one data to another. In the PR the feature is implemented as a POC for aws2-ddb and aws2-s3 components. The aws2 components often use Java POJO domain objects as input and output so the user needs to know the domain model when interacting with the components. The idea is to also add support for more generic data types on these components. As an example the user may also provide a generic Json structure as an input instead of the AWS domain object. The component takes care on transforming the data type to the required Java POJO used in the AWS client libraries. On the output side the aws2-s3 component by default produces byte[] or InputStream output. With the addition of data types the user is able to request another data type such as String. The component makes sure to apply transformation logic from the AWS domain model InputStream implementation to a String. This feature is beneficial for Camel users that do not have to know about domain specific Java types anymore. Also declarative Camel DSL use cases e.g. used in Camel K and Kamelets benefit from the auto data type conversion as discussed in https://github.com/apache/camel-k/issues/1980 A Kamelet may use the provided data types information and expose this as part of the Kamelet specification. ``` apiVersion: camel.apache.org/v1alpha1 kind: Kamelet metadata: name: aws-s3-source spec: definition: title: "AWS S3 Source" properties: ... types: out: default: binary binary: mediaType: application/octet-stream stream: mediaType: application/octet-stream json: mediaType: application/json schema: type: object required: - key - fileContent properties: key: title: Key description: The S3 file name key type: string fileContent: title: Content description: The S3 file content as String type: string dependencies: - "camel:jackson" string: mediaType: plain/text avro: mediaType: application/avro ``` Also the user is able to choose the input/output data type in a binding: ``` apiVersion: camel.apache.org/v1alpha1 kind: KameletBinding metadata: name: aws-s3-uri-binding spec: source: ref: kind: Kamelet apiVersion: camel.apache.org/v1alpha1 name: aws-s3-source output: type: avro schema: uri: http://schema-registry/person properties: outputFormat: avro bucketNameOrArn: ${aws.s3.bucketNameOrArn} sink: uri: log:info ``` The new data types implementation provided in this PR use a specific SPI annotation `@DataType`. A resolver mechanism `DefaultDataTypeResolver` is capable of doing a lookup of the respective data type implementation with given component scheme and a format name that identifies a data type. The PR definitely needs more polishing and has some ToDos in the code that seek for guidance and help how to do it the Camel way (e.g. how to do an automatic resource lookup for data type implementations provided by components). Also it would be nice to have some guidance on the already existing `org.apache.camel.spi.DataType` and how that could be related to this implementation. Many thanks in advance! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@camel.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
