This is an automated email from the ASF dual-hosted git repository.
acosentino pushed a commit to branch master
in repository
https://gitbox.apache.org/repos/asf/camel-kafka-connector-examples.git
commit 94045d5a47412401e3932735464d5114c9be875d
Author: Andrea Cosentino <anco...@gmail.com>
AuthorDate: Wed Apr 29 15:29:55 2020 +0200
Added a first example AWS-S3 to JMS
---
README.adoc | 5 +
aws-s3-to-jms/README.md | 102 +++++++++++++++++++++
.../config/CamelAWSS3SourceConnector.properties | 32 +++++++
.../config/CamelJmsSinkConnector.properties | 28 ++++++
4 files changed, 167 insertions(+)
diff --git a/README.adoc b/README.adoc
new file mode 100644
index 0000000..15b0ffe
--- /dev/null
+++ b/README.adoc
@@ -0,0 +1,5 @@
+This is camel-kafka-connector examples repository
+
+List of existing examples:
+
+- aws-s3 to JMS example
diff --git a/aws-s3-to-jms/README.md b/aws-s3-to-jms/README.md
new file mode 100644
index 0000000..58ab24e
--- /dev/null
+++ b/aws-s3-to-jms/README.md
@@ -0,0 +1,102 @@
+# Camel-Kafka-connector AWS S3 to JMS demo
+
+## Introduction
+
+This is an example for Camel-Kafka-connector
+
+## What is needed
+
+- An Artemis Broker 2.9.0 running
+- A Kafka Cluster 2.4.0 running
+- An AWS S3 bucket
+
+## Running Kafka
+
+```
+$KAFKA_HOME/bin/zookeeper-server-start.sh config/zookeeper.properties
+$KAFKA_HOME/bin/kafka-server-start.sh config/server.properties
+$KAFKA_HOME/bin/kafka-topics.sh --create --bootstrap-server localhost:9092
--replication-factor 1 --partitions 1 --topic test
+```
+
+## Running Artemis
+
+```
+$ARTEMIS_HOME/bin/$BROKER_NAME/bin/artemis run
+```
+
+## Setting up the needed bits and running the example
+
+You'll need to setup the plugin.path property in your kafka
+
+Open the `$KAFKA_HOME/config/connect-standalone.properties`
+
+and set the `plugin.path` property to your choosen location
+
+In this example we'll use `/home/oscerd/connectors/`
+
+```
+> cd /home/oscerd/connectors/
+> wget
https://repo1.maven.org/maven2/org/apache/camel/kafkaconnector/camel-sjms2-kafka-connector/0.1.0/camel-sjms2-kafka-connector-0.1.0-package.zip
+> unzip camel-sjms2-kafka-connector-0.1.0-package.zip
+> wget
https://repo1.maven.org/maven2/org/apache/camel/kafkaconnector/camel-aws-s3-kafka-connector/0.1.0/camel-aws-s3-kafka-connector-0.1.0-package.zip
+> unzip camel-aws-s3-kafka-connector-0.1.0-package.zip
+```
+
+These are the basic connectors. For camel-sjms2 we have a bunch of provided
dependencies we need to add in our path, so run the following commands (this is
a known bug of Camel-sjms2kKafka-connector 0.1.0, already fixed for 0.2.0)
+
+```
+> cd /home/oscerd/connectors/camel-sjms2-kafka-connector
+> wget
https://repo1.maven.org/maven2/org/apache/activemq/activemq-client/5.15.11/activemq-client-5.15.11.jar
+> wget
https://repo1.maven.org/maven2/org/apache/geronimo/specs/geronimo-jms_2.0_spec/1.0-alpha-2/geronimo-jms_2.0_spec-1.0-alpha-2.jar
+> wget
https://repo1.maven.org/maven2/org/apache/geronimo/specs/geronimo-annotation_1.0_spec/1.1.1/geronimo-annotation_1.0_spec-1.1.1.jar
+> wget
https://repo1.maven.org/maven2/javax/management/j2ee/management-api/1.1-rev-1/management-api-1.1-rev-1.jar
+> wget
https://repo1.maven.org/maven2/org/fusesource/hawtbuf/hawtbuf/1.11/hawtbuf-1.11.jar
+```
+
+Now it's time to setup the connectors
+
+Open the AWSS3 properties file
+
+```
+name=CamelAWSS3SourceConnector
+connector.class=org.apache.camel.kafkaconnector.awss3.CamelAwss3SourceConnector
+key.converter=org.apache.kafka.connect.storage.StringConverter
+value.converter=org.apache.camel.kafkaconnector.awss3.converters.S3ObjectConverter
+
+camel.source.maxPollDuration=10000
+
+camel.source.kafka.topic=mytopic
+
+camel.source.url=aws-s3://camel-kafka-connector?autocloseBody=false
+
+camel.component.aws-s3.access-key=xxxx
+camel.component.aws-s3.secret-key=yyyy
+camel.component.aws-s3.region=EU_WEST_1
+```
+
+and add the correct credentials for AWS.
+
+For the JMS Sink connector you don't need to modify anything, so it is
+
+```
+name=CamelJmsSinkConnector
+topics=mytopic
+tasks.max=1
+connector.class=org.apache.camel.kafkaconnector.sjms2.CamelSjms2SinkConnector
+key.converter=org.apache.kafka.connect.storage.StringConverter
+value.converter=org.apache.kafka.connect.storage.StringConverter
+
+camel.sink.url=sjms2://queue:myqueue
+
+camel.component.sjms2.connection-factory=#class:org.apache.activemq.ActiveMQConnectionFactory
+camel.component.sjms2.connection-factory.brokerURL=tcp://localhost:61616
+```
+
+Now you can run the example
+
+```
+$KAFKA_HOME/bin/connect-standalone.sh
$KAFKA_HOME/config/connect-standalone.properties
config/CamelAWSS3SourceConnector.properties
config/CamelJmsSinkConnector.properties
+```
+
+Just connect to your AWS Console and upload a text file to your bucket (we
provide the example element.txt in this repo), in the Artemis console you
should see a message with the file content in a bit.
+
diff --git a/aws-s3-to-jms/config/CamelAWSS3SourceConnector.properties
b/aws-s3-to-jms/config/CamelAWSS3SourceConnector.properties
new file mode 100644
index 0000000..5fd1ba7
--- /dev/null
+++ b/aws-s3-to-jms/config/CamelAWSS3SourceConnector.properties
@@ -0,0 +1,32 @@
+#
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+
+name=CamelAWSS3SourceConnector
+connector.class=org.apache.camel.kafkaconnector.awss3.CamelAwss3SourceConnector
+key.converter=org.apache.kafka.connect.storage.StringConverter
+value.converter=org.apache.camel.kafkaconnector.awss3.converters.S3ObjectConverter
+
+camel.source.maxPollDuration=10000
+
+camel.source.kafka.topic=mytopic
+
+camel.source.url=aws-s3://camel-kafka-connector?autocloseBody=false
+
+camel.component.aws-s3.access-key=xxxx
+camel.component.aws-s3.secret-key=yyyy
+camel.component.aws-s3.region=EU_WEST_1
+
diff --git a/aws-s3-to-jms/config/CamelJmsSinkConnector.properties
b/aws-s3-to-jms/config/CamelJmsSinkConnector.properties
new file mode 100644
index 0000000..2cf8ce4
--- /dev/null
+++ b/aws-s3-to-jms/config/CamelJmsSinkConnector.properties
@@ -0,0 +1,28 @@
+#
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+
+name=CamelJmsSinkConnector
+topics=mytopic
+tasks.max=1
+connector.class=org.apache.camel.kafkaconnector.sjms2.CamelSjms2SinkConnector
+key.converter=org.apache.kafka.connect.storage.StringConverter
+value.converter=org.apache.kafka.connect.storage.StringConverter
+
+camel.sink.url=sjms2://queue:myqueue
+
+camel.component.sjms2.connection-factory=#class:org.apache.activemq.ActiveMQConnectionFactory
+camel.component.sjms2.connection-factory.brokerURL=tcp://localhost:61616
