arjun180 opened a new issue #1216:
URL: https://github.com/apache/camel-kafka-connector/issues/1216
Hello,
I am trying to get the Camel AWSS2-S3-Kafka connector sink to work with my
Kafka cluster. I have a functioning Kafka cluster as well as a Kafka connect
cluster up and running. My aim is to move messages from my Kafka topic to the
s3 bucket I have created.
My KafkaConnect cluster is fine and has the following pulgins :
```
Status:
Conditions:
Last Transition Time: 2021-06-21T22:51:04.727332Z
Status: True
Type: Ready
Connector Plugins:
Class: org.apache.camel.kafkaconnector.CamelSinkConnector
Type: sink
Version: 0.8.0
Class: org.apache.camel.kafkaconnector.CamelSourceConnector
Type: source
Version: 0.8.0
Class:
org.apache.camel.kafkaconnector.aws2s3.CamelAws2s3SinkConnector
Type: sink
Version: 0.8.0
Class:
org.apache.camel.kafkaconnector.aws2s3.CamelAws2s3SourceConnector
Type: source
Version: 0.8.0
Class:
org.apache.camel.kafkaconnector.timer.CamelTimerSourceConnector
Type: source
Version: 0.8.0
Class: org.apache.kafka.connect.file.FileStreamSinkConnector
Type: sink
Version: 2.8.0
Class:
org.apache.kafka.connect.file.FileStreamSourceConnector
Type: source
Version: 2.8.0
Class:
org.apache.kafka.connect.mirror.MirrorCheckpointConnector
Type: source
Version: 1
Class:
org.apache.kafka.connect.mirror.MirrorHeartbeatConnector
Type: source
Version: 1
Class: org.apache.kafka.connect.mirror.MirrorSourceConnector
Type: source
Version: 1
Label Selector:
strimzi.io/cluster=my-connect-cluster,strimzi.io/name=my-connect-cluster-connect,strimzi.io/kind=KafkaConnect
Observed Generation: 1
Replicas: 1
URL:
http://my-connect-cluster-connect-api.my-connect-kafka.svc:8083
```
My KafkaConnector looks like the following :
```
apiVersion: kafka.strimzi.io/v1beta2
kind: KafkaConnector
metadata:
name: s3-sink-connector
labels:
strimzi.io/cluster: my-connect-cluster
spec:
class: org.apache.camel.kafkaconnector.aws2s3.CamelAws2s3SinkConnector
tasksMax: 1
config:
key.converter: org.apache.kafka.connect.storage.StringConverter
value.converter: org.apache.kafka.connect.storage.StringConverter
topics: kafka-connect-topic
camel.sink.path.bucketNameOrArn: my-connect-bucket
camel.sink.endpoint.keyName: ${date:now:yyyyMMdd-HHmmssSSS}-${exchangeId}
camel.component.aws2-s3.accessKey: <access_key>
camel.component.aws2-s3.secretKey: <secret_key>
camel.component.aws2-s3.region: <region>
```
After running the connector above I get the following error :
```
Caused by: org.apache.camel.ResolveEndpointFailedException: Failed to
resolve endpoint:
aws2-s3://my-cluster-connect?keyName=20210622-xxxxxxxx-xxxxxxxxxxxxxxxxxxx-0000000000000000
due to: null (Service: S3, Status Code: 403, Request ID: null, Extended
Request ID: xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx=)
```
I imagine the 403 error is because of my s3 bucket not allowing the
connector access. Are there some configuration changes I need to be making to
the s3 bucket. I did try connecting to my s3 bucket through the AWS CLI and
that worked fine. Currently the bucket is set to "Bucket and objects not public"
I'd appreciate any help! Thanks
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
