[
https://issues.apache.org/jira/browse/KAFKA-15513?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17848509#comment-17848509
]
Edgar commented on KAFKA-15513:
-------------------------------
Hello [~pprovenzano],
Are there any updates regarding this issue?
I am facing the same issue with Kafka version 3.7.0 when trying to setup
SASL_SSL using SCRAM-SHA-512 authentication for controller-to-controller
communication.
Error messages that I get:
{code:java}
[2024-05-22 10:27:42,144] lvl=ERROR [RaftManager id=3001] Unexpected error
NETWORK_EXCEPTION in VOTE response: InboundResponse(correlationId=187,
data=VoteResponseData(errorCode=13, topics=[]), sourceId=3000)
logger=org.apache.kafka.raft.KafkaRaftClient
[2024-05-22 10:27:42,165] lvl=ERROR [kafka-3001-raft-outbound-request-thread]:
Failed to send the following request due to authentication error:
ClientRequest(expectResponse=true,
callback=org.apache.kafka.raft.KafkaNetworkChannel$$Lambda$691/0x00000008010477a0@6265a25f,
destination=3002, correlationId=188, clientId=raft-client-3001,
createdTimeMs=1716366462144,
requestBuilder=VoteRequestData(clusterId='i2Eafb3tSEKas1jtIkS_vw',
topics=[TopicData(topicName='__cluster_metadata',
partitions=[PartitionData(partitionIndex=0, candidateEpoch=33981,
candidateId=3001, lastOffsetEpoch=0, lastOffset=0)])]))
logger=org.apache.kafka.raft.KafkaNetworkChannel$SendThread
[2024-05-22 10:27:42,165] lvl=ERROR Request OutboundRequest(correlationId=188,
data=VoteRequestData(clusterId='i2Eafb3tSEKas1jtIkS_vw',
topics=[TopicData(topicName='__cluster_metadata',
partitions=[PartitionData(partitionIndex=0, candidateEpoch=33981,
candidateId=3001, lastOffsetEpoch=0, lastOffset=0)])]),
createdTimeMs=1716366462144, destinationId=3002) failed due to authentication
error logger=org.apache.kafka.raft.KafkaNetworkChannel
org.apache.kafka.common.errors.SaslAuthenticationException: Authentication
failed during authentication due to invalid credentials with SASL mechanism
SCRAM-SHA-512
[2024-05-22 10:27:42,165] lvl=ERROR [RaftManager id=3001] Unexpected error
NETWORK_EXCEPTION in VOTE response: InboundResponse(correlationId=188,
data=VoteResponseData(errorCode=13, topics=[]), sourceId=3002)
logger=org.apache.kafka.raft.KafkaRaftClient {code}
For now in my testing environment I am forced to use SASL_SSL with PLAIN
authentication for controller-to-controller communication - this setup works
but it is not suitable for PROD environment.
> KRaft cluster fails with SCRAM authentication enabled for control-plane
> -----------------------------------------------------------------------
>
> Key: KAFKA-15513
> URL: https://issues.apache.org/jira/browse/KAFKA-15513
> Project: Kafka
> Issue Type: Bug
> Components: kraft
> Affects Versions: 3.6.0, 3.5.1
> Reporter: migruiz4
> Priority: Major
>
> We have observed a scenario where a KRaft cluster fails to bootstrap when
> using SCRAM authentication for controller-to-controller communications.
> The steps to reproduce are simple:
> * Deploy (at least) 2 Kafka servers using latest version 3.5.1.
> * Configure a KRaft cluster, where the controller listener uses
> SASL_PLAINTEXT + SCRAM-SHA-256 or SCRAM-SHA-512. In my case, I'm using the
> recommended in-line jaas config
> '{{{}listener.name.<controller-name>.scram-sha-512.sasl.jaas.config{}}}'
> * Run 'kafka-storage.sh' in both nodes using option '--add-scram' to create
> the SCRAM user.
> When initialized, Controllers will fail to connect to each other with an
> authentication error:
>
> {code:java}
> [2023-08-01 11:12:45,295] ERROR [kafka-1-raft-outbound-request-thread]:
> Failed to send the following request due to authentication error:
> ClientRequest(expectResponse=true,
> callback=kafka.raft.KafkaNetworkChannel$$Lambda$687/0x00007f27d443fc60@2aba6075,
> destination=0, correlationId=129, clientId=raft-client-1,
> createdTimeMs=1690888364960,
> requestBuilder=VoteRequestData(clusterId='abcdefghijklmnopqrstug',
> topics=[TopicData(topicName='__cluster_metadata',
> partitions=[PartitionData(partitionIndex=0, candidateEpoch=4, candidateId=1,
> lastOffsetEpoch=0, lastOffset=0)])])) (kafka.raft.RaftSendThread) {code}
> Some additional details about the scenario that we tested out:
> * Controller listener does work when configured with SASL+PLAIN
> * The issue only affects the Controller listener, SCRAM users created using
> the same method work for data-plane listeners and inter-broker listeners.
>
> Below you can find the exact configuration and command used to deploy:
> * server.properties
> {code:java}
> listeners=INTERNAL://:9092,CLIENT://:9091,CONTROLLER://:9093
> advertised.listeners=INTERNAL://kafka-0:9092,CLIENT://:9091
> listener.security.protocol.map=INTERNAL:PLAINTEXT,CLIENT:PLAINTEXT,CONTROLLER:SASL_PLAINTEXT
> num.network.threads=3
> num.io.threads=8
> socket.send.buffer.bytes=102400
> socket.receive.buffer.bytes=102400
> socket.request.max.bytes=104857600
> log.dirs=/bitnami/kafka/data
> num.partitions=1
> num.recovery.threads.per.data.dir=1
> offsets.topic.replication.factor=1
> transaction.state.log.replication.factor=1
> transaction.state.log.min.isr=1
> log.retention.hours=168
> log.retention.check.interval.ms=300000
> controller.listener.names=CONTROLLER
> controller.quorum.voters=0@kafka-0:9093,1@kafka-1:9093
> inter.broker.listener.name=INTERNAL
> node.id=0
> process.roles=controller,broker
> sasl.enabled.mechanisms=PLAIN,SCRAM-SHA-256,SCRAM-SHA-512
> sasl.mechanism.controller.protocol=SCRAM-SHA-512
> listener.name.controller.sasl.enabled.mechanisms=SCRAM-SHA-512
> listener.name.controller.scram-sha-512.sasl.jaas.config=org.apache.kafka.common.security.scram.ScramLoginModule
> required username="controller_user" password="controller_password";{code}
> * kafka-storage.sh command
> {code:java}
> kafka-storage.sh format --config /path/to/server.properties
> --ignore-formatted --cluster-id abcdefghijklmnopqrstuv --add-scram
> SCRAM-SHA-512=[name=controller_user,password=controller_password] {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
