jirar created KAFKA-17490:
-----------------------------
Summary: UpdateMetadataRequest failed because of failed
authorization
Key: KAFKA-17490
URL: https://issues.apache.org/jira/browse/KAFKA-17490
Project: Kafka
Issue Type: Bug
Components: core
Affects Versions: 2.8.1
Environment: kafka 2.8.1 + Ranger 2.3.0
Reporter: jirar
Assignee: jirar
Fix For: 4.0.0
I build a kafka cluster with kafka-2.8.1, use ranger as the authorization
platform. After the installation, when I firstly start the cluster, and then I
got ClusterAuthorizationException in kafka-server log files. In the same time,
SocketServer was started, but we cannot create topic with kafka-topics.sh tool,
at last, the creation would get an timeout exception.
There arr some important logs bellow
[2024-01-31 15:38:04,904] [INFO ] [controller-event-thread:2605]
[kafka.controller.KafkaController] [Logging.scala:66] - [Controller id=1] 1
successfully elected as the controller. Epoch incremented to 1 and epoch zk
version is now
[2024-01-31 15:38:05,540] [INFO ] [main:3241]
[org.apache.ranger.plugin.service.RangerBasePlugin] [RangerBasePlugin.java:227]
- Created PolicyRefresher
Thread(PolicyRefresher(serviceName=kafka-abcd-rmgwrqt1)-66)
// load roles to local file
[2024-01-31 15:38:06,056] [INFO ] [main:3757]
[org.apache.ranger.plugin.util.RangerRolesProvider]
[RangerRolesProvider.java:190] -
RangerRolesProvider(serviceName=kafka-abcd-rmgwrqt1): found updated version.
lastKnownRoleVersion=-1; newVersion=1
[2024-01-31 15:38:06,108] [WARN ] [main:3809]
[org.apache.ranger.plugin.util.PolicyRefresher] [PolicyRefresher.java:393] -
cache file does not exist or not readable
'/data/emr/kafka/policycache/kafka_kafka-abcd-rmgwrqt1.json'
[2024-01-31 15:38:06,411] [ERROR] [data-plane-kafka-request-handler-0:4112]
[kafka.server.RequestHandlerHelper] [Logging.scala:76] - [KafkaApi-1] Error
when handling request: clientId=1, correlationId=0, api=UPDATE_METADATA,
version=7, body=UpdateMetadataRequestData(controllerId=1, controllerEpoch=1,
brokerEpoch=4294967593, ungroupedPartitionStates=[], topicStates=[],
liveBrokers=[UpdateMetadataBroker(id=2, v0Host='', v0Port=0,
endpoints=[UpdateMetadataEndpoint(port=9092, host='10.0.0.22',
listener='SASL_PLAINTEXT', securityProtocol=2)], rack=null),
UpdateMetadataBroker(id=3, v0Host='', v0Port=0,
endpoints=[UpdateMetadataEndpoint(port=9092, host='10.0.0.29',
listener='SASL_PLAINTEXT', securityProtocol=2)], rack=null),
UpdateMetadataBroker(id=1, v0Host='', v0Port=0,
endpoints=[UpdateMetadataEndpoint(port=9092, host='10.0.0.45',
listener='SASL_PLAINTEXT', securityProtocol=2)], rack=null)])
org.apache.kafka.common.errors.ClusterAuthorizationException: Request
Request(processor=0, connectionId=10.0.0.45:9092-10.0.0.45:34308-0,
session=Session(User:hadoop,/10.0.0.45),
listenerName=ListenerName(SASL_PLAINTEXT), securityProtocol=SASL_PLAINTEXT,
buffer=null, envelope=None) is not authorized.
// load policy to local file
[2024-01-31 15:38:09,405] [INFO ]
[PolicyRefresher(serviceName=kafka-abcd-rmgwrqt1)-66:7106]
[org.apache.ranger.plugin.util.PolicyRefresher] [PolicyRefresher.java:321] -
PolicyRefresher(serviceName=kafka-abcd-rmgwrqt1): found updated version.
lastKnownVersion=-1; newVersion=6
[2024-01-31 15:38:09,406] [INFO ]
[PolicyRefresher(serviceName=kafka-abcd-rmgwrqt1)-66:7107]
[org.apache.ranger.plugin.policyengine.PolicyEngine] [PolicyEngine.java:202] -
Policy engine will not perform in place update while processing policy-deltas.
[2024-01-31 15:38:09,419] [INFO ]
[PolicyRefresher(serviceName=kafka-abcd-rmgwrqt1)-66:7120]
[org.apache.ranger.plugin.policyengine.RangerPolicyRepository]
[RangerPolicyRepository.java:950] - This policy engine contains 5 policy
evaluators
[2024-01-31 16:27:09,840] [INFO ] [data-plane-kafka-request-handler-5:2947541]
[kafka.server.ZkAdminManager] [Logging.scala:68] - [Admin Manager on Broker 1]:
Error processing create topic request
CreatableTopic(name='test-producer-consumer', numPartitions=1,
replicationFactor=2, assignments=[], configs=[])
org.apache.kafka.common.errors.InvalidReplicationFactorException: Replication
factor: 2 larger than available brokers: 0.
If I restart the kafka cluster, everything works well.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
