[
https://issues.apache.org/jira/browse/KAFKA-17083?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Josep Prat updated KAFKA-17083:
-------------------------------
Fix Version/s: 3.8.0
> KRaft Upgrade Failures in SystemTests
> -------------------------------------
>
> Key: KAFKA-17083
> URL: https://issues.apache.org/jira/browse/KAFKA-17083
> Project: Kafka
> Issue Type: Bug
> Components: system tests
> Affects Versions: 3.8.0
> Reporter: Josep Prat
> Assignee: Igor Soarez
> Priority: Critical
> Fix For: 3.8.0
>
>
> 2 System tests for "TestKRaftUpgrade are consistently failing on 3.8 in the
> system tests.
> {noformat}
> Module: kafkatest.tests.core.kraft_upgrade_test
> Class: TestKRaftUpgrade
> Method: test_isolated_mode_upgrade
> Arguments:
> {
> "from_kafka_version": "dev",
> "metadata_quorum": "ISOLATED_KRAFT"
> }
> {noformat}
>
> and
>
> {code:java}
> Module: kafkatest.tests.core.kraft_upgrade_test
> Class: TestKRaftUpgrade
> Method: test_combined_mode_upgrade
> Arguments:
> {
> "from_kafka_version": "dev",
> "metadata_quorum": "COMBINED_KRAFT"
> }
> {code}
>
> Failure for Isolated is:
> {noformat}
> RemoteCommandError({'ssh_config': {'host': 'worker15', 'hostname':
> '10.140.39.207', 'user': 'ubuntu', 'port': 22, 'password': None,
> 'identityfile': '/home/semaphore/kafka-overlay/semaphore-muckrake.pem'},
> 'hostname': 'worker15', 'ssh_hostname': '10.140.39.207', 'user': 'ubuntu',
> 'externally_routable_ip': '10.140.39.207', '_logger': <Logger
> kafkatest.tests.core.kraft_upgrade_test.TestKRaftUpgrade.test_isolated_mode_upgrade.from_kafka_version=dev.metadata_quorum=ISOLATED_KRAFT-674
> (DEBUG)>, 'os': 'linux', '_ssh_client': <paramiko.client.SSHClient object at
> 0x7f07e2e800a0>, '_sftp_client': <paramiko.sftp_client.SFTPClient object at
> 0x7f07e2f3d070>, '_custom_ssh_exception_checks': None},
> '/opt/kafka-dev/bin/kafka-features.sh --bootstrap-server
> worker15:9092,worker16:9092,worker17:9092 upgrade --metadata 3.7', 1,
> b'SLF4J: Class path contains multiple SLF4J bindings.\nSLF4J: Found binding
> in
> [jar:file:/vagrant/tools/build/dependant-libs-2.13.14/slf4j-reload4j-1.7.36.jar!/org/slf4j/impl/StaticLoggerBinder.class]\nSLF4J:
> Found binding in
> [jar:file:/vagrant/trogdor/build/dependant-libs-2.13.14/slf4j-reload4j-1.7.36.jar!/org/slf4j/impl/StaticLoggerBinder.class]\nSLF4J:
> See http://www.slf4j.org/codes.html#multiple_bindings for an
> explanation.\nSLF4J: Actual binding is of type
> [org.slf4j.impl.Reload4jLoggerFactory]\n1 out of 1 operation(s) failed.\n')
> Traceback (most recent call last):
> File
> "/home/semaphore/kafka-overlay/kafka/venv/lib/python3.8/site-packages/ducktape/tests/runner_client.py",
> line 184, in _do_run
> data = self.run_test()
> File
> "/home/semaphore/kafka-overlay/kafka/venv/lib/python3.8/site-packages/ducktape/tests/runner_client.py",
> line 262, in run_test
> return self.test_context.function(self.test)
> File
> "/home/semaphore/kafka-overlay/kafka/venv/lib/python3.8/site-packages/ducktape/mark/_mark.py",
> line 433, in wrapper
> return functools.partial(f, *args, **kwargs)(*w_args, **w_kwargs)
> File
> "/home/semaphore/kafka-overlay/kafka/tests/kafkatest/tests/core/kraft_upgrade_test.py",
> line 121, in test_isolated_mode_upgrade
> self.run_upgrade(from_kafka_version)
> File
> "/home/semaphore/kafka-overlay/kafka/tests/kafkatest/tests/core/kraft_upgrade_test.py",
> line 105, in run_upgrade
> self.run_produce_consume_validate(core_test_action=lambda:
> self.perform_version_change(from_kafka_version))
> File
> "/home/semaphore/kafka-overlay/kafka/tests/kafkatest/tests/produce_consume_validate.py",
> line 105, in run_produce_consume_validate
> core_test_action(*args)
> File
> "/home/semaphore/kafka-overlay/kafka/tests/kafkatest/tests/core/kraft_upgrade_test.py",
> line 105, in <lambda>
> self.run_produce_consume_validate(core_test_action=lambda:
> self.perform_version_change(from_kafka_version))
> File
> "/home/semaphore/kafka-overlay/kafka/tests/kafkatest/tests/core/kraft_upgrade_test.py",
> line 75, in perform_version_change
> self.kafka.upgrade_metadata_version(LATEST_STABLE_METADATA_VERSION)
> File
> "/home/semaphore/kafka-overlay/kafka/tests/kafkatest/services/kafka/kafka.py",
> line 920, in upgrade_metadata_version
> self.run_features_command("upgrade", new_version)
> File
> "/home/semaphore/kafka-overlay/kafka/tests/kafkatest/services/kafka/kafka.py",
> line 930, in run_features_command
> self.nodes[0].account.ssh(cmd)
> File
> "/home/semaphore/kafka-overlay/kafka/venv/lib/python3.8/site-packages/ducktape/cluster/remoteaccount.py",
> line 35, in wrapper
> return method(self, *args, **kwargs)
> File
> "/home/semaphore/kafka-overlay/kafka/venv/lib/python3.8/site-packages/ducktape/cluster/remoteaccount.py",
> line 293, in ssh
> raise RemoteCommandError(self, cmd, exit_status, stderr.read())
> ducktape.cluster.remoteaccount.RemoteCommandError: ubuntu@worker15: Command
> '/opt/kafka-dev/bin/kafka-features.sh --bootstrap-server
> worker15:9092,worker16:9092,worker17:9092 upgrade --metadata 3.7' returned
> non-zero exit status 1. Remote error message: b'SLF4J: Class path contains
> multiple SLF4J bindings.\nSLF4J: Found binding in
> [jar:file:/vagrant/tools/build/dependant-libs-2.13.14/slf4j-reload4j-1.7.36.jar!/org/slf4j/impl/StaticLoggerBinder.class]\nSLF4J:
> Found binding in
> [jar:file:/vagrant/trogdor/build/dependant-libs-2.13.14/slf4j-reload4j-1.7.36.jar!/org/slf4j/impl/StaticLoggerBinder.class]\nSLF4J:
> See http://www.slf4j.org/codes.html#multiple_bindings for an
> explanation.\nSLF4J: Actual binding is of type
> [org.slf4j.impl.Reload4jLoggerFactory]\n1 out of 1 operation(s) failed.\n'
> {noformat}
> Failure for combined is:
> {noformat}
> RemoteCommandError({'ssh_config': {'host': 'worker29', 'hostname':
> '10.140.50.210', 'user': 'ubuntu', 'port': 22, 'password': None,
> 'identityfile': '/home/semaphore/kafka-overlay/semaphore-muckrake.pem'},
> 'hostname': 'worker29', 'ssh_hostname': '10.140.50.210', 'user': 'ubuntu',
> 'externally_routable_ip': '10.140.50.210', '_logger': <Logger
> kafkatest.tests.core.kraft_upgrade_test.TestKRaftUpgrade.test_combined_mode_upgrade.from_kafka_version=dev.metadata_quorum=COMBINED_KRAFT-1789
> (DEBUG)>, 'os': 'linux', '_ssh_client': <paramiko.client.SSHClient object at
> 0x7f07e2a616a0>, '_sftp_client': <paramiko.sftp_client.SFTPClient object at
> 0x7f07e25ac7c0>, '_custom_ssh_exception_checks': None},
> '/opt/kafka-dev/bin/kafka-features.sh --bootstrap-server
> worker29:9092,worker4:9092,worker1:9092 upgrade --metadata 3.7', 1, b'SLF4J:
> Class path contains multiple SLF4J bindings.\nSLF4J: Found binding in
> [jar:file:/vagrant/tools/build/dependant-libs-2.13.14/slf4j-reload4j-1.7.36.jar!/org/slf4j/impl/StaticLoggerBinder.class]\nSLF4J:
> Found binding in
> [jar:file:/vagrant/trogdor/build/dependant-libs-2.13.14/slf4j-reload4j-1.7.36.jar!/org/slf4j/impl/StaticLoggerBinder.class]\nSLF4J:
> See http://www.slf4j.org/codes.html#multiple_bindings for an
> explanation.\nSLF4J: Actual binding is of type
> [org.slf4j.impl.Reload4jLoggerFactory]\n1 out of 1 operation(s) failed.\n')
> Traceback (most recent call last):
> File
> "/home/semaphore/kafka-overlay/kafka/venv/lib/python3.8/site-packages/ducktape/tests/runner_client.py",
> line 184, in _do_run
> data = self.run_test()
> File
> "/home/semaphore/kafka-overlay/kafka/venv/lib/python3.8/site-packages/ducktape/tests/runner_client.py",
> line 262, in run_test
> return self.test_context.function(self.test)
> File
> "/home/semaphore/kafka-overlay/kafka/venv/lib/python3.8/site-packages/ducktape/mark/_mark.py",
> line 433, in wrapper
> return functools.partial(f, *args, **kwargs)(*w_args, **w_kwargs)
> File
> "/home/semaphore/kafka-overlay/kafka/tests/kafkatest/tests/core/kraft_upgrade_test.py",
> line 115, in test_combined_mode_upgrade
> self.run_upgrade(from_kafka_version)
> File
> "/home/semaphore/kafka-overlay/kafka/tests/kafkatest/tests/core/kraft_upgrade_test.py",
> line 105, in run_upgrade
> self.run_produce_consume_validate(core_test_action=lambda:
> self.perform_version_change(from_kafka_version))
> File
> "/home/semaphore/kafka-overlay/kafka/tests/kafkatest/tests/produce_consume_validate.py",
> line 105, in run_produce_consume_validate
> core_test_action(*args)
> File
> "/home/semaphore/kafka-overlay/kafka/tests/kafkatest/tests/core/kraft_upgrade_test.py",
> line 105, in <lambda>
> self.run_produce_consume_validate(core_test_action=lambda:
> self.perform_version_change(from_kafka_version))
> File
> "/home/semaphore/kafka-overlay/kafka/tests/kafkatest/tests/core/kraft_upgrade_test.py",
> line 75, in perform_version_change
> self.kafka.upgrade_metadata_version(LATEST_STABLE_METADATA_VERSION)
> File
> "/home/semaphore/kafka-overlay/kafka/tests/kafkatest/services/kafka/kafka.py",
> line 920, in upgrade_metadata_version
> self.run_features_command("upgrade", new_version)
> File
> "/home/semaphore/kafka-overlay/kafka/tests/kafkatest/services/kafka/kafka.py",
> line 930, in run_features_command
> self.nodes[0].account.ssh(cmd)
> File
> "/home/semaphore/kafka-overlay/kafka/venv/lib/python3.8/site-packages/ducktape/cluster/remoteaccount.py",
> line 35, in wrapper
> return method(self, *args, **kwargs)
> File
> "/home/semaphore/kafka-overlay/kafka/venv/lib/python3.8/site-packages/ducktape/cluster/remoteaccount.py",
> line 293, in ssh
> raise RemoteCommandError(self, cmd, exit_status, stderr.read())
> ducktape.cluster.remoteaccount.RemoteCommandError: ubuntu@worker29: Command
> '/opt/kafka-dev/bin/kafka-features.sh --bootstrap-server
> worker29:9092,worker4:9092,worker1:9092 upgrade --metadata 3.7' returned
> non-zero exit status 1. Remote error message: b'SLF4J: Class path contains
> multiple SLF4J bindings.\nSLF4J: Found binding in
> [jar:file:/vagrant/tools/build/dependant-libs-2.13.14/slf4j-reload4j-1.7.36.jar!/org/slf4j/impl/StaticLoggerBinder.class]\nSLF4J:
> Found binding in
> [jar:file:/vagrant/trogdor/build/dependant-libs-2.13.14/slf4j-reload4j-1.7.36.jar!/org/slf4j/impl/StaticLoggerBinder.class]\nSLF4J:
> See http://www.slf4j.org/codes.html#multiple_bindings for an
> explanation.\nSLF4J: Actual binding is of type
> [org.slf4j.impl.Reload4jLoggerFactory]\n1 out of 1 operation(s)
> failed.\n'{noformat}
> Here is an instance of this failure
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
