[
https://issues.apache.org/jira/browse/IMPALA-14888?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael Smith resolved IMPALA-14888.
------------------------------------
Fix Version/s: Impala 5.0.0
Resolution: Fixed
> TestLogFragments.test_slow_rpc_stack_trace Failures
> ---------------------------------------------------
>
> Key: IMPALA-14888
> URL: https://issues.apache.org/jira/browse/IMPALA-14888
> Project: IMPALA
> Issue Type: Bug
> Affects Versions: Impala 5.0.0
> Reporter: Jason Fehr
> Assignee: Michael Smith
> Priority: Critical
> Fix For: Impala 5.0.0
>
>
> h2. Error Message
> AssertionError: Expected at least one line in file
> /data0/jenkins/workspace/impala-asf-master-core-asan/repos/Impala/logs/ee_tests/impalad.impala-ec2-redhat86-m6i-4xlarge-ondemand-0759.vpc.cloudera.com.jenkins.log.WARNING.20260413-014350.3130458
> matching regex 'kernel_stack_watchdog.cc:.* Thread .* stuck', but found none.
> h2. Stacktrace
> /data/jenkins/workspace/impala-asf-master-core-asan/repos/Impala/tests/observability/test_log_fragments.py:68:
> in test_slow_rpc_stack_trace
> self.assert_impalad_log_contains("WARNING",
> query = 'select tinyint_col, count(*) from functional.alltypes\n
> group by tinyint_col order by tinyint_col limit 5'
> self = <tests.observability.test_log_fragments.TestLogFragments
> object at 0x7ff5c09850d0>
> common/impala_test_suite.py:1691: in assert_impalad_log_contains
> return self.assert_log_contains(
> dry_run = False
> expected_count = -1
> level = 'WARNING'
> line_regex = 'kernel_stack_watchdog.cc:.* Thread .* stuck'
> self = <tests.observability.test_log_fragments.TestLogFragments
> object at 0x7ff5c09850d0>
> timeout_s = 6
> common/impala_test_suite.py:1802: in assert_log_contains
> assert found > 0, "Expected at least one line in file %s matching regex
> '%s'"\
> E AssertionError: Expected at least one line in file
> /data0/jenkins/workspace/impala-asf-master-core-asan/repos/Impala/logs/ee_tests/impalad.impala-ec2-redhat86-m6i-4xlarge-ondemand-0759.vpc.cloudera.com.jenkins.log.WARNING.20260413-014350.3130458
> matching regex 'kernel_stack_watchdog.cc:.* Thread .* stuck', but found none.
> daemon = 'impalad'
> dry_run = False
> expected_count = -1
> found = 0
> last_re_result = None
> level = 'WARNING'
> line = 'W20260413 02:23:19.615694 3130958 outbound_call.cc:322]
> RPC callback for RPC call
> impala.ControlService.ExecQueryFIns...9.vpc.cloudera.com),
> user_credentials={real_user=impala}, network_plane=control} blocked reactor
> thread for 252692us\n'
> line_regex = 'kernel_stack_watchdog.cc:.* Thread .* stuck'
> log_file = <_io.BufferedReader
> name='/data0/jenkins/workspace/impala-asf-master-core-asan/repos/Impala/logs/ee_tests/impalad.impala-ec2-redhat86-m6i-4xlarge-ondemand-0759.vpc.cloudera.com.jenkins.log.WARNING.20260413-014350.3130458'>
> log_file_path =
> '/data0/jenkins/workspace/impala-asf-master-core-asan/repos/Impala/logs/ee_tests/impalad.impala-ec2-redhat86-m6i-4xlarge-ondemand-0759.vpc.cloudera.com.jenkins.log.WARNING.20260413-014350.3130458'
> pattern = re.compile('kernel_stack_watchdog.cc:.* Thread .* stuck')
> re_result = None
> self = <tests.observability.test_log_fragments.TestLogFragments
> object at 0x7ff5c09850d0>
> start_time = 1776072199.7363474
> timeout_s = 6
> h2. Standard Error
> --------------------------------- Captured Err
> ---------------------------------
> -- 2026-04-13 02:23:18,939 INFO MainThread: beeswax:
> set
> client_identifier=observability/test_log_fragments.py::TestLogFragments::test_slow_rpc_stack_trace;
> -- 2026-04-13 02:23:18,939 INFO MainThread: Could not connect to ('::1',
> 21000, 0, 0)
> Traceback (most recent call last):
> File
> "/data/jenkins/workspace/impala-asf-master-core-asan/repos/Impala/infra/python/env-gcc10.4.0-py3/lib/python3.11/site-packages/thrift/transport/TSocket.py",
> line 137, in open
> handle.connect(sockaddr)
> ConnectionRefusedError: [Errno 111] Connection refused
> -- 2026-04-13 02:23:18,940 INFO MainThread: beeswax: connected to
> localhost:21000 with beeswax
> -- 2026-04-13 02:23:18,940 INFO MainThread: hs2:
> set
> client_identifier=observability/test_log_fragments.py::TestLogFragments::test_slow_rpc_stack_trace;
> -- 2026-04-13 02:23:18,940 INFO MainThread: Could not connect to ('::1',
> 21050, 0, 0)
> Traceback (most recent call last):
> File
> "/data/jenkins/workspace/impala-asf-master-core-asan/repos/Impala/infra/python/env-gcc10.4.0-py3/lib/python3.11/site-packages/thrift/transport/TSocket.py",
> line 137, in open
> handle.connect(sockaddr)
> ConnectionRefusedError: [Errno 111] Connection refused
> -- 2026-04-13 02:23:18,940 INFO MainThread: hs2: connected to
> localhost:21050 with impyla hs2
> -- 2026-04-13 02:23:18,940 INFO MainThread: hs2-http:
> set
> client_identifier=observability/test_log_fragments.py::TestLogFragments::test_slow_rpc_stack_trace;
> -- 2026-04-13 02:23:18,942 INFO MainThread: hs2-http: connected to
> localhost:28000 with impyla hs2-http
> -- 2026-04-13 02:23:18,942 INFO MainThread: beeswax:
> set
> client_identifier=observability/test_log_fragments.py::TestLogFragments::test_slow_rpc_stack_trace;
> -- 2026-04-13 02:23:18,942 INFO MainThread: hs2:
> set
> client_identifier=observability/test_log_fragments.py::TestLogFragments::test_slow_rpc_stack_trace;
> -- 2026-04-13 02:23:18,942 INFO MainThread: hs2-http:
> set
> client_identifier=observability/test_log_fragments.py::TestLogFragments::test_slow_rpc_stack_trace;
> -- 2026-04-13 02:23:18,948 INFO MainThread: hs2:
> set
> client_identifier=observability/test_log_fragments.py::TestLogFragments::test_slow_rpc_stack_trace;
> -- 2026-04-13 02:23:18,948 INFO MainThread: hs2: set_configuration:
> set debug_action=IMPALA_MISS_EXEC_COMPLETE_CB:SLEEP@300;
> -- 2026-04-13 02:23:18,953 INFO MainThread: hs2: executing against Impala
> at localhost:21050. session: 7c452659057357fa:28f90d953b7a4182 main_cursor:
> True user: None
> select tinyint_col, count(*) from functional.alltypes
> group by tinyint_col order by tinyint_col limit 5;
> -- 2026-04-13 02:23:19,660 INFO MainThread:
> 514b5a6859b25d0f:156b48d600000000: query started
> -- 2026-04-13 02:23:19,689 INFO MainThread:
> 514b5a6859b25d0f:156b48d600000000: getting log for operation
> -- 2026-04-13 02:23:19,690 INFO MainThread:
> 514b5a6859b25d0f:156b48d600000000: getting runtime profile operation
> -- 2026-04-13 02:23:19,731 INFO MainThread:
> 514b5a6859b25d0f:156b48d600000000: closing query for operation
> -- 2026-04-13 02:23:19,745 INFO MainThread: Expected log lines could not
> be found, sleeping before retrying: Expected at least one line in file
> /data0/jenkins/workspace/impala-asf-master-core-asan/repos/Impala/logs/ee_tests/impalad.impala-ec2-redhat86-m6i-4xlarge-ondemand-0759.vpc.cloudera.com.jenkins.log.WARNING.20260413-014350.3130458
> matching regex 'kernel_stack_watchdog.cc:.* Thread .* stuck', but found none.
> -- 2026-04-13 02:23:20,751 INFO MainThread: Expected log lines could not
> be found, sleeping before retrying: Expected at least one line in file
> /data0/jenkins/workspace/impala-asf-master-core-asan/repos/Impala/logs/ee_tests/impalad.impala-ec2-redhat86-m6i-4xlarge-ondemand-0759.vpc.cloudera.com.jenkins.log.WARNING.20260413-014350.3130458
> matching regex 'kernel_stack_watchdog.cc:.* Thread .* stuck', but found none.
> -- 2026-04-13 02:23:21,756 INFO MainThread: Expected log lines could not
> be found, sleeping before retrying: Expected at least one line in file
> /data0/jenkins/workspace/impala-asf-master-core-asan/repos/Impala/logs/ee_tests/impalad.impala-ec2-redhat86-m6i-4xlarge-ondemand-0759.vpc.cloudera.com.jenkins.log.WARNING.20260413-014350.3130458
> matching regex 'kernel_stack_watchdog.cc:.* Thread .* stuck', but found none.
> -- 2026-04-13 02:23:22,762 INFO MainThread: Expected log lines could not
> be found, sleeping before retrying: Expected at least one line in file
> /data0/jenkins/workspace/impala-asf-master-core-asan/repos/Impala/logs/ee_tests/impalad.impala-ec2-redhat86-m6i-4xlarge-ondemand-0759.vpc.cloudera.com.jenkins.log.WARNING.20260413-014350.3130458
> matching regex 'kernel_stack_watchdog.cc:.* Thread .* stuck', but found none.
> -- 2026-04-13 02:23:23,768 INFO MainThread: Expected log lines could not
> be found, sleeping before retrying: Expected at least one line in file
> /data0/jenkins/workspace/impala-asf-master-core-asan/repos/Impala/logs/ee_tests/impalad.impala-ec2-redhat86-m6i-4xlarge-ondemand-0759.vpc.cloudera.com.jenkins.log.WARNING.20260413-014350.3130458
> matching regex 'kernel_stack_watchdog.cc:.* Thread .* stuck', but found none.
> -- 2026-04-13 02:23:24,773 INFO MainThread: Expected log lines could not
> be found, sleeping before retrying: Expected at least one line in file
> /data0/jenkins/workspace/impala-asf-master-core-asan/repos/Impala/logs/ee_tests/impalad.impala-ec2-redhat86-m6i-4xlarge-ondemand-0759.vpc.cloudera.com.jenkins.log.WARNING.20260413-014350.3130458
> matching regex 'kernel_stack_watchdog.cc:.* Thread .* stuck', but found none.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
