[
https://issues.apache.org/jira/browse/KAFKA-17553?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17881712#comment-17881712
]
David Arthur edited comment on KAFKA-17553 at 9/13/24 10:58 PM:
----------------------------------------------------------------
Actually this looks like the culprit
{code:java}
"clientId-StreamThread-1-TaskExecutor-1" #12953 prio=5 os_prio=0 cpu=0.59ms
elapsed=5651.50s tid=0x00007f44fe5f5f80 nid=0x7d8b2 waiting on condition
[0x00007f44967fa000]
java.lang.Thread.State: WAITING (parking)
at jdk.internal.misc.Unsafe.park([email protected]/Native Method)
- parking to wait for <0x00000000e31a06b8> (a
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at
java.util.concurrent.locks.LockSupport.park([email protected]/LockSupport.java:341)
at
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionNode.block([email protected]/AbstractQueuedSynchronizer.java:506)
at
java.util.concurrent.ForkJoinPool.unmanagedBlock([email protected]/ForkJoinPool.java:3465)
at
java.util.concurrent.ForkJoinPool.managedBlock([email protected]/ForkJoinPool.java:3436)
at
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await([email protected]/AbstractQueuedSynchronizer.java:1625)
at
org.apache.kafka.streams.processor.internals.tasks.DefaultTaskManager.lambda$awaitProcessableTasks$1(DefaultTaskManager.java:142)
at
org.apache.kafka.streams.processor.internals.tasks.DefaultTaskManager$$Lambda$5698/0x00007f449cd66d38.get(Unknown
Source)
at
org.apache.kafka.streams.processor.internals.tasks.DefaultTaskManager.returnWithTasksLocked(DefaultTaskManager.java:353)
at
org.apache.kafka.streams.processor.internals.tasks.DefaultTaskManager.awaitProcessableTasks(DefaultTaskManager.java:129)
at
org.apache.kafka.streams.processor.internals.tasks.DefaultTaskExecutor$TaskExecutorThread.runOnce(DefaultTaskExecutor.java:119)
at
org.apache.kafka.streams.processor.internals.tasks.DefaultTaskExecutor$TaskExecutorThread.run(DefaultTaskExecutor.java:62)
{code}
In DefaultTaskManager#awaitProcessableTasks we are awaiting a condition
variable indefinitely.
cc [~mjsax] [~lucasbru]
was (Author: davidarthur):
Actually this looks like the culprit
{code:java}
"clientId-StreamThread-1-TaskExecutor-1" #12953 prio=5 os_prio=0 cpu=0.59ms
elapsed=5651.50s tid=0x00007f44fe5f5f80 nid=0x7d8b2 waiting on condition
[0x00007f44967fa000]
java.lang.Thread.State: WAITING (parking)
at jdk.internal.misc.Unsafe.park([email protected]/Native Method)
- parking to wait for <0x00000000e31a06b8> (a
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at
java.util.concurrent.locks.LockSupport.park([email protected]/LockSupport.java:341)
at
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionNode.block([email protected]/AbstractQueuedSynchronizer.java:506)
at
java.util.concurrent.ForkJoinPool.unmanagedBlock([email protected]/ForkJoinPool.java:3465)
at
java.util.concurrent.ForkJoinPool.managedBlock([email protected]/ForkJoinPool.java:3436)
at
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await([email protected]/AbstractQueuedSynchronizer.java:1625)
at
org.apache.kafka.streams.processor.internals.tasks.DefaultTaskManager.lambda$awaitProcessableTasks$1(DefaultTaskManager.java:142)
at
org.apache.kafka.streams.processor.internals.tasks.DefaultTaskManager$$Lambda$5698/0x00007f449cd66d38.get(Unknown
Source)
at
org.apache.kafka.streams.processor.internals.tasks.DefaultTaskManager.returnWithTasksLocked(DefaultTaskManager.java:353)
at
org.apache.kafka.streams.processor.internals.tasks.DefaultTaskManager.awaitProcessableTasks(DefaultTaskManager.java:129)
at
org.apache.kafka.streams.processor.internals.tasks.DefaultTaskExecutor$TaskExecutorThread.runOnce(DefaultTaskExecutor.java:119)
at
org.apache.kafka.streams.processor.internals.tasks.DefaultTaskExecutor$TaskExecutorThread.run(DefaultTaskExecutor.java:62)
{code}
In DefaultTaskManager#awaitProcessableTasks we are awaiting a condition
variable indefinitely.
> DefaultTaskManager.shutdown stalling on Github CI
> -------------------------------------------------
>
> Key: KAFKA-17553
> URL: https://issues.apache.org/jira/browse/KAFKA-17553
> Project: Kafka
> Issue Type: Bug
> Components: build, streams
> Reporter: David Arthur
> Priority: Major
> Attachments: GradleWorkerMain-142716.txt
>
>
> After merging [https://github.com/apache/kafka/pull/17181,] we now have
> thread dumps available for build timeouts. Coincidentally, the trunk build of
> that very PR encountered a timeout.
> [https://github.com/apache/kafka/actions/runs/10851711578]
> It looks like we are getting stuck in a shutdown method
> {code:java}
> "stream-thread-test-87bf53a8-54f2-485f-a4b6-acdbec0a8b3d-StreamThread-1" #1
> prio=5 os_prio=0 cpu=42316.77ms elapsed=8450.21s tid=0x00007f44fc02c700
> nid=0x22d7f in Object.wait() [0x00007f45041fa000]
> java.lang.Thread.State: TIMED_WAITING (on object monitor)
> at java.lang.Object.wait([email protected]/Native Method)
> - waiting on <no object reference available>
> at java.lang.Thread.join([email protected]/Thread.java:1307)
> - locked <0x00000000e31a0aa8> (a
> org.apache.kafka.streams.processor.internals.tasks.DefaultTaskExecutor$TaskExecutorThread)
> at
> org.apache.kafka.streams.processor.internals.tasks.DefaultTaskExecutor.awaitShutdown(DefaultTaskExecutor.java:264)
> at
> org.apache.kafka.streams.processor.internals.tasks.DefaultTaskManager.shutdown(DefaultTaskManager.java:377)
> {code}
>
> Attached is the full thread dump
> [^GradleWorkerMain-142716.txt]
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
