Bug#1060775: tmuxp: flaky autopkgtest: assert pane.capture_pane()[1] -> IndexError

[Paul Gevers]

Source: tmuxp
Version: 1.31.0-2
Severity: serious
User: debian...@lists.debian.org
Usertags: flaky

Dear maintainer(s),



Because the unstable-to-testing migration software now blocks on
regressions in testing, flaky tests, i.e. tests that flip between
passing and failing without changes to the list of installed packages,
are causing people unrelated to your package to spend time on these
tests.

Don't hesitate to reach out if you need help and some more information
from our infrastructure.

Paul

From
https://ci.debian.net/packages/t/tmuxp/testing/s390x/41364351/

 59s     @pytest.mark.skipif(
 59s         has_lt_version("3.0"),

 59s     )
 59s     def test_environment_variables(session: Session) -> None:
 59s         workspace = ConfigReader._from_file(

 59s         )
 59s         workspace = loader.expand(workspace)
 59s

 59s         builder.build(session)

 59s         time.sleep(0.3)
 59s
 59s         assert session.getenv("FOO") == "SESSION"
 59s         assert session.getenv("PATH") == "/tmp"
 59s
 59s         no_overrides_win = session.windows[0]
 59s         pane = no_overrides_win.panes[0]
 59s         pane.send_keys("echo $FOO")
 59s         assert pane.capture_pane()[1] == "SESSION"
 59s
 59s         window_overrides_win = session.windows[1]
 59s         pane = window_overrides_win.panes[0]
 59s         pane.send_keys("echo $FOO")
 59s         assert pane.capture_pane()[1] == "WINDOW"
 59s
 59s         pane_overrides_win = session.windows[2]
 59s         pane = pane_overrides_win.panes[0]
 59s         pane.send_keys("echo $FOO")
 59s         assert pane.capture_pane()[1] == "PANE"
 59s
 59s         both_overrides_win = session.windows[3]
 59s         pane = both_overrides_win.panes[0]
 59s         pane.send_keys("echo $FOO")
 59s >       assert pane.capture_pane()[1] == "WINDOW"
 59s E       IndexError: list index out of range
 59s

I looked at the results of the autopkgtest of your package. I noticed that it regularly fails. I'm seeing a pattern that it happens more often on our amd64 host ci-worker13 and often on our s390x which are rather powerful hosts. The test fails in different locations, but as far as I saw always on a line that asserts capture_pane (see example below). Could it be that the code (either the real code or the test code) should wait a bit until the "pane" has been substantiated? Or maybe that comment earlier is of interest: Give slow shells some time to settle as otherwise tests might fail. 59s reason="needs -e flag for new-window and split-window introduced in tmux 3.0", 59s test_utils.get_workspace_file("workspace/builder/environment_vars.yaml") 59s builder = WorkspaceBuilder(session_config=workspace, server=session.server) 59s # Give slow shells some time to settle as otherwise tests might fail.

OpenPGP_signature.asc
Description: OpenPGP digital signature