I worry giving up on testing using Python's stdout for the immediate output stream. This is a very useful feature, allowing users to make Python based commands that produce a bunch of output, and stream the output as it is being gathered rather than having the command stall and then dump a bunch of text at the end. This isn't speculative, that's how many of the commands that the OS X kernel team ships for inspecting kernel state work.
So we really should be testing this feature. Maybe the flakiness on Linux is just a pexpect bug, and streaming to stdout would always work correctly hooked up to a "real" terminal. But until we know that we should presume that it is something in LLDB->Python or in the way we use Python, and keep testing it. If you want to separate the two issues, then it would be fine to write another test that just tests the type maps for FILE *, but I still think this one is valuable. Jim > On Jan 14, 2016, at 10:16 AM, Zachary Turner via lldb-commits > <lldb-commits@lists.llvm.org> wrote: > > How much time do you think it would take to determine whether or not using > the file-based approach would work? Because on the surface it sounds fairly > straightforward, and fixing it that way would allow us to not have to xfail > this on more platforms for reasons that we don't understand. > > On Thu, Jan 14, 2016 at 10:15 AM Enrico Granata via lldb-commits > <lldb-commits@lists.llvm.org> wrote: > The log just shows a timeout is happening in pexpect() - which I don’t have a > ready explanation for > > X-failing for now is the proper recourse. But you might want to debug this at > some point, since it’s weird behavior. It looks like the command is not even > just silently erroring out - from the log you sent it looked stuck somewhere.. > > Is there any chance you can step through and see where the hang is happening? > >> On Jan 14, 2016, at 3:03 AM, Tamas Berghammer <tbergham...@google.com> wrote: >> >> I XFAIL-ed the test for Linux to get the build bot green again and filed a >> bug at https://llvm.org/bugs/show_bug.cgi?id=26139 >> >> On Thu, Jan 14, 2016 at 2:18 AM Ying Chen via lldb-commits >> <lldb-commits@lists.llvm.org> wrote: >> Please see attached log file. >> >> Thanks, >> Ying >> >> On Wed, Jan 13, 2016 at 5:39 PM, Enrico Granata <egran...@apple.com> wrote: >> From the buildbot log it’s quite hard to tell what could be going on >> >> Is there any chance you guys could run the test by hand with the “-t -v” >> flags to the dotest.py driver and attach the output of the run? >> >> That might help figure out where the issue lies >> >>> On Jan 13, 2016, at 5:35 PM, Ying Chen <chy...@google.com> wrote: >>> >>> Hello Enrico, >>> >>> The new test has been failing on Ubuntu buildbot. But it's passing on some >>> offline Ubuntu machines, I don't understand what caused the difference. >>> Could you please help to take a look? >>> >>> http://lab.llvm.org:8011/builders/lldb-x86_64-ubuntu-14.04-cmake/builds/10299 >>> >>> Thanks, >>> Ying >>> >>> On Wed, Jan 13, 2016 at 11:32 AM, Enrico Granata via lldb-commits >>> <lldb-commits@lists.llvm.org> wrote: >>> >>>> On Jan 13, 2016, at 11:26 AM, Zachary Turner <ztur...@google.com> wrote: >>>> >>>> Thanks! btw would having the command write its output to a file instead >>>> of stdout solve the pexpect problme as well? >>>> >>> >>> That’s possible - but I would need to play with it a little bit to convince >>> myself that it really is a faithful reproduction of my original issue >>> It’ll take me a little while to get to it - stay tuned. >>> >>>> On Wed, Jan 13, 2016 at 11:22 AM Enrico Granata <egran...@apple.com> wrote: >>>> >>>>> On Jan 13, 2016, at 10:34 AM, Zachary Turner <ztur...@google.com> wrote: >>>>> >>>>> >>>>> >>>>> On Wed, Jan 13, 2016 at 10:25 AM Enrico Granata <egran...@apple.com> >>>>> wrote: >>>>>> On Jan 13, 2016, at 10:21 AM, Zachary Turner <ztur...@google.com> wrote: >>>>>> >>>>>> >>>>>> On Wed, Jan 13, 2016 at 10:15 AM Enrico Granata via lldb-commits >>>>>> <lldb-commits@lists.llvm.org> wrote: >>>>>> + >>>>>> +class CommandScriptImmediateOutputTestCase (PExpectTest): >>>>>> Does the bug that you were trying to fix occur only when using the >>>>>> command_script.py file from the lldb command line? If you load it from >>>>>> within lldb via an LLDB command, does the problem still occur? If the >>>>>> problem you are fixing is not specific to the LLDB command line, I would >>>>>> prefer if you write this not as a pexpect test. >>>>> >>>>> I would love to not touch pexpect :-) But in this case, I can’t see a way >>>>> around it. I am trying to detect whether some text is “physically” >>>>> printed to stdout. And pexpect seems the most obvious straightforward way >>>>> to get that to happen. Note that, in this bug, the result object is >>>>> filled in correctly even if nothing gets printed, so looking at the >>>>> result won’t quite cut it - it really needs to detect output to stdout. >>>>> You're calling result.SetImmediateOutputFile(sys.__stdout__). Wouldn't >>>>> it work to use a file on the file system here, and then you open that >>>>> file and look at it after running the commands? It wouldn't work in >>>>> remote cases, but it already doesn't work on remote cases anyway (as you >>>>> point out below) >>>>> >>>>> >>>>>> >>>>>> + >>>>>> + mydir = TestBase.compute_mydir(__file__) >>>>>> + >>>>>> + def setUp(self): >>>>>> + # Call super's setUp(). >>>>>> + PExpectTest.setUp(self) >>>>>> + >>>>>> + @skipIfRemote # test not remote-ready llvm.org/pr24813 >>>>>> + @expectedFlakeyFreeBSD("llvm.org/pr25172 fails rarely on the >>>>>> buildbot") >>>>>> + @expectedFlakeyLinux("llvm.org/pr25172") >>>>>> Are these necessary? The windows one is necessary (but not if you >>>>>> change this to not being a pexpect test as I've requested above), but >>>>>> why are the other ones necessary? >>>>> >>>>> Do we support remote pexpect? As for FreeBSD and Linux, they might not be >>>>> necessary, but I’d rather much remove them (or let the relevant platform >>>>> owners) remove them in a separate commit >>>>> >>>>> No remote pexpect, so the @skipIfRemote probably needs to be there. But >>>>> I think everyone should be checking in tests enabled by default in the >>>>> widest set of environments possible that you aren't completely sure are >>>>> broken. It should be up to the platform holders to disable broken tests, >>>>> not to enable working tests. Because it's much easier to notice a broken >>>>> test going in than it is to notice a working test went in disabled >>>>> (because who's going to think to test it out?). >>>> >>>> This is a fair point. I’ll enable those platforms in a subsequent commit >>>> ASAP >>>> >>>> >>>> Thanks, >>>> - Enrico >>>> 📩 egranata@.com ☎️ 27683 >>>> >>> >>> >>> Thanks, >>> - Enrico >>> 📩 egranata@.com ☎️ 27683 >>> >>> >>> _______________________________________________ >>> lldb-commits mailing list >>> lldb-commits@lists.llvm.org >>> http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-commits >>> >>> >> >> >> Thanks, >> - Enrico >> 📩 egranata@.com ☎️ 27683 >> >> >> _______________________________________________ >> lldb-commits mailing list >> lldb-commits@lists.llvm.org >> http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-commits > > > Thanks, > - Enrico > 📩 egranata@.com ☎️ 27683 > > _______________________________________________ > lldb-commits mailing list > lldb-commits@lists.llvm.org > http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-commits > _______________________________________________ > lldb-commits mailing list > lldb-commits@lists.llvm.org > http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-commits _______________________________________________ lldb-commits mailing list lldb-commits@lists.llvm.org http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-commits
