[lldb-dev] [Bug 25785] New: TestReturnValue fails remotely windows->android

https://llvm.org/bugs/show_bug.cgi?id=25785 Bug ID: 25785 Summary: TestReturnValue fails remotely windows->android Product: lldb Version: unspecified Hardware: PC OS: other Status: NEW Severity: normal

[lldb-dev] BasicResultsFormatter - new test results summary

Hi all, Per a previous thread on this, I've made all the changes I intended to make last night to get the intended replacement of test run results meet or exceed current requirements. I'd like to switch over to that by default. I'm depending on the test event system to be able to handle test met

Re: [lldb-dev] BasicResultsFormatter - new test results summary

That's a good point, Tamas. I use (so I claim) the same all upper-case markers for the test result details. Including, not using XPASS but rather UNEXPECTED SUCCESS for unexpected successes. (The former would trigger the lit script IIRC to parse that as a failing-style result). The intent is th

Re: [lldb-dev] BasicResultsFormatter - new test results summary

Specifically, the markers for issue details are: FAIL ERROR UNEXPECTED SUCCESS TIMEOUT (These are the fourth field in the array entries (lines 275 - 290) of packages/Python/lldbsuite/test/basic_results_formatter.py). -Todd On Wed, Dec 9, 2015 at 8:04 AM, Todd Fiala wrote: > That's a good poin

Re: [lldb-dev] BasicResultsFormatter - new test results summary

+Ying Chen Ying, what do we have to do on the build bot side to support a change in the default test result summary formatter? On Wed, Dec 9, 2015 at 4:00 PM Todd Fiala via lldb-dev < lldb-dev@lists.llvm.org> wrote: > Hi all, > > Per a previous thread on this, I've made all the changes I intend

Re: [lldb-dev] BasicResultsFormatter - new test results summary

Here's what I can do. Put in the change (setting the default to use the new format). Separately, put in a trial balloon commit with one failing test, one exceptional exit test, and one timeout test, and watch the ubuntu 14.04 buildbot catch it and fail. Then reverse this out. That should show b

Re: [lldb-dev] BasicResultsFormatter - new test results summary

It is a small change. I almost have all the trial tests ready, so I'll just commit both changes at the same time (the flip on, and the trial balloon issues). If all goes well and the three types of issue show up, then the last of the two will get reverted (the one with the failures). If none (or

Re: [lldb-dev] BasicResultsFormatter - new test results summary

If it's not too much work, I think the extra bit of noise will not be a problem. But I don't think it is really necessary either. I assume the actual flip will be a small change that we can back out easily if we notice troubles... After a sufficient grace period we can remove the old formatter alt

Re: [lldb-dev] BasicResultsFormatter - new test results summary

These went in as: r255130 - turn it on by default r255131 - create known issues. This one is to be reverted if all 3 types show up properly. On Wed, Dec 9, 2015 at 9:41 AM, Todd Fiala wrote: > It is a small change. > > I almost have all the trial tests ready, so I'll just commit both changes >

Re: [lldb-dev] BasicResultsFormatter - new test results summary

I forced a build on the ubuntu 14.04 cmake builder. The build _after_ 9292 will contain the two changes (and we will expect failures on it). On Wed, Dec 9, 2015 at 10:05 AM, Todd Fiala wrote: > These went in as: > > r255130 - turn it on by default > r255131 - create known issues. This one is t

Re: [lldb-dev] BasicResultsFormatter - new test results summary

I am going to stop the current build on that builder. There was one change in it, and it will be another 20 minutes before it completes. I don't want the repo in a known broken state that long. On Wed, Dec 9, 2015 at 10:07 AM, Todd Fiala wrote: > I forced a build on the ubuntu 14.04 cmake buil

Re: [lldb-dev] BasicResultsFormatter - new test results summary

The reports look good at the test level: http://lab.llvm.org:8011/builders/lldb-x86_64-ubuntu-14.04-cmake/builds/9294 I'd say the buildbot reflection script missed the ERROR, so that is something maybe Ying can look at (the summary line in the build run), but that is unrelated AFAICT. I'm going

Re: [lldb-dev] BasicResultsFormatter - new test results summary

Verification tests parked (i.e. disabled) here: r255134 I decided to leave them in the repo so it is faster/easier to do this in the future. -Todd On Wed, Dec 9, 2015 at 10:26 AM, Todd Fiala wrote: > The reports look good at the test level: > > > http://lab.llvm.org:8011/builders/lldb-x86_64-u

Re: [lldb-dev] BasicResultsFormatter - new test results summary

Thank you for making the experiment. It looks reasonable. For the ERROR the buildbot detected it and it will fail the build but it isn't listed in the list of failing tests what should be fixed. After this experiment I think it is fine to change the default output formatter from our side. Tamas O

Re: [lldb-dev] BasicResultsFormatter - new test results summary

Great, thanks Tamas! I left the default turned on, and just essentially removed the issues by parking them as .py.parked files. That way we can flip them on in the future if we want to verify a testbot's detection of these. I will be going back to the xUnit Results formatter and making sure it m

Re: [lldb-dev] BasicResultsFormatter - new test results summary

I submitted this patch to include "ERROR" lines in buildbot step results. http://reviews.llvm.org/rL255145 Error results will be displayed in step result like this after the patch, "ERROR: 9 (SIGKILL) test_buildbot_catches_exceptional_exit_dwarf" Thanks, Ying On Wed, Dec 9, 2015 at 10:45 AM, Tod

[lldb-dev] Benchmark tests

Is anyone using the benchmark tests? None of the command line options related to the benchmark tests were claimed as being used by anyone. Which makes me wonder if the tests are even being used

Re: [lldb-dev] LLDB/NetBSD test suite results as of 8 Dec 2015

-BEGIN PGP SIGNED MESSAGE- Hash: SHA256 On 08.12.2015 17:21, Ed Maste wrote: > On 7 December 2015 at 19:28, Kamil Rytarowski via lldb-dev > wrote: >> >> I ran the LLDB test suite of the LLDB-current version on >> NetBSD/amd64 (v. 7.99.21). >> >> The results are as follows: >> >> 415 o

Re: [lldb-dev] Benchmark tests

Hey Jason, Are you the benchmark user? -Todd On Wed, Dec 9, 2015 at 12:32 PM, Zachary Turner via lldb-dev < lldb-dev@lists.llvm.org> wrote: > Is anyone using the benchmark tests? None of the command line options >

[lldb-dev] swig generation and "six" module

r252764 changes finishSwigPythonLLDB.py to symlink the "six" module in site-packages. six.py is a symlink to /tools/lldb/third_party/Python/module/six/six.py. This assumes I have access to this build's sources when I run lldb, which I don't - our workstations don't have access to the buildbots' fil

Re: [lldb-dev] swig generation and "six" module

What about requiring that the user has done "pip install six" on their machine? On Wed, Dec 9, 2015 at 1:40 PM Ted Woodward via lldb-dev < lldb-dev@lists.llvm.org> wrote: > r252764 changes finishSwigPythonLLDB.py to symlink the “six” module in > site-packages. six.py is a symlink to > /tools/lldb

Re: [lldb-dev] Benchmark tests

I have actually added a few benchmark tests recently. We admittedly are not that good with running those tests ever (because they're not run by default most likely - and I do wonder if some of them would take a long time to run.. I don't think I have ever run the full set, just my own as I incre

Re: [lldb-dev] Benchmark tests

When you do run the benchmark tests, what command line options do you use? At the moment I'm mostly just trying to remove dead options from the test suite. I removed one already that allowed you to specify the benchmark executable, but then when I started looking at the rest and seeing how tightly

Re: [lldb-dev] Benchmark tests

FWIW, nope, I've never messed with the benchmark tests. > On Dec 9, 2015, at 1:22 PM, Todd Fiala wrote: > > Hey Jason, > > Are you the benchmark user? > > -Todd > > On Wed, Dec 9, 2015 at 12:32 PM, Zachary Turner via lldb-dev > wrote: > Is anyone using the benchmark tests? None of the comm

Re: [lldb-dev] swig generation and "six" module

Unfortunately, the LLDB we build needs to run on SLES 11, Ubuntu 10+ and who knows what else. I don’t think I can require they install a six package. I can make our build do a copy instead of a link, if you want to keep it a link. Are the static swig bindings checked in yet, or do we still n

Re: [lldb-dev] Benchmark tests

Historically I would do $ ./dotest.py +b but I am not strongly attached to that workflow - it's just what I learnt the first time I needed to run one Sent from my iPhone > On Dec 9, 2015, at 2:08 PM, Zachary Turner wrote: > > When you do run the benchmark tests, what command line options do

Re: [lldb-dev] swig generation and "six" module

I don't mind it being a copy instead of a link, I had just been thinking about getting rid of that workflow anyway and requiring a pip install. But if you can't do that, then scratch that idea. Feel free to change it to a copy and upstream that if it makes your life easier. Just to be clear thou