[lldb-dev] [Bug 25785] New: TestReturnValue fails remotely windows->android

2015-12-09 Thread via lldb-dev 
https://llvm.org/bugs/show_bug.cgi?id=25785

Bug ID: 25785
   Summary: TestReturnValue fails remotely windows->android
   Product: lldb
   Version: unspecified
  Hardware: PC
OS: other
Status: NEW
  Severity: normal
  Priority: P
 Component: All Bugs
  Assignee: lldb-dev@lists.llvm.org
  Reporter: lab...@google.com
CC: llvm-b...@lists.llvm.org
Classification: Unclassified

I will XFAIL the test until we get a chance to investigate the root cause.

-- 
You are receiving this mail because:
You are the assignee for the bug.
___
lldb-dev mailing list
lldb-dev@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev

[lldb-dev] BasicResultsFormatter - new test results summary

2015-12-09 Thread Todd Fiala via lldb-dev 
Hi all,

Per a previous thread on this, I've made all the changes I intended to make
last night to get the intended replacement of test run results meet or
exceed current requirements.

I'd like to switch over to that by default.  I'm depending on the test
event system to be able to handle test method reruns in test results
accounting.

The primary thing missing before was that timeouts were not routed through
the test events system, nor were exception process exits (i.e. test
inferiors exiting with a signal on POSIX systems).  Those were added last
night so that test events are generated for those, and the
BasicResultsFormatter presents that information properly.

I will switch it over to being the default output in a bit here.  Please
let me know if you have any concerns once I flip it on by default.

Thanks!
-- 
-Todd
___
lldb-dev mailing list
lldb-dev@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev

Re: [lldb-dev] BasicResultsFormatter - new test results summary

2015-12-09 Thread Todd Fiala via lldb-dev 
That's a good point, Tamas.

I use (so I claim) the same all upper-case markers for the test result
details.  Including, not using XPASS but rather UNEXPECTED SUCCESS for
unexpected successes.  (The former would trigger the lit script IIRC to
parse that as a failing-style result).

The intent is this is a no-op on the test runner.

On Wed, Dec 9, 2015 at 8:02 AM, Tamas Berghammer 
wrote:

> +Ying Chen 
>
> Ying, what do we have to do on the build bot side to support a change in
> the default test result summary formatter?
>
> On Wed, Dec 9, 2015 at 4:00 PM Todd Fiala via lldb-dev <
> lldb-dev@lists.llvm.org> wrote:
>
>> Hi all,
>>
>> Per a previous thread on this, I've made all the changes I intended to
>> make last night to get the intended replacement of test run results meet or
>> exceed current requirements.
>>
>> I'd like to switch over to that by default.  I'm depending on the test
>> event system to be able to handle test method reruns in test results
>> accounting.
>>
>> The primary thing missing before was that timeouts were not routed
>> through the test events system, nor were exception process exits (i.e. test
>> inferiors exiting with a signal on POSIX systems).  Those were added last
>> night so that test events are generated for those, and the
>> BasicResultsFormatter presents that information properly.
>>
>> I will switch it over to being the default output in a bit here.  Please
>> let me know if you have any concerns once I flip it on by default.
>>
>> Thanks!
>> --
>> -Todd
>> ___
>> lldb-dev mailing list
>> lldb-dev@lists.llvm.org
>> http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev
>>
>


-- 
-Todd
___
lldb-dev mailing list
lldb-dev@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev

Re: [lldb-dev] BasicResultsFormatter - new test results summary

2015-12-09 Thread Todd Fiala via lldb-dev 
Specifically, the markers for issue details are:

FAIL
ERROR
UNEXPECTED SUCCESS
TIMEOUT

(These are the fourth field in the array entries (lines 275 - 290) of
packages/Python/lldbsuite/test/basic_results_formatter.py).

-Todd

On Wed, Dec 9, 2015 at 8:04 AM, Todd Fiala  wrote:

> That's a good point, Tamas.
>
> I use (so I claim) the same all upper-case markers for the test result
> details.  Including, not using XPASS but rather UNEXPECTED SUCCESS for
> unexpected successes.  (The former would trigger the lit script IIRC to
> parse that as a failing-style result).
>
> The intent is this is a no-op on the test runner.
>
> On Wed, Dec 9, 2015 at 8:02 AM, Tamas Berghammer 
> wrote:
>
>> +Ying Chen 
>>
>> Ying, what do we have to do on the build bot side to support a change in
>> the default test result summary formatter?
>>
>> On Wed, Dec 9, 2015 at 4:00 PM Todd Fiala via lldb-dev <
>> lldb-dev@lists.llvm.org> wrote:
>>
>>> Hi all,
>>>
>>> Per a previous thread on this, I've made all the changes I intended to
>>> make last night to get the intended replacement of test run results meet or
>>> exceed current requirements.
>>>
>>> I'd like to switch over to that by default.  I'm depending on the test
>>> event system to be able to handle test method reruns in test results
>>> accounting.
>>>
>>> The primary thing missing before was that timeouts were not routed
>>> through the test events system, nor were exception process exits (i.e. test
>>> inferiors exiting with a signal on POSIX systems).  Those were added last
>>> night so that test events are generated for those, and the
>>> BasicResultsFormatter presents that information properly.
>>>
>>> I will switch it over to being the default output in a bit here.  Please
>>> let me know if you have any concerns once I flip it on by default.
>>>
>>> Thanks!
>>> --
>>> -Todd
>>> ___
>>> lldb-dev mailing list
>>> lldb-dev@lists.llvm.org
>>> http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev
>>>
>>
>
>
> --
> -Todd
>



-- 
-Todd
___
lldb-dev mailing list
lldb-dev@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev

Re: [lldb-dev] BasicResultsFormatter - new test results summary

2015-12-09 Thread Tamas Berghammer via lldb-dev 
+Ying Chen 

Ying, what do we have to do on the build bot side to support a change in
the default test result summary formatter?

On Wed, Dec 9, 2015 at 4:00 PM Todd Fiala via lldb-dev <
lldb-dev@lists.llvm.org> wrote:

> Hi all,
>
> Per a previous thread on this, I've made all the changes I intended to
> make last night to get the intended replacement of test run results meet or
> exceed current requirements.
>
> I'd like to switch over to that by default.  I'm depending on the test
> event system to be able to handle test method reruns in test results
> accounting.
>
> The primary thing missing before was that timeouts were not routed through
> the test events system, nor were exception process exits (i.e. test
> inferiors exiting with a signal on POSIX systems).  Those were added last
> night so that test events are generated for those, and the
> BasicResultsFormatter presents that information properly.
>
> I will switch it over to being the default output in a bit here.  Please
> let me know if you have any concerns once I flip it on by default.
>
> Thanks!
> --
> -Todd
> ___
> lldb-dev mailing list
> lldb-dev@lists.llvm.org
> http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev
>
___
lldb-dev mailing list
lldb-dev@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev

Re: [lldb-dev] BasicResultsFormatter - new test results summary

2015-12-09 Thread Todd Fiala via lldb-dev 
Here's what I can do.

Put in the change (setting the default to use the new format).

Separately, put in a trial balloon commit with one failing test, one
exceptional exit test, and one timeout test, and watch the ubuntu 14.04
buildbot catch it and fail.  Then reverse this out.  That should show
beyond a reasonable doubt whether the buildbot catches new failures and
errors.  (I think this is a noisy way to accomplish this, but it certainly
would validate if its working).

-Todd

On Wed, Dec 9, 2015 at 8:06 AM, Todd Fiala  wrote:

> Specifically, the markers for issue details are:
>
> FAIL
> ERROR
> UNEXPECTED SUCCESS
> TIMEOUT
>
> (These are the fourth field in the array entries (lines 275 - 290) of
> packages/Python/lldbsuite/test/basic_results_formatter.py).
>
> -Todd
>
> On Wed, Dec 9, 2015 at 8:04 AM, Todd Fiala  wrote:
>
>> That's a good point, Tamas.
>>
>> I use (so I claim) the same all upper-case markers for the test result
>> details.  Including, not using XPASS but rather UNEXPECTED SUCCESS for
>> unexpected successes.  (The former would trigger the lit script IIRC to
>> parse that as a failing-style result).
>>
>> The intent is this is a no-op on the test runner.
>>
>> On Wed, Dec 9, 2015 at 8:02 AM, Tamas Berghammer 
>> wrote:
>>
>>> +Ying Chen 
>>>
>>> Ying, what do we have to do on the build bot side to support a change in
>>> the default test result summary formatter?
>>>
>>> On Wed, Dec 9, 2015 at 4:00 PM Todd Fiala via lldb-dev <
>>> lldb-dev@lists.llvm.org> wrote:
>>>
 Hi all,

 Per a previous thread on this, I've made all the changes I intended to
 make last night to get the intended replacement of test run results meet or
 exceed current requirements.

 I'd like to switch over to that by default.  I'm depending on the test
 event system to be able to handle test method reruns in test results
 accounting.

 The primary thing missing before was that timeouts were not routed
 through the test events system, nor were exception process exits (i.e. test
 inferiors exiting with a signal on POSIX systems).  Those were added last
 night so that test events are generated for those, and the
 BasicResultsFormatter presents that information properly.

 I will switch it over to being the default output in a bit here.
 Please let me know if you have any concerns once I flip it on by default.

 Thanks!
 --
 -Todd
 ___
 lldb-dev mailing list
 lldb-dev@lists.llvm.org
 http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev

>>>
>>
>>
>> --
>> -Todd
>>
>
>
>
> --
> -Todd
>



-- 
-Todd
___
lldb-dev mailing list
lldb-dev@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev

Re: [lldb-dev] BasicResultsFormatter - new test results summary

2015-12-09 Thread Todd Fiala via lldb-dev 
It is a small change.

I almost have all the trial tests ready, so I'll just commit both changes
at the same time (the flip on, and the trial balloon issues).

If all goes well and the three types of issue show up, then the last of the
two will get reverted (the one with the failures).

If none (or only some) of the issues show up, they'll both get reverted.

-Todd

On Wed, Dec 9, 2015 at 9:39 AM, Pavel Labath  wrote:

> If it's not too much work, I think the extra bit of noise will not be
> a problem. But I don't think it is really necessary either.
>
> I assume the actual flip will be a small change that we can back out
> easily if we notice troubles... After a sufficient grace period we can
> remove the old formatter altogether and hopefully simplify the code
> somewhat.
>
> pl
>
> On 9 December 2015 at 17:08, Todd Fiala  wrote:
> > Here's what I can do.
> >
> > Put in the change (setting the default to use the new format).
> >
> > Separately, put in a trial balloon commit with one failing test, one
> > exceptional exit test, and one timeout test, and watch the ubuntu 14.04
> > buildbot catch it and fail.  Then reverse this out.  That should show
> beyond
> > a reasonable doubt whether the buildbot catches new failures and
> errors.  (I
> > think this is a noisy way to accomplish this, but it certainly would
> > validate if its working).
> >
> > -Todd
> >
> > On Wed, Dec 9, 2015 at 8:06 AM, Todd Fiala  wrote:
> >>
> >> Specifically, the markers for issue details are:
> >>
> >> FAIL
> >> ERROR
> >> UNEXPECTED SUCCESS
> >> TIMEOUT
> >>
> >> (These are the fourth field in the array entries (lines 275 - 290) of
> >> packages/Python/lldbsuite/test/basic_results_formatter.py).
> >>
> >> -Todd
> >>
> >> On Wed, Dec 9, 2015 at 8:04 AM, Todd Fiala 
> wrote:
> >>>
> >>> That's a good point, Tamas.
> >>>
> >>> I use (so I claim) the same all upper-case markers for the test result
> >>> details.  Including, not using XPASS but rather UNEXPECTED SUCCESS for
> >>> unexpected successes.  (The former would trigger the lit script IIRC to
> >>> parse that as a failing-style result).
> >>>
> >>> The intent is this is a no-op on the test runner.
> >>>
> >>> On Wed, Dec 9, 2015 at 8:02 AM, Tamas Berghammer <
> tbergham...@google.com>
> >>> wrote:
> 
>  +Ying Chen
> 
>  Ying, what do we have to do on the build bot side to support a change
> in
>  the default test result summary formatter?
> 
>  On Wed, Dec 9, 2015 at 4:00 PM Todd Fiala via lldb-dev
>   wrote:
> >
> > Hi all,
> >
> > Per a previous thread on this, I've made all the changes I intended
> to
> > make last night to get the intended replacement of test run results
> meet or
> > exceed current requirements.
> >
> > I'd like to switch over to that by default.  I'm depending on the
> test
> > event system to be able to handle test method reruns in test results
> > accounting.
> >
> > The primary thing missing before was that timeouts were not routed
> > through the test events system, nor were exception process exits
> (i.e. test
> > inferiors exiting with a signal on POSIX systems).  Those were added
> last
> > night so that test events are generated for those, and the
> > BasicResultsFormatter presents that information properly.
> >
> > I will switch it over to being the default output in a bit here.
> > Please let me know if you have any concerns once I flip it on by
> default.
> >
> > Thanks!
> > --
> > -Todd
> > ___
> > lldb-dev mailing list
> > lldb-dev@lists.llvm.org
> > http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev
> >>>
> >>>
> >>>
> >>>
> >>> --
> >>> -Todd
> >>
> >>
> >>
> >>
> >> --
> >> -Todd
> >
> >
> >
> >
> > --
> > -Todd
>



-- 
-Todd
___
lldb-dev mailing list
lldb-dev@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev

Re: [lldb-dev] BasicResultsFormatter - new test results summary

2015-12-09 Thread Pavel Labath via lldb-dev 
If it's not too much work, I think the extra bit of noise will not be
a problem. But I don't think it is really necessary either.

I assume the actual flip will be a small change that we can back out
easily if we notice troubles... After a sufficient grace period we can
remove the old formatter altogether and hopefully simplify the code
somewhat.

pl

On 9 December 2015 at 17:08, Todd Fiala  wrote:
> Here's what I can do.
>
> Put in the change (setting the default to use the new format).
>
> Separately, put in a trial balloon commit with one failing test, one
> exceptional exit test, and one timeout test, and watch the ubuntu 14.04
> buildbot catch it and fail.  Then reverse this out.  That should show beyond
> a reasonable doubt whether the buildbot catches new failures and errors.  (I
> think this is a noisy way to accomplish this, but it certainly would
> validate if its working).
>
> -Todd
>
> On Wed, Dec 9, 2015 at 8:06 AM, Todd Fiala  wrote:
>>
>> Specifically, the markers for issue details are:
>>
>> FAIL
>> ERROR
>> UNEXPECTED SUCCESS
>> TIMEOUT
>>
>> (These are the fourth field in the array entries (lines 275 - 290) of
>> packages/Python/lldbsuite/test/basic_results_formatter.py).
>>
>> -Todd
>>
>> On Wed, Dec 9, 2015 at 8:04 AM, Todd Fiala  wrote:
>>>
>>> That's a good point, Tamas.
>>>
>>> I use (so I claim) the same all upper-case markers for the test result
>>> details.  Including, not using XPASS but rather UNEXPECTED SUCCESS for
>>> unexpected successes.  (The former would trigger the lit script IIRC to
>>> parse that as a failing-style result).
>>>
>>> The intent is this is a no-op on the test runner.
>>>
>>> On Wed, Dec 9, 2015 at 8:02 AM, Tamas Berghammer 
>>> wrote:

 +Ying Chen

 Ying, what do we have to do on the build bot side to support a change in
 the default test result summary formatter?

 On Wed, Dec 9, 2015 at 4:00 PM Todd Fiala via lldb-dev
  wrote:
>
> Hi all,
>
> Per a previous thread on this, I've made all the changes I intended to
> make last night to get the intended replacement of test run results meet 
> or
> exceed current requirements.
>
> I'd like to switch over to that by default.  I'm depending on the test
> event system to be able to handle test method reruns in test results
> accounting.
>
> The primary thing missing before was that timeouts were not routed
> through the test events system, nor were exception process exits (i.e. 
> test
> inferiors exiting with a signal on POSIX systems).  Those were added last
> night so that test events are generated for those, and the
> BasicResultsFormatter presents that information properly.
>
> I will switch it over to being the default output in a bit here.
> Please let me know if you have any concerns once I flip it on by default.
>
> Thanks!
> --
> -Todd
> ___
> lldb-dev mailing list
> lldb-dev@lists.llvm.org
> http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev
>>>
>>>
>>>
>>>
>>> --
>>> -Todd
>>
>>
>>
>>
>> --
>> -Todd
>
>
>
>
> --
> -Todd
___
lldb-dev mailing list
lldb-dev@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev

Re: [lldb-dev] BasicResultsFormatter - new test results summary

2015-12-09 Thread Todd Fiala via lldb-dev 
These went in as:

r255130 - turn it on by default
r255131 - create known issues.  This one is to be reverted if all 3 types
show up properly.

On Wed, Dec 9, 2015 at 9:41 AM, Todd Fiala  wrote:

> It is a small change.
>
> I almost have all the trial tests ready, so I'll just commit both changes
> at the same time (the flip on, and the trial balloon issues).
>
> If all goes well and the three types of issue show up, then the last of
> the two will get reverted (the one with the failures).
>
> If none (or only some) of the issues show up, they'll both get reverted.
>
> -Todd
>
> On Wed, Dec 9, 2015 at 9:39 AM, Pavel Labath  wrote:
>
>> If it's not too much work, I think the extra bit of noise will not be
>> a problem. But I don't think it is really necessary either.
>>
>> I assume the actual flip will be a small change that we can back out
>> easily if we notice troubles... After a sufficient grace period we can
>> remove the old formatter altogether and hopefully simplify the code
>> somewhat.
>>
>> pl
>>
>> On 9 December 2015 at 17:08, Todd Fiala  wrote:
>> > Here's what I can do.
>> >
>> > Put in the change (setting the default to use the new format).
>> >
>> > Separately, put in a trial balloon commit with one failing test, one
>> > exceptional exit test, and one timeout test, and watch the ubuntu 14.04
>> > buildbot catch it and fail.  Then reverse this out.  That should show
>> beyond
>> > a reasonable doubt whether the buildbot catches new failures and
>> errors.  (I
>> > think this is a noisy way to accomplish this, but it certainly would
>> > validate if its working).
>> >
>> > -Todd
>> >
>> > On Wed, Dec 9, 2015 at 8:06 AM, Todd Fiala 
>> wrote:
>> >>
>> >> Specifically, the markers for issue details are:
>> >>
>> >> FAIL
>> >> ERROR
>> >> UNEXPECTED SUCCESS
>> >> TIMEOUT
>> >>
>> >> (These are the fourth field in the array entries (lines 275 - 290) of
>> >> packages/Python/lldbsuite/test/basic_results_formatter.py).
>> >>
>> >> -Todd
>> >>
>> >> On Wed, Dec 9, 2015 at 8:04 AM, Todd Fiala 
>> wrote:
>> >>>
>> >>> That's a good point, Tamas.
>> >>>
>> >>> I use (so I claim) the same all upper-case markers for the test result
>> >>> details.  Including, not using XPASS but rather UNEXPECTED SUCCESS for
>> >>> unexpected successes.  (The former would trigger the lit script IIRC
>> to
>> >>> parse that as a failing-style result).
>> >>>
>> >>> The intent is this is a no-op on the test runner.
>> >>>
>> >>> On Wed, Dec 9, 2015 at 8:02 AM, Tamas Berghammer <
>> tbergham...@google.com>
>> >>> wrote:
>> 
>>  +Ying Chen
>> 
>>  Ying, what do we have to do on the build bot side to support a
>> change in
>>  the default test result summary formatter?
>> 
>>  On Wed, Dec 9, 2015 at 4:00 PM Todd Fiala via lldb-dev
>>   wrote:
>> >
>> > Hi all,
>> >
>> > Per a previous thread on this, I've made all the changes I intended
>> to
>> > make last night to get the intended replacement of test run results
>> meet or
>> > exceed current requirements.
>> >
>> > I'd like to switch over to that by default.  I'm depending on the
>> test
>> > event system to be able to handle test method reruns in test results
>> > accounting.
>> >
>> > The primary thing missing before was that timeouts were not routed
>> > through the test events system, nor were exception process exits
>> (i.e. test
>> > inferiors exiting with a signal on POSIX systems).  Those were
>> added last
>> > night so that test events are generated for those, and the
>> > BasicResultsFormatter presents that information properly.
>> >
>> > I will switch it over to being the default output in a bit here.
>> > Please let me know if you have any concerns once I flip it on by
>> default.
>> >
>> > Thanks!
>> > --
>> > -Todd
>> > ___
>> > lldb-dev mailing list
>> > lldb-dev@lists.llvm.org
>> > http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev
>> >>>
>> >>>
>> >>>
>> >>>
>> >>> --
>> >>> -Todd
>> >>
>> >>
>> >>
>> >>
>> >> --
>> >> -Todd
>> >
>> >
>> >
>> >
>> > --
>> > -Todd
>>
>
>
>
> --
> -Todd
>



-- 
-Todd
___
lldb-dev mailing list
lldb-dev@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev

Re: [lldb-dev] BasicResultsFormatter - new test results summary

2015-12-09 Thread Todd Fiala via lldb-dev 
I forced a build on the ubuntu 14.04 cmake builder.  The build _after_ 9292
will contain the two changes (and we will expect failures on it).

On Wed, Dec 9, 2015 at 10:05 AM, Todd Fiala  wrote:

> These went in as:
>
> r255130 - turn it on by default
> r255131 - create known issues.  This one is to be reverted if all 3 types
> show up properly.
>
> On Wed, Dec 9, 2015 at 9:41 AM, Todd Fiala  wrote:
>
>> It is a small change.
>>
>> I almost have all the trial tests ready, so I'll just commit both changes
>> at the same time (the flip on, and the trial balloon issues).
>>
>> If all goes well and the three types of issue show up, then the last of
>> the two will get reverted (the one with the failures).
>>
>> If none (or only some) of the issues show up, they'll both get reverted.
>>
>> -Todd
>>
>> On Wed, Dec 9, 2015 at 9:39 AM, Pavel Labath  wrote:
>>
>>> If it's not too much work, I think the extra bit of noise will not be
>>> a problem. But I don't think it is really necessary either.
>>>
>>> I assume the actual flip will be a small change that we can back out
>>> easily if we notice troubles... After a sufficient grace period we can
>>> remove the old formatter altogether and hopefully simplify the code
>>> somewhat.
>>>
>>> pl
>>>
>>> On 9 December 2015 at 17:08, Todd Fiala  wrote:
>>> > Here's what I can do.
>>> >
>>> > Put in the change (setting the default to use the new format).
>>> >
>>> > Separately, put in a trial balloon commit with one failing test, one
>>> > exceptional exit test, and one timeout test, and watch the ubuntu 14.04
>>> > buildbot catch it and fail.  Then reverse this out.  That should show
>>> beyond
>>> > a reasonable doubt whether the buildbot catches new failures and
>>> errors.  (I
>>> > think this is a noisy way to accomplish this, but it certainly would
>>> > validate if its working).
>>> >
>>> > -Todd
>>> >
>>> > On Wed, Dec 9, 2015 at 8:06 AM, Todd Fiala 
>>> wrote:
>>> >>
>>> >> Specifically, the markers for issue details are:
>>> >>
>>> >> FAIL
>>> >> ERROR
>>> >> UNEXPECTED SUCCESS
>>> >> TIMEOUT
>>> >>
>>> >> (These are the fourth field in the array entries (lines 275 - 290) of
>>> >> packages/Python/lldbsuite/test/basic_results_formatter.py).
>>> >>
>>> >> -Todd
>>> >>
>>> >> On Wed, Dec 9, 2015 at 8:04 AM, Todd Fiala 
>>> wrote:
>>> >>>
>>> >>> That's a good point, Tamas.
>>> >>>
>>> >>> I use (so I claim) the same all upper-case markers for the test
>>> result
>>> >>> details.  Including, not using XPASS but rather UNEXPECTED SUCCESS
>>> for
>>> >>> unexpected successes.  (The former would trigger the lit script IIRC
>>> to
>>> >>> parse that as a failing-style result).
>>> >>>
>>> >>> The intent is this is a no-op on the test runner.
>>> >>>
>>> >>> On Wed, Dec 9, 2015 at 8:02 AM, Tamas Berghammer <
>>> tbergham...@google.com>
>>> >>> wrote:
>>> 
>>>  +Ying Chen
>>> 
>>>  Ying, what do we have to do on the build bot side to support a
>>> change in
>>>  the default test result summary formatter?
>>> 
>>>  On Wed, Dec 9, 2015 at 4:00 PM Todd Fiala via lldb-dev
>>>   wrote:
>>> >
>>> > Hi all,
>>> >
>>> > Per a previous thread on this, I've made all the changes I
>>> intended to
>>> > make last night to get the intended replacement of test run
>>> results meet or
>>> > exceed current requirements.
>>> >
>>> > I'd like to switch over to that by default.  I'm depending on the
>>> test
>>> > event system to be able to handle test method reruns in test
>>> results
>>> > accounting.
>>> >
>>> > The primary thing missing before was that timeouts were not routed
>>> > through the test events system, nor were exception process exits
>>> (i.e. test
>>> > inferiors exiting with a signal on POSIX systems).  Those were
>>> added last
>>> > night so that test events are generated for those, and the
>>> > BasicResultsFormatter presents that information properly.
>>> >
>>> > I will switch it over to being the default output in a bit here.
>>> > Please let me know if you have any concerns once I flip it on by
>>> default.
>>> >
>>> > Thanks!
>>> > --
>>> > -Todd
>>> > ___
>>> > lldb-dev mailing list
>>> > lldb-dev@lists.llvm.org
>>> > http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev
>>> >>>
>>> >>>
>>> >>>
>>> >>>
>>> >>> --
>>> >>> -Todd
>>> >>
>>> >>
>>> >>
>>> >>
>>> >> --
>>> >> -Todd
>>> >
>>> >
>>> >
>>> >
>>> > --
>>> > -Todd
>>>
>>
>>
>>
>> --
>> -Todd
>>
>
>
>
> --
> -Todd
>



-- 
-Todd
___
lldb-dev mailing list
lldb-dev@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev

Re: [lldb-dev] BasicResultsFormatter - new test results summary

2015-12-09 Thread Todd Fiala via lldb-dev 
I am going to stop the current build on that builder.  There was one change
in it, and it will be another 20 minutes before it completes.  I don't want
the repo in a known broken state that long.

On Wed, Dec 9, 2015 at 10:07 AM, Todd Fiala  wrote:

> I forced a build on the ubuntu 14.04 cmake builder.  The build _after_
> 9292 will contain the two changes (and we will expect failures on it).
>
> On Wed, Dec 9, 2015 at 10:05 AM, Todd Fiala  wrote:
>
>> These went in as:
>>
>> r255130 - turn it on by default
>> r255131 - create known issues.  This one is to be reverted if all 3 types
>> show up properly.
>>
>> On Wed, Dec 9, 2015 at 9:41 AM, Todd Fiala  wrote:
>>
>>> It is a small change.
>>>
>>> I almost have all the trial tests ready, so I'll just commit both
>>> changes at the same time (the flip on, and the trial balloon issues).
>>>
>>> If all goes well and the three types of issue show up, then the last of
>>> the two will get reverted (the one with the failures).
>>>
>>> If none (or only some) of the issues show up, they'll both get reverted.
>>>
>>> -Todd
>>>
>>> On Wed, Dec 9, 2015 at 9:39 AM, Pavel Labath  wrote:
>>>
 If it's not too much work, I think the extra bit of noise will not be
 a problem. But I don't think it is really necessary either.

 I assume the actual flip will be a small change that we can back out
 easily if we notice troubles... After a sufficient grace period we can
 remove the old formatter altogether and hopefully simplify the code
 somewhat.

 pl

 On 9 December 2015 at 17:08, Todd Fiala  wrote:
 > Here's what I can do.
 >
 > Put in the change (setting the default to use the new format).
 >
 > Separately, put in a trial balloon commit with one failing test, one
 > exceptional exit test, and one timeout test, and watch the ubuntu
 14.04
 > buildbot catch it and fail.  Then reverse this out.  That should show
 beyond
 > a reasonable doubt whether the buildbot catches new failures and
 errors.  (I
 > think this is a noisy way to accomplish this, but it certainly would
 > validate if its working).
 >
 > -Todd
 >
 > On Wed, Dec 9, 2015 at 8:06 AM, Todd Fiala 
 wrote:
 >>
 >> Specifically, the markers for issue details are:
 >>
 >> FAIL
 >> ERROR
 >> UNEXPECTED SUCCESS
 >> TIMEOUT
 >>
 >> (These are the fourth field in the array entries (lines 275 - 290) of
 >> packages/Python/lldbsuite/test/basic_results_formatter.py).
 >>
 >> -Todd
 >>
 >> On Wed, Dec 9, 2015 at 8:04 AM, Todd Fiala 
 wrote:
 >>>
 >>> That's a good point, Tamas.
 >>>
 >>> I use (so I claim) the same all upper-case markers for the test
 result
 >>> details.  Including, not using XPASS but rather UNEXPECTED SUCCESS
 for
 >>> unexpected successes.  (The former would trigger the lit script
 IIRC to
 >>> parse that as a failing-style result).
 >>>
 >>> The intent is this is a no-op on the test runner.
 >>>
 >>> On Wed, Dec 9, 2015 at 8:02 AM, Tamas Berghammer <
 tbergham...@google.com>
 >>> wrote:
 
  +Ying Chen
 
  Ying, what do we have to do on the build bot side to support a
 change in
  the default test result summary formatter?
 
  On Wed, Dec 9, 2015 at 4:00 PM Todd Fiala via lldb-dev
   wrote:
 >
 > Hi all,
 >
 > Per a previous thread on this, I've made all the changes I
 intended to
 > make last night to get the intended replacement of test run
 results meet or
 > exceed current requirements.
 >
 > I'd like to switch over to that by default.  I'm depending on the
 test
 > event system to be able to handle test method reruns in test
 results
 > accounting.
 >
 > The primary thing missing before was that timeouts were not routed
 > through the test events system, nor were exception process exits
 (i.e. test
 > inferiors exiting with a signal on POSIX systems).  Those were
 added last
 > night so that test events are generated for those, and the
 > BasicResultsFormatter presents that information properly.
 >
 > I will switch it over to being the default output in a bit here.
 > Please let me know if you have any concerns once I flip it on by
 default.
 >
 > Thanks!
 > --
 > -Todd
 > ___
 > lldb-dev mailing list
 > lldb-dev@lists.llvm.org
 > http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev
 >>>
 >>>
 >>>
 >>>
 >>> --
 >>> -Todd
 >>
 >>
 >>
 >>
 >> --
 >> -Todd
 >
 >
 >
 >
 > --
 > -Todd

>>>
>>>
>>>
>>> --
>>> -Todd
>>>
>>
>>
>>
>> --
>> -Todd
>>

Re: [lldb-dev] BasicResultsFormatter - new test results summary

2015-12-09 Thread Todd Fiala via lldb-dev 
The reports look good at the test level:

http://lab.llvm.org:8011/builders/lldb-x86_64-ubuntu-14.04-cmake/builds/9294

I'd say the buildbot reflection script missed the ERROR, so that is
something maybe Ying can look at (the summary line in the build run), but
that is unrelated AFAICT.

I'm going to move aside the failures.

On Wed, Dec 9, 2015 at 10:13 AM, Todd Fiala  wrote:

> I am going to stop the current build on that builder.  There was one
> change in it, and it will be another 20 minutes before it completes.  I
> don't want the repo in a known broken state that long.
>
> On Wed, Dec 9, 2015 at 10:07 AM, Todd Fiala  wrote:
>
>> I forced a build on the ubuntu 14.04 cmake builder.  The build _after_
>> 9292 will contain the two changes (and we will expect failures on it).
>>
>> On Wed, Dec 9, 2015 at 10:05 AM, Todd Fiala  wrote:
>>
>>> These went in as:
>>>
>>> r255130 - turn it on by default
>>> r255131 - create known issues.  This one is to be reverted if all 3
>>> types show up properly.
>>>
>>> On Wed, Dec 9, 2015 at 9:41 AM, Todd Fiala  wrote:
>>>
 It is a small change.

 I almost have all the trial tests ready, so I'll just commit both
 changes at the same time (the flip on, and the trial balloon issues).

 If all goes well and the three types of issue show up, then the last of
 the two will get reverted (the one with the failures).

 If none (or only some) of the issues show up, they'll both get reverted.

 -Todd

 On Wed, Dec 9, 2015 at 9:39 AM, Pavel Labath  wrote:

> If it's not too much work, I think the extra bit of noise will not be
> a problem. But I don't think it is really necessary either.
>
> I assume the actual flip will be a small change that we can back out
> easily if we notice troubles... After a sufficient grace period we can
> remove the old formatter altogether and hopefully simplify the code
> somewhat.
>
> pl
>
> On 9 December 2015 at 17:08, Todd Fiala  wrote:
> > Here's what I can do.
> >
> > Put in the change (setting the default to use the new format).
> >
> > Separately, put in a trial balloon commit with one failing test, one
> > exceptional exit test, and one timeout test, and watch the ubuntu
> 14.04
> > buildbot catch it and fail.  Then reverse this out.  That should
> show beyond
> > a reasonable doubt whether the buildbot catches new failures and
> errors.  (I
> > think this is a noisy way to accomplish this, but it certainly would
> > validate if its working).
> >
> > -Todd
> >
> > On Wed, Dec 9, 2015 at 8:06 AM, Todd Fiala 
> wrote:
> >>
> >> Specifically, the markers for issue details are:
> >>
> >> FAIL
> >> ERROR
> >> UNEXPECTED SUCCESS
> >> TIMEOUT
> >>
> >> (These are the fourth field in the array entries (lines 275 - 290)
> of
> >> packages/Python/lldbsuite/test/basic_results_formatter.py).
> >>
> >> -Todd
> >>
> >> On Wed, Dec 9, 2015 at 8:04 AM, Todd Fiala 
> wrote:
> >>>
> >>> That's a good point, Tamas.
> >>>
> >>> I use (so I claim) the same all upper-case markers for the test
> result
> >>> details.  Including, not using XPASS but rather UNEXPECTED SUCCESS
> for
> >>> unexpected successes.  (The former would trigger the lit script
> IIRC to
> >>> parse that as a failing-style result).
> >>>
> >>> The intent is this is a no-op on the test runner.
> >>>
> >>> On Wed, Dec 9, 2015 at 8:02 AM, Tamas Berghammer <
> tbergham...@google.com>
> >>> wrote:
> 
>  +Ying Chen
> 
>  Ying, what do we have to do on the build bot side to support a
> change in
>  the default test result summary formatter?
> 
>  On Wed, Dec 9, 2015 at 4:00 PM Todd Fiala via lldb-dev
>   wrote:
> >
> > Hi all,
> >
> > Per a previous thread on this, I've made all the changes I
> intended to
> > make last night to get the intended replacement of test run
> results meet or
> > exceed current requirements.
> >
> > I'd like to switch over to that by default.  I'm depending on
> the test
> > event system to be able to handle test method reruns in test
> results
> > accounting.
> >
> > The primary thing missing before was that timeouts were not
> routed
> > through the test events system, nor were exception process exits
> (i.e. test
> > inferiors exiting with a signal on POSIX systems).  Those were
> added last
> > night so that test events are generated for those, and the
> > BasicResultsFormatter presents that information properly.
> >
> > I will switch it over to being the default output in a bit here.
> > Please let me know if you have any concerns on

Re: [lldb-dev] BasicResultsFormatter - new test results summary

2015-12-09 Thread Todd Fiala via lldb-dev 
Verification tests parked (i.e. disabled) here:
r255134

I decided to leave them in the repo so it is faster/easier to do this in
the future.

-Todd

On Wed, Dec 9, 2015 at 10:26 AM, Todd Fiala  wrote:

> The reports look good at the test level:
>
>
> http://lab.llvm.org:8011/builders/lldb-x86_64-ubuntu-14.04-cmake/builds/9294
>
> I'd say the buildbot reflection script missed the ERROR, so that is
> something maybe Ying can look at (the summary line in the build run), but
> that is unrelated AFAICT.
>
> I'm going to move aside the failures.
>
> On Wed, Dec 9, 2015 at 10:13 AM, Todd Fiala  wrote:
>
>> I am going to stop the current build on that builder.  There was one
>> change in it, and it will be another 20 minutes before it completes.  I
>> don't want the repo in a known broken state that long.
>>
>> On Wed, Dec 9, 2015 at 10:07 AM, Todd Fiala  wrote:
>>
>>> I forced a build on the ubuntu 14.04 cmake builder.  The build _after_
>>> 9292 will contain the two changes (and we will expect failures on it).
>>>
>>> On Wed, Dec 9, 2015 at 10:05 AM, Todd Fiala 
>>> wrote:
>>>
 These went in as:

 r255130 - turn it on by default
 r255131 - create known issues.  This one is to be reverted if all 3
 types show up properly.

 On Wed, Dec 9, 2015 at 9:41 AM, Todd Fiala 
 wrote:

> It is a small change.
>
> I almost have all the trial tests ready, so I'll just commit both
> changes at the same time (the flip on, and the trial balloon issues).
>
> If all goes well and the three types of issue show up, then the last
> of the two will get reverted (the one with the failures).
>
> If none (or only some) of the issues show up, they'll both get
> reverted.
>
> -Todd
>
> On Wed, Dec 9, 2015 at 9:39 AM, Pavel Labath 
> wrote:
>
>> If it's not too much work, I think the extra bit of noise will not be
>> a problem. But I don't think it is really necessary either.
>>
>> I assume the actual flip will be a small change that we can back out
>> easily if we notice troubles... After a sufficient grace period we can
>> remove the old formatter altogether and hopefully simplify the code
>> somewhat.
>>
>> pl
>>
>> On 9 December 2015 at 17:08, Todd Fiala  wrote:
>> > Here's what I can do.
>> >
>> > Put in the change (setting the default to use the new format).
>> >
>> > Separately, put in a trial balloon commit with one failing test, one
>> > exceptional exit test, and one timeout test, and watch the ubuntu
>> 14.04
>> > buildbot catch it and fail.  Then reverse this out.  That should
>> show beyond
>> > a reasonable doubt whether the buildbot catches new failures and
>> errors.  (I
>> > think this is a noisy way to accomplish this, but it certainly would
>> > validate if its working).
>> >
>> > -Todd
>> >
>> > On Wed, Dec 9, 2015 at 8:06 AM, Todd Fiala 
>> wrote:
>> >>
>> >> Specifically, the markers for issue details are:
>> >>
>> >> FAIL
>> >> ERROR
>> >> UNEXPECTED SUCCESS
>> >> TIMEOUT
>> >>
>> >> (These are the fourth field in the array entries (lines 275 - 290)
>> of
>> >> packages/Python/lldbsuite/test/basic_results_formatter.py).
>> >>
>> >> -Todd
>> >>
>> >> On Wed, Dec 9, 2015 at 8:04 AM, Todd Fiala 
>> wrote:
>> >>>
>> >>> That's a good point, Tamas.
>> >>>
>> >>> I use (so I claim) the same all upper-case markers for the test
>> result
>> >>> details.  Including, not using XPASS but rather UNEXPECTED
>> SUCCESS for
>> >>> unexpected successes.  (The former would trigger the lit script
>> IIRC to
>> >>> parse that as a failing-style result).
>> >>>
>> >>> The intent is this is a no-op on the test runner.
>> >>>
>> >>> On Wed, Dec 9, 2015 at 8:02 AM, Tamas Berghammer <
>> tbergham...@google.com>
>> >>> wrote:
>> 
>>  +Ying Chen
>> 
>>  Ying, what do we have to do on the build bot side to support a
>> change in
>>  the default test result summary formatter?
>> 
>>  On Wed, Dec 9, 2015 at 4:00 PM Todd Fiala via lldb-dev
>>   wrote:
>> >
>> > Hi all,
>> >
>> > Per a previous thread on this, I've made all the changes I
>> intended to
>> > make last night to get the intended replacement of test run
>> results meet or
>> > exceed current requirements.
>> >
>> > I'd like to switch over to that by default.  I'm depending on
>> the test
>> > event system to be able to handle test method reruns in test
>> results
>> > accounting.
>> >
>> > The primary thing missing before was that timeouts were not
>> routed
>> > through the test events system, nor were exception process
>> exits (i.e. test
>> >

Re: [lldb-dev] BasicResultsFormatter - new test results summary

2015-12-09 Thread Tamas Berghammer via lldb-dev 
Thank you for making the experiment. It looks reasonable. For the ERROR the
buildbot detected it and it will fail the build but it isn't listed in the
list of failing tests what should be fixed. After this experiment I think
it is fine to change the default output formatter from our side.

Tamas

On Wed, Dec 9, 2015 at 6:26 PM Todd Fiala  wrote:

> The reports look good at the test level:
>
>
> http://lab.llvm.org:8011/builders/lldb-x86_64-ubuntu-14.04-cmake/builds/9294
>
> I'd say the buildbot reflection script missed the ERROR, so that is
> something maybe Ying can look at (the summary line in the build run), but
> that is unrelated AFAICT.
>
> I'm going to move aside the failures.
>
> On Wed, Dec 9, 2015 at 10:13 AM, Todd Fiala  wrote:
>
>> I am going to stop the current build on that builder.  There was one
>> change in it, and it will be another 20 minutes before it completes.  I
>> don't want the repo in a known broken state that long.
>>
>> On Wed, Dec 9, 2015 at 10:07 AM, Todd Fiala  wrote:
>>
>>> I forced a build on the ubuntu 14.04 cmake builder.  The build _after_
>>> 9292 will contain the two changes (and we will expect failures on it).
>>>
>>> On Wed, Dec 9, 2015 at 10:05 AM, Todd Fiala 
>>> wrote:
>>>
 These went in as:

 r255130 - turn it on by default
 r255131 - create known issues.  This one is to be reverted if all 3
 types show up properly.

 On Wed, Dec 9, 2015 at 9:41 AM, Todd Fiala 
 wrote:

> It is a small change.
>
> I almost have all the trial tests ready, so I'll just commit both
> changes at the same time (the flip on, and the trial balloon issues).
>
> If all goes well and the three types of issue show up, then the last
> of the two will get reverted (the one with the failures).
>
> If none (or only some) of the issues show up, they'll both get
> reverted.
>
> -Todd
>
> On Wed, Dec 9, 2015 at 9:39 AM, Pavel Labath 
> wrote:
>
>> If it's not too much work, I think the extra bit of noise will not be
>> a problem. But I don't think it is really necessary either.
>>
>> I assume the actual flip will be a small change that we can back out
>> easily if we notice troubles... After a sufficient grace period we can
>> remove the old formatter altogether and hopefully simplify the code
>> somewhat.
>>
>> pl
>>
>> On 9 December 2015 at 17:08, Todd Fiala  wrote:
>> > Here's what I can do.
>> >
>> > Put in the change (setting the default to use the new format).
>> >
>> > Separately, put in a trial balloon commit with one failing test, one
>> > exceptional exit test, and one timeout test, and watch the ubuntu
>> 14.04
>> > buildbot catch it and fail.  Then reverse this out.  That should
>> show beyond
>> > a reasonable doubt whether the buildbot catches new failures and
>> errors.  (I
>> > think this is a noisy way to accomplish this, but it certainly would
>> > validate if its working).
>> >
>> > -Todd
>> >
>> > On Wed, Dec 9, 2015 at 8:06 AM, Todd Fiala 
>> wrote:
>> >>
>> >> Specifically, the markers for issue details are:
>> >>
>> >> FAIL
>> >> ERROR
>> >> UNEXPECTED SUCCESS
>> >> TIMEOUT
>> >>
>> >> (These are the fourth field in the array entries (lines 275 - 290)
>> of
>> >> packages/Python/lldbsuite/test/basic_results_formatter.py).
>> >>
>> >> -Todd
>> >>
>> >> On Wed, Dec 9, 2015 at 8:04 AM, Todd Fiala 
>> wrote:
>> >>>
>> >>> That's a good point, Tamas.
>> >>>
>> >>> I use (so I claim) the same all upper-case markers for the test
>> result
>> >>> details.  Including, not using XPASS but rather UNEXPECTED
>> SUCCESS for
>> >>> unexpected successes.  (The former would trigger the lit script
>> IIRC to
>> >>> parse that as a failing-style result).
>> >>>
>> >>> The intent is this is a no-op on the test runner.
>> >>>
>> >>> On Wed, Dec 9, 2015 at 8:02 AM, Tamas Berghammer <
>> tbergham...@google.com>
>> >>> wrote:
>> 
>>  +Ying Chen
>> 
>>  Ying, what do we have to do on the build bot side to support a
>> change in
>>  the default test result summary formatter?
>> 
>>  On Wed, Dec 9, 2015 at 4:00 PM Todd Fiala via lldb-dev
>>   wrote:
>> >
>> > Hi all,
>> >
>> > Per a previous thread on this, I've made all the changes I
>> intended to
>> > make last night to get the intended replacement of test run
>> results meet or
>> > exceed current requirements.
>> >
>> > I'd like to switch over to that by default.  I'm depending on
>> the test
>> > event system to be able to handle test method reruns in test
>> results
>> > accounting.
>> >
>> > The primary thing missing before was

Re: [lldb-dev] BasicResultsFormatter - new test results summary

2015-12-09 Thread Todd Fiala via lldb-dev 
Great, thanks Tamas!

I left the default turned on, and just essentially removed the issues by
parking them as .py.parked files.  That way we can flip them on in the
future if we want to verify a testbot's detection of these.

I will be going back to the xUnit Results formatter and making sure it maps
timeouts and exceptional errors to the xUnit error type with details.

On Wed, Dec 9, 2015 at 10:30 AM, Tamas Berghammer 
wrote:

> Thank you for making the experiment. It looks reasonable. For the ERROR
> the buildbot detected it and it will fail the build but it isn't listed in
> the list of failing tests what should be fixed. After this experiment I
> think it is fine to change the default output formatter from our side.
>
> Tamas
>
> On Wed, Dec 9, 2015 at 6:26 PM Todd Fiala  wrote:
>
>> The reports look good at the test level:
>>
>>
>> http://lab.llvm.org:8011/builders/lldb-x86_64-ubuntu-14.04-cmake/builds/9294
>>
>> I'd say the buildbot reflection script missed the ERROR, so that is
>> something maybe Ying can look at (the summary line in the build run), but
>> that is unrelated AFAICT.
>>
>> I'm going to move aside the failures.
>>
>> On Wed, Dec 9, 2015 at 10:13 AM, Todd Fiala  wrote:
>>
>>> I am going to stop the current build on that builder.  There was one
>>> change in it, and it will be another 20 minutes before it completes.  I
>>> don't want the repo in a known broken state that long.
>>>
>>> On Wed, Dec 9, 2015 at 10:07 AM, Todd Fiala 
>>> wrote:
>>>
 I forced a build on the ubuntu 14.04 cmake builder.  The build _after_
 9292 will contain the two changes (and we will expect failures on it).

 On Wed, Dec 9, 2015 at 10:05 AM, Todd Fiala 
 wrote:

> These went in as:
>
> r255130 - turn it on by default
> r255131 - create known issues.  This one is to be reverted if all 3
> types show up properly.
>
> On Wed, Dec 9, 2015 at 9:41 AM, Todd Fiala 
> wrote:
>
>> It is a small change.
>>
>> I almost have all the trial tests ready, so I'll just commit both
>> changes at the same time (the flip on, and the trial balloon issues).
>>
>> If all goes well and the three types of issue show up, then the last
>> of the two will get reverted (the one with the failures).
>>
>> If none (or only some) of the issues show up, they'll both get
>> reverted.
>>
>> -Todd
>>
>> On Wed, Dec 9, 2015 at 9:39 AM, Pavel Labath 
>> wrote:
>>
>>> If it's not too much work, I think the extra bit of noise will not be
>>> a problem. But I don't think it is really necessary either.
>>>
>>> I assume the actual flip will be a small change that we can back out
>>> easily if we notice troubles... After a sufficient grace period we
>>> can
>>> remove the old formatter altogether and hopefully simplify the code
>>> somewhat.
>>>
>>> pl
>>>
>>> On 9 December 2015 at 17:08, Todd Fiala 
>>> wrote:
>>> > Here's what I can do.
>>> >
>>> > Put in the change (setting the default to use the new format).
>>> >
>>> > Separately, put in a trial balloon commit with one failing test,
>>> one
>>> > exceptional exit test, and one timeout test, and watch the ubuntu
>>> 14.04
>>> > buildbot catch it and fail.  Then reverse this out.  That should
>>> show beyond
>>> > a reasonable doubt whether the buildbot catches new failures and
>>> errors.  (I
>>> > think this is a noisy way to accomplish this, but it certainly
>>> would
>>> > validate if its working).
>>> >
>>> > -Todd
>>> >
>>> > On Wed, Dec 9, 2015 at 8:06 AM, Todd Fiala 
>>> wrote:
>>> >>
>>> >> Specifically, the markers for issue details are:
>>> >>
>>> >> FAIL
>>> >> ERROR
>>> >> UNEXPECTED SUCCESS
>>> >> TIMEOUT
>>> >>
>>> >> (These are the fourth field in the array entries (lines 275 -
>>> 290) of
>>> >> packages/Python/lldbsuite/test/basic_results_formatter.py).
>>> >>
>>> >> -Todd
>>> >>
>>> >> On Wed, Dec 9, 2015 at 8:04 AM, Todd Fiala 
>>> wrote:
>>> >>>
>>> >>> That's a good point, Tamas.
>>> >>>
>>> >>> I use (so I claim) the same all upper-case markers for the test
>>> result
>>> >>> details.  Including, not using XPASS but rather UNEXPECTED
>>> SUCCESS for
>>> >>> unexpected successes.  (The former would trigger the lit script
>>> IIRC to
>>> >>> parse that as a failing-style result).
>>> >>>
>>> >>> The intent is this is a no-op on the test runner.
>>> >>>
>>> >>> On Wed, Dec 9, 2015 at 8:02 AM, Tamas Berghammer <
>>> tbergham...@google.com>
>>> >>> wrote:
>>> 
>>>  +Ying Chen
>>> 
>>>  Ying, what do we have to do on the build bot side to support a
>>> change in
>>>  the default test result summary formatter?
>>> 
>>>  On Wed, Dec 9, 2015 at 4:00 P

Re: [lldb-dev] BasicResultsFormatter - new test results summary

2015-12-09 Thread Ying Chen via lldb-dev 
I submitted this patch to include "ERROR" lines in buildbot step results.
http://reviews.llvm.org/rL255145

Error results will be displayed in step result like this after the patch,
"ERROR: 9 (SIGKILL) test_buildbot_catches_exceptional_exit_dwarf"

Thanks,
Ying

On Wed, Dec 9, 2015 at 10:45 AM, Todd Fiala  wrote:

> Great, thanks Tamas!
>
> I left the default turned on, and just essentially removed the issues by
> parking them as .py.parked files.  That way we can flip them on in the
> future if we want to verify a testbot's detection of these.
>
> I will be going back to the xUnit Results formatter and making sure it
> maps timeouts and exceptional errors to the xUnit error type with details.
>
> On Wed, Dec 9, 2015 at 10:30 AM, Tamas Berghammer 
> wrote:
>
>> Thank you for making the experiment. It looks reasonable. For the ERROR
>> the buildbot detected it and it will fail the build but it isn't listed in
>> the list of failing tests what should be fixed. After this experiment I
>> think it is fine to change the default output formatter from our side.
>>
>> Tamas
>>
>> On Wed, Dec 9, 2015 at 6:26 PM Todd Fiala  wrote:
>>
>>> The reports look good at the test level:
>>>
>>>
>>> http://lab.llvm.org:8011/builders/lldb-x86_64-ubuntu-14.04-cmake/builds/9294
>>>
>>> I'd say the buildbot reflection script missed the ERROR, so that is
>>> something maybe Ying can look at (the summary line in the build run), but
>>> that is unrelated AFAICT.
>>>
>>> I'm going to move aside the failures.
>>>
>>> On Wed, Dec 9, 2015 at 10:13 AM, Todd Fiala 
>>> wrote:
>>>
 I am going to stop the current build on that builder.  There was one
 change in it, and it will be another 20 minutes before it completes.  I
 don't want the repo in a known broken state that long.

 On Wed, Dec 9, 2015 at 10:07 AM, Todd Fiala 
 wrote:

> I forced a build on the ubuntu 14.04 cmake builder.  The build _after_
> 9292 will contain the two changes (and we will expect failures on it).
>
> On Wed, Dec 9, 2015 at 10:05 AM, Todd Fiala 
> wrote:
>
>> These went in as:
>>
>> r255130 - turn it on by default
>> r255131 - create known issues.  This one is to be reverted if all 3
>> types show up properly.
>>
>> On Wed, Dec 9, 2015 at 9:41 AM, Todd Fiala 
>> wrote:
>>
>>> It is a small change.
>>>
>>> I almost have all the trial tests ready, so I'll just commit both
>>> changes at the same time (the flip on, and the trial balloon issues).
>>>
>>> If all goes well and the three types of issue show up, then the last
>>> of the two will get reverted (the one with the failures).
>>>
>>> If none (or only some) of the issues show up, they'll both get
>>> reverted.
>>>
>>> -Todd
>>>
>>> On Wed, Dec 9, 2015 at 9:39 AM, Pavel Labath 
>>> wrote:
>>>
 If it's not too much work, I think the extra bit of noise will not
 be
 a problem. But I don't think it is really necessary either.

 I assume the actual flip will be a small change that we can back out
 easily if we notice troubles... After a sufficient grace period we
 can
 remove the old formatter altogether and hopefully simplify the code
 somewhat.

 pl

 On 9 December 2015 at 17:08, Todd Fiala 
 wrote:
 > Here's what I can do.
 >
 > Put in the change (setting the default to use the new format).
 >
 > Separately, put in a trial balloon commit with one failing test,
 one
 > exceptional exit test, and one timeout test, and watch the ubuntu
 14.04
 > buildbot catch it and fail.  Then reverse this out.  That should
 show beyond
 > a reasonable doubt whether the buildbot catches new failures and
 errors.  (I
 > think this is a noisy way to accomplish this, but it certainly
 would
 > validate if its working).
 >
 > -Todd
 >
 > On Wed, Dec 9, 2015 at 8:06 AM, Todd Fiala 
 wrote:
 >>
 >> Specifically, the markers for issue details are:
 >>
 >> FAIL
 >> ERROR
 >> UNEXPECTED SUCCESS
 >> TIMEOUT
 >>
 >> (These are the fourth field in the array entries (lines 275 -
 290) of
 >> packages/Python/lldbsuite/test/basic_results_formatter.py).
 >>
 >> -Todd
 >>
 >> On Wed, Dec 9, 2015 at 8:04 AM, Todd Fiala 
 wrote:
 >>>
 >>> That's a good point, Tamas.
 >>>
 >>> I use (so I claim) the same all upper-case markers for the test
 result
 >>> details.  Including, not using XPASS but rather UNEXPECTED
 SUCCESS for
 >>> unexpected successes.  (The former would trigger the lit script
 IIRC to
 >>> parse that as a failing-st

[lldb-dev] Benchmark tests

2015-12-09 Thread Zachary Turner via lldb-dev 
Is anyone using the benchmark tests?  None of the command line options

related to the benchmark tests were claimed as being used by anyone.  Which
makes me wonder if the tests are even being used by anyone.

What I really want to know is: Is it really ok to delete the -x and -y
command line options?  And what is the status of these tests?  Does anyone
use them?
___
lldb-dev mailing list
lldb-dev@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev

Re: [lldb-dev] LLDB/NetBSD test suite results as of 8 Dec 2015

2015-12-09 Thread Kamil Rytarowski via lldb-dev 
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA256

On 08.12.2015 17:21, Ed Maste wrote:
> On 7 December 2015 at 19:28, Kamil Rytarowski via lldb-dev 
>  wrote:
>> 
>> I ran the LLDB test suite of the LLDB-current version on
>> NetBSD/amd64 (v. 7.99.21).
>> 
>> The results are as follows:
>> 
>> 415 out of 415 test suites processed - TestRecursiveTypes.py
>> 
>> Ran 415 test suites (266 failed) (64.096386%) Ran 222 test cases
>> (145 failed) (65.315315%) Failing Tests (266)
>> 
>> The major missing piece is native process tracking support (with
>> the ptrace(2) interface). Once added, LLDB should start to be
>> usable on this platform.
> 
> Great progress, and thanks for the update! Are you planning to
> base the NetBSD native support on NativeProcessLinux? I need to
> migrate the FreeBSD support to work that way still.
> 


In my initial scratch (before upstreaming of the current NetBSD
support) I cloned the current FreeBSD approach. It was halfbaked,
building and not working.

I will see the NativeProcessLinux way.
-BEGIN PGP SIGNATURE-
Version: GnuPG v2

iQIcBAEBCAAGBQJWaJMmAAoJEEuzCOmwLnZs9PEP/ieCtuUqDE+G9hDm3sk4u4I+
83aMdtOzWFO4I26MLRjqKqytn5iEgltWuLT0VW47IU/hynkEOET8m3cwTLkIakGC
pBohk16yl6dQo97rU3/dVlLuKpkuN1wk19/IRxbqVSV8m86IFAPI1T8xQCWXa9iT
6dTRli4pW0OemlBxvWXM0tRm7d2GISgkAlGbHkZ3grfQYufJ8ulZyGDI+5Kss0oV
Ku0ovL9U+vtyLdHlcpimuTaMvMZThnv3eJdImGlwXJrvS4yZifP2PiyM3qGXxiqb
0ZzBT9Gb8fqVoVOD9Al95NycIHu/jptudt+3YJoeMN903eQz6wT7izZm055MpEfu
HPKvrbpeQyOZ2j7Z8xbqlNl0vm0Q2c41/LaoneMOcf6Uu7XZ/5yaKT+j2oGT6O4m
vRZ3pfyZqlTMtlUxs6+HfFT08AfW1cRqmpNInw/fkQVtmmIkytmOELF/fpsks8XC
cXF3XQtFRGmbDyvGZdeKwXO55qCyL9w3BbhMTrIopm+sPaayB4OcGnV00++3mwJn
Y3iUXTb5Xrqm1ORFUaXmaoDdiSSSHhAZRmS+VgqJ4/+AWWiWcGbZd2oKIg3b+FmO
yu1DIeFsmWnaFNkp+cjnjr049ofO8kUbr6dCIJ6EyiydRWwXLufiKZhpYmpT48i1
kenLjLCuVTYFjkzM8HXA
=0zUx
-END PGP SIGNATURE-
___
lldb-dev mailing list
lldb-dev@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev

Re: [lldb-dev] Benchmark tests

2015-12-09 Thread Todd Fiala via lldb-dev 
Hey Jason,

Are you the benchmark user?

-Todd

On Wed, Dec 9, 2015 at 12:32 PM, Zachary Turner via lldb-dev <
lldb-dev@lists.llvm.org> wrote:

> Is anyone using the benchmark tests?  None of the command line options
> 
> related to the benchmark tests were claimed as being used by anyone.  Which
> makes me wonder if the tests are even being used by anyone.
>
> What I really want to know is: Is it really ok to delete the -x and -y
> command line options?  And what is the status of these tests?  Does anyone
> use them?
>
> ___
> lldb-dev mailing list
> lldb-dev@lists.llvm.org
> http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev
>
>


-- 
-Todd
___
lldb-dev mailing list
lldb-dev@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev

[lldb-dev] swig generation and "six" module

2015-12-09 Thread Ted Woodward via lldb-dev 
r252764 changes finishSwigPythonLLDB.py to symlink the "six" module in
site-packages. six.py is a symlink to
/tools/lldb/third_party/Python/module/six/six.py. This assumes I have
access to this build's sources when I run lldb, which I don't - our
workstations don't have access to the buildbots' filesystem, and even if
they did, the sources may change between build and run.

 

Shouldn't six.py just be copied? It's only 30k.

 

Zach, how do I do a build using static swig bindings? I don't think we're
going to change the bindings internally, so using the standard ones should
be fine.

 

--

Qualcomm Innovation Center, Inc.

The Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum, a
Linux Foundation Collaborative Project

 

___
lldb-dev mailing list
lldb-dev@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev

Re: [lldb-dev] swig generation and "six" module

2015-12-09 Thread Zachary Turner via lldb-dev 
What about requiring that the user has done "pip install six" on their
machine?

On Wed, Dec 9, 2015 at 1:40 PM Ted Woodward via lldb-dev <
lldb-dev@lists.llvm.org> wrote:

> r252764 changes finishSwigPythonLLDB.py to symlink the “six” module in
> site-packages. six.py is a symlink to
> /tools/lldb/third_party/Python/module/six/six.py. This assumes I have
> access to this build’s sources when I run lldb, which I don’t – our
> workstations don’t have access to the buildbots’ filesystem, and even if
> they did, the sources may change between build and run.
>
>
>
> Shouldn’t six.py just be copied? It’s only 30k.
>
>
>
> Zach, how do I do a build using static swig bindings? I don’t think we’re
> going to change the bindings internally, so using the standard ones should
> be fine.
>
>
>
> --
>
> Qualcomm Innovation Center, Inc.
>
> The Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum, a
> Linux Foundation Collaborative Project
>
>
> ___
> lldb-dev mailing list
> lldb-dev@lists.llvm.org
> http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev
>
___
lldb-dev mailing list
lldb-dev@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev

Re: [lldb-dev] Benchmark tests

2015-12-09 Thread Enrico Granata via lldb-dev 
I have actually added a few benchmark tests recently. We admittedly are not 
that good with running those tests ever (because they're not run by default 
most likely - and I do wonder if some of them would take a long time to run.. I 
don't think I have ever run the full set, just my own as I increment on 
performance work).

Maybe we could try flipping the default to be "run the benchmarks", see if test 
suite run times explode and take it from there in terms of feasibility as well 
as whether they all still make sense.

The other problem with the tests as they stand is that they mark themselves as 
PASS or FAIL purely on the basis of whether they encounter command or API 
errors, and do nothing to track performance regressions. That is admittedly a 
harder problem to tackle given heterogeneous hardware and workload - but maybe 
we could have them fail if the timings go wildly crazy over some threshold?

Sent from my iPhone

> On Dec 9, 2015, at 1:22 PM, Todd Fiala via lldb-dev  
> wrote:
> 
> Hey Jason,
> 
> Are you the benchmark user?
> 
> -Todd
> 
>> On Wed, Dec 9, 2015 at 12:32 PM, Zachary Turner via lldb-dev 
>>  wrote:
>> Is anyone using the benchmark tests?  None of the command line options 
>> related to the benchmark tests were claimed as being used by anyone.  Which 
>> makes me wonder if the tests are even being used by anyone.  
>> 
>> What I really want to know is: Is it really ok to delete the -x and -y 
>> command line options?  And what is the status of these tests?  Does anyone 
>> use them?
>> 
>> ___
>> lldb-dev mailing list
>> lldb-dev@lists.llvm.org
>> http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev
> 
> 
> 
> -- 
> -Todd
> ___
> lldb-dev mailing list
> lldb-dev@lists.llvm.org
> http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev
___
lldb-dev mailing list
lldb-dev@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev

Re: [lldb-dev] Benchmark tests

2015-12-09 Thread Zachary Turner via lldb-dev 
When you do run the benchmark tests, what command line options do you use?
At the moment I'm mostly just trying to remove dead options from the test
suite.  I removed one already that allowed you to specify the benchmark
executable, but then when I started looking at the rest and seeing how
tightly integrated they are with the benchamrk tests in general, I started
to wonder.

The three benchmark related command line options are:

1. An option to specify the benchmark executable (defaults to lldb.exe)
2. An option to specify the breakpoint spec (defaults to -n main)
3. An option to specify the breakpoint iteration count (defaults to 30 I
think)
4. An option to specify that you only want to run benchmark tests and no
other tests.

I deleted #4 because you can use the category system for that.  I deleted
#1 because nobody said they needed it on the spreadsheet.  Nobody said they
needed #2 or #3 either, but I just want to double check that deleting them
is fine.

On Wed, Dec 9, 2015 at 2:01 PM Enrico Granata  wrote:

> I have actually added a few benchmark tests recently. We admittedly are
> not that good with running those tests ever (because they're not run by
> default most likely - and I do wonder if some of them would take a long
> time to run.. I don't think I have ever run the full set, just my own as I
> increment on performance work).
>
> Maybe we could try flipping the default to be "run the benchmarks", see if
> test suite run times explode and take it from there in terms of feasibility
> as well as whether they all still make sense.
>
> The other problem with the tests as they stand is that they mark
> themselves as PASS or FAIL purely on the basis of whether they encounter
> command or API errors, and do nothing to track performance regressions.
> That is admittedly a harder problem to tackle given heterogeneous hardware
> and workload - but maybe we could have them fail if the timings go wildly
> crazy over some threshold?
>
> Sent from my iPhone
>
> On Dec 9, 2015, at 1:22 PM, Todd Fiala via lldb-dev <
> lldb-dev@lists.llvm.org> wrote:
>
> Hey Jason,
>
> Are you the benchmark user?
>
> -Todd
>
> On Wed, Dec 9, 2015 at 12:32 PM, Zachary Turner via lldb-dev <
> lldb-dev@lists.llvm.org> wrote:
>
>> Is anyone using the benchmark tests?  None of the command line options
>> 
>> related to the benchmark tests were claimed as being used by anyone.  Which
>> makes me wonder if the tests are even being used by anyone.
>>
>> What I really want to know is: Is it really ok to delete the -x and -y
>> command line options?  And what is the status of these tests?  Does anyone
>> use them?
>>
>> ___
>> lldb-dev mailing list
>> lldb-dev@lists.llvm.org
>> http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev
>>
>>
>
>
> --
> -Todd
>
> ___
> lldb-dev mailing list
> lldb-dev@lists.llvm.org
> http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev
>
>
___
lldb-dev mailing list
lldb-dev@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev

Re: [lldb-dev] Benchmark tests

2015-12-09 Thread Jason Molenda via lldb-dev 
FWIW, nope, I've never messed with the benchmark tests.

> On Dec 9, 2015, at 1:22 PM, Todd Fiala  wrote:
> 
> Hey Jason,
> 
> Are you the benchmark user?
> 
> -Todd
> 
> On Wed, Dec 9, 2015 at 12:32 PM, Zachary Turner via lldb-dev 
>  wrote:
> Is anyone using the benchmark tests?  None of the command line options 
> related to the benchmark tests were claimed as being used by anyone.  Which 
> makes me wonder if the tests are even being used by anyone.  
> 
> What I really want to know is: Is it really ok to delete the -x and -y 
> command line options?  And what is the status of these tests?  Does anyone 
> use them?
> 
> ___
> lldb-dev mailing list
> lldb-dev@lists.llvm.org
> http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev
> 
> 
> 
> 
> -- 
> -Todd

___
lldb-dev mailing list
lldb-dev@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev

Re: [lldb-dev] swig generation and "six" module

2015-12-09 Thread Ted Woodward via lldb-dev 
Unfortunately, the LLDB we build needs to run on SLES 11, Ubuntu 10+ and who 
knows what else.  I don’t think I can require they install a six package. I can 
make our build do a copy instead of a link, if you want to keep it a link.

 

Are the static swig bindings checked in yet, or do we still need to run swig to 
generate them on each build?

 

--

Qualcomm Innovation Center, Inc.

The Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum, a Linux 
Foundation Collaborative Project

 

From: Zachary Turner [mailto:ztur...@google.com] 
Sent: Wednesday, December 09, 2015 3:47 PM
To: Ted Woodward; LLDB
Subject: Re: [lldb-dev] swig generation and "six" module

 

What about requiring that the user has done "pip install six" on their machine? 
 

 

On Wed, Dec 9, 2015 at 1:40 PM Ted Woodward via lldb-dev 
mailto:lldb-dev@lists.llvm.org> > wrote:

r252764 changes finishSwigPythonLLDB.py to symlink the “six” module in 
site-packages. six.py is a symlink to 
/tools/lldb/third_party/Python/module/six/six.py. This assumes I have 
access to this build’s sources when I run lldb, which I don’t – our 
workstations don’t have access to the buildbots’ filesystem, and even if they 
did, the sources may change between build and run.

 

Shouldn’t six.py just be copied? It’s only 30k.

 

Zach, how do I do a build using static swig bindings? I don’t think we’re going 
to change the bindings internally, so using the standard ones should be fine.

 

--

Qualcomm Innovation Center, Inc.

The Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum, a Linux 
Foundation Collaborative Project

 

___
lldb-dev mailing list
lldb-dev@lists.llvm.org  
http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev

___
lldb-dev mailing list
lldb-dev@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev

Re: [lldb-dev] Benchmark tests

2015-12-09 Thread Enrico Granata via lldb-dev 
Historically I would do

$ ./dotest.py +b 

but I am not strongly attached to that workflow - it's just what I learnt the 
first time I needed to run one

Sent from my iPhone

> On Dec 9, 2015, at 2:08 PM, Zachary Turner  wrote:
> 
> When you do run the benchmark tests, what command line options do you use?  
> At the moment I'm mostly just trying to remove dead options from the test 
> suite.  I removed one already that allowed you to specify the benchmark 
> executable, but then when I started looking at the rest and seeing how 
> tightly integrated they are with the benchamrk tests in general, I started to 
> wonder.
> 
> The three benchmark related command line options are:
> 
> 1. An option to specify the benchmark executable (defaults to lldb.exe)
> 2. An option to specify the breakpoint spec (defaults to -n main)
> 3. An option to specify the breakpoint iteration count (defaults to 30 I 
> think)
> 4. An option to specify that you only want to run benchmark tests and no 
> other tests.
> 
> I deleted #4 because you can use the category system for that.  I deleted #1 
> because nobody said they needed it on the spreadsheet.  Nobody said they 
> needed #2 or #3 either, but I just want to double check that deleting them is 
> fine.
> 
>> On Wed, Dec 9, 2015 at 2:01 PM Enrico Granata  wrote:
>> I have actually added a few benchmark tests recently. We admittedly are not 
>> that good with running those tests ever (because they're not run by default 
>> most likely - and I do wonder if some of them would take a long time to 
>> run.. I don't think I have ever run the full set, just my own as I increment 
>> on performance work).
>> 
>> Maybe we could try flipping the default to be "run the benchmarks", see if 
>> test suite run times explode and take it from there in terms of feasibility 
>> as well as whether they all still make sense.
>> 
>> The other problem with the tests as they stand is that they mark themselves 
>> as PASS or FAIL purely on the basis of whether they encounter command or API 
>> errors, and do nothing to track performance regressions. That is admittedly 
>> a harder problem to tackle given heterogeneous hardware and workload - but 
>> maybe we could have them fail if the timings go wildly crazy over some 
>> threshold?
>> 
>> Sent from my iPhone
>> 
>>> On Dec 9, 2015, at 1:22 PM, Todd Fiala via lldb-dev 
>>>  wrote:
>>> 
>>> Hey Jason,
>>> 
>>> Are you the benchmark user?
>>> 
>>> -Todd
>>> 
 On Wed, Dec 9, 2015 at 12:32 PM, Zachary Turner via lldb-dev 
  wrote:
 Is anyone using the benchmark tests?  None of the command line options 
 related to the benchmark tests were claimed as being used by anyone.  
 Which makes me wonder if the tests are even being used by anyone.  
 
 What I really want to know is: Is it really ok to delete the -x and -y 
 command line options?  And what is the status of these tests?  Does anyone 
 use them?
 
 ___
 lldb-dev mailing list
 lldb-dev@lists.llvm.org
 http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev
>>> 
>>> 
>>> 
>>> -- 
>>> -Todd
>>> ___
>>> lldb-dev mailing list
>>> lldb-dev@lists.llvm.org
>>> http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev
___
lldb-dev mailing list
lldb-dev@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev

Re: [lldb-dev] swig generation and "six" module

2015-12-09 Thread Zachary Turner via lldb-dev 
I don't mind it being a copy instead of a link, I had just been thinking
about getting rid of that workflow anyway and requiring a pip install.  But
if you can't do that, then scratch that idea.

Feel free to change it to a copy and upstream that if it makes your life
easier.

Just to be clear though, the issue with six is orthogonal to the issue with
static bindings, right?  We couldn't all agree on a way to get static
bindings in, so as of now it's not a thing.  I did some work to get a
swig-as-a-service thing going, and it's mostly complete except for the part
where I get a server and make it run on that server.  If I can push that
through static bindings will be a thing again, but I'm blocked on that for
the moment.

On Wed, Dec 9, 2015 at 2:33 PM Ted Woodward 
wrote:

> Unfortunately, the LLDB we build needs to run on SLES 11, Ubuntu 10+ and
> who knows what else.  I don’t think I can require they install a six
> package. I can make our build do a copy instead of a link, if you want to
> keep it a link.
>
>
>
> Are the static swig bindings checked in yet, or do we still need to run
> swig to generate them on each build?
>
>
>
> --
>
> Qualcomm Innovation Center, Inc.
>
> The Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum, a
> Linux Foundation Collaborative Project
>
>
>
> *From:* Zachary Turner [mailto:ztur...@google.com]
> *Sent:* Wednesday, December 09, 2015 3:47 PM
> *To:* Ted Woodward; LLDB
> *Subject:* Re: [lldb-dev] swig generation and "six" module
>
>
>
> What about requiring that the user has done "pip install six" on their
> machine?
>
>
>
> On Wed, Dec 9, 2015 at 1:40 PM Ted Woodward via lldb-dev <
> lldb-dev@lists.llvm.org> wrote:
>
> r252764 changes finishSwigPythonLLDB.py to symlink the “six” module in
> site-packages. six.py is a symlink to
> /tools/lldb/third_party/Python/module/six/six.py. This assumes I have
> access to this build’s sources when I run lldb, which I don’t – our
> workstations don’t have access to the buildbots’ filesystem, and even if
> they did, the sources may change between build and run.
>
>
>
> Shouldn’t six.py just be copied? It’s only 30k.
>
>
>
> Zach, how do I do a build using static swig bindings? I don’t think we’re
> going to change the bindings internally, so using the standard ones should
> be fine.
>
>
>
> --
>
> Qualcomm Innovation Center, Inc.
>
> The Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum, a
> Linux Foundation Collaborative Project
>
>
>
> ___
> lldb-dev mailing list
> lldb-dev@lists.llvm.org
> http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev
>
>
___
lldb-dev mailing list
lldb-dev@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev