Re: [lldb-dev] Exclusively build and install LLDB?

[Todd Fiala via lldb-dev]

Sorry for being late the the party here.

Sean Callanan and some of the other members can comment more on this, but
LLDB's expression parser for C/C++ is going to need access to the clang
include headers, so somehow lldb has to be able to find them.  Out of tree
llvm/clang usage is certainly possible as others have pointed out.  Using
that as the one way it is done, though, is likely to lead to pain.  Parts
of lldb's source will adjust as needed when the API surface area of LLVM or
clang changes.  It may not be happening quite as frequently as it had say 2
or 3 years ago, but it definitely happens.  So my expectation would be that
if you decouple lldb from llvm/clang (i.e. let them drift), sooner or later
you will get bitten by that.  Particularly when things like clang modules
and whatnot come along and actually require different logic on the lldb
side to deal with content generated on the clang/llvm side.  Once
expression evaluation is potentially compromised (due to the drift), I
suspect the lldb experience will degrade significantly.

On Sun, Nov 29, 2015 at 9:28 PM, Kamil Rytarowski via lldb-dev <
lldb-dev@lists.llvm.org> wrote:

> -BEGIN PGP SIGNED MESSAGE-
> Hash: SHA256
>
> On 27.11.2015 00:57, Kamil Rytarowski via lldb-dev wrote:
> > On 11/23/15 10:28, Pavel Labath wrote:
> >> I believe that for purposes of building distribution packages you
> >>  should use the out-of-tree mode of building lldb. This means,
> >> you build llvm and clang separately, and then point your LLDB
> >> build to their installation path with LLDB_PATH_TO_LLVM_BUILD and
> >>  LLDB_PATH_TO_CLANG_BUILD variables. This way you can avoid
> >> building llvm/clang twice, you can have a separate package for
> >> each logical component of llvm and you can make lldb optional for
> >> your users (e.g. have only clang installed normally, if user
> >> chooses to install lldb, it will automatically pull in clang if
> >> needed). In this mode "make install" should install only the lldb
> >> components, which should be correctly linked to the
> >> already-installed llvm libraries.
> >
> >> That said, I can't guarantee that this mode will work for you
> >> out-of-the-box. We occasionally get patches to fix it up, but I
> >> don't know anyone who is using it extensively. However, I think
> >> this would be the best way forward for you and I'm prepared yo
> >> help you out if you choose to go that way.
> >
> >> What do you think about that?
> >
> >> pl
> >
> >
> > Thank you for your note on this mode. I was trying to prototype a
> > set of packages with: sources of llvm and clang, build dirs of llvm
> > and clang and installations of llvm and clang.
> >
> > Badly this approach doesn't work with pkgsrc, as this framework
> > contains various checks against using sources, headers, executables
> > or other files out of the build tree. Packaging sources and build
> > tree triggers errors with moving invalid files into ${DESTDIR}.
> > Everything is wolkaroundable, but I think it's not the correct way
> > of handling it.
> >
> > I've checked that libcxx, cfe and compiler-rt ship with mechanism
> > to build against preinstalled LLVM. I will try them out and I'm
> > going to prepare new pkgsrc packages using this approach. Then I
> > will try to research doing the same with LLDB, exporting needed
> > libraries and headers for the compiler withing llvm and lldb.
>
> For the cross reference.
>
> A patch allowing to build (tested on NetBSD) out of sources pushed to
> review: reviews.llvm.org/D15067
> -BEGIN PGP SIGNATURE-
> Version: GnuPG v2
>
> iQIcBAEBCAAGBQJWW96JAAoJEEuzCOmwLnZsCLAP/0LrhGlzivOjtykjW3ywvXia
> wtZRLYsPwsNBJJERdOGVOJVPovnT02H+Bf1a4eDf0dJXbecklyfiupNfthlvFr9l
> PxCLZI4GQPLcP+jqWbhcFRdhzFeyEnLLy0Wjt1MNYG0s3m2u4jJM2ViNLA10/kwS
> XX82e2K7q5MUb51MMEJ09ufpYyGff7XmjVE78w1ekfNSRlKFMc0DNBsaIx4oKZfM
> G2IUtRNL59ad2pkw/xA3D2OPtoTk7+a2jjF8Z4nYY6kUSyBPUlCYrjyfavVCreR6
> 6Zo2E3lkkEb6PSIfb57RlMtxBIfmIBjv5w6OcFjSK6aYvffY9IMgyBJvGbdA+Ee7
> DlCFZax25eJPglEnfzAI2XOHOUQJtDwhb45N+XWshLfUax2e52KJvj9nq9J8pOse
> AC00qRQN+KTZsil43dlOfEn5m18mJ9o+CohK5eLMoTnS9QdtP8OEv72zGjOsSqrx
> vXDx01ziuQRCgsJ+niZXHgRLA65hxD5XgSGEBzr5prRLtU6q6V/HpYOsC46+pySc
> ibLrRWHnaeBVJknwz11iBo4gBZRk3lGhi5aTfu9+kcX6ylKSn2nn34+HGHr//FZi
> SrKcb3z7WikAR0c9cHBNOnwbro6o08j8zUE2l2S08risLRDDu01KBo3yFebjHz8D
> vQqFJNDkRLywQbXezcjB
> =7C7K
> -END PGP SIGNATURE-
> ___
> lldb-dev mailing list
> lldb-dev@lists.llvm.org
> http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev
>



-- 
-Todd
___
lldb-dev mailing list
lldb-dev@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev

Re: [lldb-dev] static swig bindings and xcode workspace

[Todd Fiala via lldb-dev]

Hi Zachary,

On Mon, Nov 30, 2015 at 9:23 AM, Zachary Turner  wrote:

> Has the xcode build been changed to use static bindings yet?
>

It is only in our downstream branches.  I stripped out support in llvm.org
lldb per our other threads.


>   I got to thinking that maybe it would make sense to put them alongside
> the xcode workspace folders, just to emphasize that the static bindings
> were an artifact of how the xcode build works.
>

We could do that.  Internally we also use that for builds other than Xcode,
so whatever solution I use (which is currently what I had proposed earlier
but now have only in our branches) really needs to work for more than Xcode.


>   This way we just say "Xcode build uses static bindings" and "CMake build
> needs an installed swig", and this is enforced at the directory level.
>
>
That's a great compromise, I appreciate your thoughts on that.  Since I
need it for more than Xcode, right now the solution I have in our branch is
working okay.


> In order to do this you'd have to probably make a new toplevel folder to
> house both the lldb.xcodeproj and lldb.xcworkspace folders, but I think
> that would be useful for other reasons as well.  For example, I want to
> check in a visual studio python solution for the test suite at some point,
> and it would make sense if all of this additional stuff was in one place.
> So perhaps something like:
>
> lldb
> |__ contrib
> |__ xcode
> |__ LLDBWrapPython.cpp
> |__ lldb.py
> |__ lldb.xcodeproj
> |__ lldb.xcworkspace
> |__ msvc
>
>
That structure may make sense.  That could live in llvm.org.  Then for
other OSes where I want similar behavior, I could just keep those parts in
our branch.  Ultimately I'd end up with multiple copies of the wrapper (for
any OS we may build for internally), but I could symlink so that's not
really any kind of issue.

This might work.


> I have been thinking about this idea of a contrib folder for a while
> anyway, but wanted to have more reasons to make use of it before I brought
> it up.
>
> Good idea?  Bad idea?  Thoughts?
>

I could see that layout making sense.  If we did something like that, I
think I'd separate moving the lldb.xcodeproj and lldb.xcworkspace from the
creation of the contrib folder.  (i.e. I'd start with the wrapper part in
there, and have the others move there at lower priority as a scheduling
thing --- there's a bit of work to make the workspace/project change but
should be totally doable).

I think I like the idea since it reduces the number of merge issues I'd
have to deal with.

-- 
-Todd
___
lldb-dev mailing list
lldb-dev@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev

Re: [lldb-dev] Exclusively build and install LLDB?

[Pavel Labath via lldb-dev]

On 2 December 2015 at 16:19, Todd Fiala  wrote:
> Sorry for being late the the party here.
>
> Sean Callanan and some of the other members can comment more on this, but
> LLDB's expression parser for C/C++ is going to need access to the clang
> include headers, so somehow lldb has to be able to find them.  Out of tree
> llvm/clang usage is certainly possible as others have pointed out.  Using
> that as the one way it is done, though, is likely to lead to pain.  Parts of
> lldb's source will adjust as needed when the API surface area of LLVM or
> clang changes.  It may not be happening quite as frequently as it had say 2
> or 3 years ago, but it definitely happens.  So my expectation would be that
> if you decouple lldb from llvm/clang (i.e. let them drift), sooner or later
> you will get bitten by that.  Particularly when things like clang modules
> and whatnot come along and actually require different logic on the lldb side
> to deal with content generated on the clang/llvm side.  Once expression
> evaluation is potentially compromised (due to the drift), I suspect the lldb
> experience will degrade significantly.

I think you have misunderstood our intentions here.

Kamil, correct me if I am wrong, but I don't think we are talking
about building lldb against a different version of clang. What we want
is just to be able to build and link lldb against an already-built
clang (of the same version). This is quite useful when you (as a
distribution maintainer) want to provide prebuilt packages. So, for
example you can have a "clang" and an "lldb" package. Users wishing to
install clang, just get the first one, while someone installing lldb
will get the correct clang package pulled automatically. I believe the
easiest way to build these packages is to use the standalone mode of
lldb (which already exists, and some people use that).

hope that makes sense,
pl
___
lldb-dev mailing list
lldb-dev@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev

Re: [lldb-dev] Linux core dump doesn't show listing when loaded

[Todd Fiala via lldb-dev]

Does our init file mechanism have the ability to do something conditionally
if it's a core file?  (i.e. do we already have a way to get Ted's desired
behavior via an inserted call to "thread backtrace all" that somehow gets
triggered by the init, but only when we're talking about a core file?)

Alternatively, Ted, you could have a wrapper script of some sort (think
lldb-core.{sh,bat} or something) that you call that sources an lldb
core-file-specific init file that sets up aliases and the like to start up
lldb how you want, maybe?

On Mon, Nov 30, 2015 at 9:32 AM, Greg Clayton via lldb-dev <
lldb-dev@lists.llvm.org> wrote:

> "thread list" should just list the threads and their stop reasons (no
> backtraces). If you want backtraces just do "thread backtrace all".
>
>
> On Nov 24, 2015, at 1:09 PM, Ted Woodward via lldb-dev <
> lldb-dev@lists.llvm.org> wrote:
> >
> > I’ve been working on an old rev that we’d released on; now I’m much
> closer to ToT as we move towards our next major Hexagon release.
> >
> > Core dumps on the old rev would print out a listing/disassembly for each
> thread in the core dump. Now it doesn’t.
> >
> > ToT does this, on x86 Linux:
> >
> > >bin/lldb ~/lldb_test/coredump/lincrash -c ~/lldb_test/coredump/lincore
> > (lldb) target create "/usr2/tedwood/lldb_test/coredump/lincrash" --core
> "/usr2/tedwood/lldb_test/coredump/lincore"
> > Core file '/usr2/tedwood/lldb_test/coredump/lincore' (x86_64) was loaded.
> > (lldb) thread list
> > Process 0 stopped
> > * thread #1: tid = 0, 0x00401190 lincrash`main + 16 at
> lincrash.c:5, name = 'lincrash', stop reason = signal SIGSEGV
> > (lldb)
> >
> > I can see the listing by going up and down the stack, but I’d like to
> see the listing on load. Is the no listing intended?
> >
> > Ted
> >
> > --
> > Qualcomm Innovation Center, Inc.
> > The Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum, a
> Linux Foundation Collaborative Project
> >
> > ___
> > lldb-dev mailing list
> > lldb-dev@lists.llvm.org
> > http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev
>
> ___
> lldb-dev mailing list
> lldb-dev@lists.llvm.org
> http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev
>



-- 
-Todd
___
lldb-dev mailing list
lldb-dev@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev

Re: [lldb-dev] Exclusively build and install LLDB?

[Todd Fiala via lldb-dev]

Yes, that concept came out in the thread.  I just wanted to make sure there
wasn't also a desire to park on a version of llvm/clang, and if so, that
the path there is not pleasant and definitely not intended to be supported
on top of tree svn/trunk.

Thanks for clarifying!

-Todd

On Wed, Dec 2, 2015 at 8:34 AM, Pavel Labath  wrote:

> On 2 December 2015 at 16:19, Todd Fiala  wrote:
> > Sorry for being late the the party here.
> >
> > Sean Callanan and some of the other members can comment more on this, but
> > LLDB's expression parser for C/C++ is going to need access to the clang
> > include headers, so somehow lldb has to be able to find them.  Out of
> tree
> > llvm/clang usage is certainly possible as others have pointed out.  Using
> > that as the one way it is done, though, is likely to lead to pain.
> Parts of
> > lldb's source will adjust as needed when the API surface area of LLVM or
> > clang changes.  It may not be happening quite as frequently as it had
> say 2
> > or 3 years ago, but it definitely happens.  So my expectation would be
> that
> > if you decouple lldb from llvm/clang (i.e. let them drift), sooner or
> later
> > you will get bitten by that.  Particularly when things like clang modules
> > and whatnot come along and actually require different logic on the lldb
> side
> > to deal with content generated on the clang/llvm side.  Once expression
> > evaluation is potentially compromised (due to the drift), I suspect the
> lldb
> > experience will degrade significantly.
>
> I think you have misunderstood our intentions here.
>
> Kamil, correct me if I am wrong, but I don't think we are talking
> about building lldb against a different version of clang. What we want
> is just to be able to build and link lldb against an already-built
> clang (of the same version). This is quite useful when you (as a
> distribution maintainer) want to provide prebuilt packages. So, for
> example you can have a "clang" and an "lldb" package. Users wishing to
> install clang, just get the first one, while someone installing lldb
> will get the correct clang package pulled automatically. I believe the
> easiest way to build these packages is to use the standalone mode of
> lldb (which already exists, and some people use that).
>
> hope that makes sense,
> pl
>



-- 
-Todd
___
lldb-dev mailing list
lldb-dev@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev

Re: [lldb-dev] Linux core dump doesn't show listing when loaded

[Jim Ingham via lldb-dev]

> On Dec 2, 2015, at 8:35 AM, Todd Fiala via lldb-dev  
> wrote:
> 
> Does our init file mechanism have the ability to do something conditionally 
> if it's a core file?  (i.e. do we already have a way to get Ted's desired 
> behavior via an inserted call to "thread backtrace all" that somehow gets 
> triggered by the init, but only when we're talking about a core file?)
> 
> Alternatively, Ted, you could have a wrapper script of some sort (think 
> lldb-core.{sh,bat} or something) that you call that sources an lldb 
> core-file-specific init file that sets up aliases and the like to start up 
> lldb how you want, maybe?

Seems to me, the question is, is "target create --core" the same as "target 
create" or is it "target create" followed by "attach".  In the latter case, for 
"real" processes, we always print the stop state.  Since you really are 
attaching to the core process when you load a core file, the behavior of 
printing the stop state feels more right to me.  As to how that prints, 
following this logic, that should be the same as any other stop printing, and 
is I think orthogonal to Ted's original question.

Jim


> 
> On Mon, Nov 30, 2015 at 9:32 AM, Greg Clayton via lldb-dev 
>  wrote:
> "thread list" should just list the threads and their stop reasons (no 
> backtraces). If you want backtraces just do "thread backtrace all".
> 
> 
> On Nov 24, 2015, at 1:09 PM, Ted Woodward via lldb-dev 
>  wrote:
> >
> > I’ve been working on an old rev that we’d released on; now I’m much closer 
> > to ToT as we move towards our next major Hexagon release.
> >
> > Core dumps on the old rev would print out a listing/disassembly for each 
> > thread in the core dump. Now it doesn’t.
> >
> > ToT does this, on x86 Linux:
> >
> > >bin/lldb ~/lldb_test/coredump/lincrash -c ~/lldb_test/coredump/lincore
> > (lldb) target create "/usr2/tedwood/lldb_test/coredump/lincrash" --core 
> > "/usr2/tedwood/lldb_test/coredump/lincore"
> > Core file '/usr2/tedwood/lldb_test/coredump/lincore' (x86_64) was loaded.
> > (lldb) thread list
> > Process 0 stopped
> > * thread #1: tid = 0, 0x00401190 lincrash`main + 16 at 
> > lincrash.c:5, name = 'lincrash', stop reason = signal SIGSEGV
> > (lldb)
> >
> > I can see the listing by going up and down the stack, but I’d like to see 
> > the listing on load. Is the no listing intended?
> >
> > Ted
> >
> > --
> > Qualcomm Innovation Center, Inc.
> > The Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum, a 
> > Linux Foundation Collaborative Project
> >
> > ___
> > lldb-dev mailing list
> > lldb-dev@lists.llvm.org
> > http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev
> 
> ___
> lldb-dev mailing list
> lldb-dev@lists.llvm.org
> http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev
> 
> 
> 
> -- 
> -Todd
> ___
> lldb-dev mailing list
> lldb-dev@lists.llvm.org
> http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev

___
lldb-dev mailing list
lldb-dev@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev

[lldb-dev] New test summary results formatter

[Todd Fiala via lldb-dev]

Hi all,

I just put up an optional test results formatter that is a prototype of
what we may move towards for our default test summary results.  It went in
here:

r254530

and you can try it out with something like:

time test/dotest.py --executable `pwd`/build/Debug/lldb --results-formatter
lldbsuite.test.basic_results_formatter.BasicResultsFormatter --results-file
st
out

Let me know if this satisfies the basic needs of counts and whatnot.  It
counts test method runs rather than all the oddball "file, class, etc."
counts we had before.

It prints out the Details section when there are details, and keeps it nice
and clean when there are none.

It also mentions a bit about test reruns up top, but that won't come into
play until I get the multi-test-pass, single-worker/low-load mechanism in
place, which will depend on newer rerun count awareness support.

The change also cleans up places where the test event framework was using
string codes and replaces them with symbolic constants.

Let me know what you think.  I can tweak it as needed to address testbot
and other needs.  Once it looks reasonable, I'd like to move over to using
it by default in the parallel test runner rather than the legacy support.

Thanks!
-- 
-Todd
___
lldb-dev mailing list
lldb-dev@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev

Re: [lldb-dev] New test summary results formatter

[Todd Fiala via lldb-dev]

On Wed, Dec 2, 2015 at 10:57 AM, Todd Fiala  wrote:

> Hi all,
>
> I just put up an optional test results formatter that is a prototype of
> what we may move towards for our default test summary results.  It went in
> here:
>
> r254530
>
> and you can try it out with something like:
>
> time test/dotest.py --executable `pwd`/build/Debug/lldb
> --results-formatter
> lldbsuite.test.basic_results_formatter.BasicResultsFormatter --results-file
> st
> out
>
>
I cut and paste my line, but more than likely for most people you'd just
want this:

test/dotest.py --results-formatter
lldbsuite.test.basic_results_formatter.BasicResultsFormatter --results-file
stdout

The other stuff was specific to my setup.  That line assumes you run from
the lldb source dir root.


Let me know if this satisfies the basic needs of counts and whatnot.  It
> counts test method runs rather than all the oddball "file, class, etc."
> counts we had before.
>
> It prints out the Details section when there are details, and keeps it
> nice and clean when there are none.
>
> It also mentions a bit about test reruns up top, but that won't come into
> play until I get the multi-test-pass, single-worker/low-load mechanism in
> place, which will depend on newer rerun count awareness support.
>
> The change also cleans up places where the test event framework was using
> string codes and replaces them with symbolic constants.
>
> Let me know what you think.  I can tweak it as needed to address testbot
> and other needs.  Once it looks reasonable, I'd like to move over to using
> it by default in the parallel test runner rather than the legacy support.
>
> Thanks!
> --
> -Todd
>



-- 
-Todd
___
lldb-dev mailing list
lldb-dev@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev

Re: [lldb-dev] New test summary results formatter

[Todd Fiala via lldb-dev]

Also, all the text in the summary is fixed-width lined up nicely, which may
not show in the commit message description if you're using a variable-width
font.  On a terminal it looks nice.

On Wed, Dec 2, 2015 at 11:01 AM, Todd Fiala  wrote:

>
>
> On Wed, Dec 2, 2015 at 10:57 AM, Todd Fiala  wrote:
>
>> Hi all,
>>
>> I just put up an optional test results formatter that is a prototype of
>> what we may move towards for our default test summary results.  It went in
>> here:
>>
>> r254530
>>
>> and you can try it out with something like:
>>
>> time test/dotest.py --executable `pwd`/build/Debug/lldb
>> --results-formatter
>> lldbsuite.test.basic_results_formatter.BasicResultsFormatter --results-file
>> st
>> out
>>
>>
> I cut and paste my line, but more than likely for most people you'd just
> want this:
>
> test/dotest.py --results-formatter
> lldbsuite.test.basic_results_formatter.BasicResultsFormatter --results-file
> stdout
>
> The other stuff was specific to my setup.  That line assumes you run from
> the lldb source dir root.
>
>
> Let me know if this satisfies the basic needs of counts and whatnot.  It
>> counts test method runs rather than all the oddball "file, class, etc."
>> counts we had before.
>>
>> It prints out the Details section when there are details, and keeps it
>> nice and clean when there are none.
>>
>> It also mentions a bit about test reruns up top, but that won't come into
>> play until I get the multi-test-pass, single-worker/low-load mechanism in
>> place, which will depend on newer rerun count awareness support.
>>
>> The change also cleans up places where the test event framework was using
>> string codes and replaces them with symbolic constants.
>>
>> Let me know what you think.  I can tweak it as needed to address testbot
>> and other needs.  Once it looks reasonable, I'd like to move over to using
>> it by default in the parallel test runner rather than the legacy support.
>>
>> Thanks!
>> --
>> -Todd
>>
>
>
>
> --
> -Todd
>



-- 
-Todd
___
lldb-dev mailing list
lldb-dev@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev

Re: [lldb-dev] Exclusively build and install LLDB?

[Jim Ingham via lldb-dev]

Todd is right, at runtime lldb does need to find some of the clang include 
files in order to build modules for its own purposes.  On an OS X install, 
these headers are put in:

LLDB.framework/Resources/Clang

and are:

 > ls
./  avx512vlbwintrin.h  lzcntintrin.h   
stdatomic.h
../ avx512vlintrin.hmm3dnow.h   
stdbool.h
Intrin.havxintrin.h mm_malloc.h stddef.h
__stddef_max_align_t.h  bmi2intrin.hmmintrin.h  stdint.h
__wmmintrin_aes.h   bmiintrin.h module.modulemap
stdnoreturn.h
__wmmintrin_pclmul.hcpuid.h nmmintrin.h 
tbmintrin.h
adxintrin.h emmintrin.h pmmintrin.h tgmath.h
altivec.h   f16cintrin.hpopcntintrin.h  
tmmintrin.h
ammintrin.h float.h prfchwintrin.h  unwind.h
arm_acle.h  fma4intrin.hrdseedintrin.h  vadefs.h
arm_neon.h  fmaintrin.h rtmintrin.h 
varargs.h
avx2intrin.hia32intrin.hshaintrin.h 
wmmintrin.h
avx512bwintrin.himmintrin.h smmintrin.h 
x86intrin.h
avx512erintrin.hiso646.hstdalign.h  
xmmintrin.h
avx512fintrin.h limits.hstdarg.h
xopintrin.h

Other than that, lldb shouldn't need to install any other clang bits for its 
own purposes - and on OS X at least lldb only installs itself and not any of 
the clang binaries, so in practice this can be made to work.

Also, building lldb requires a lot of clang/llvm headers that I don't think get 
installed in the normal course of things.  So I'm not sure how easy it is going 
to be to build lldb against an installed llvm/clang.  I couldn't tell for sure 
whether this was another of your goals, but it may take a fair bit of monkeying 
around if you want to do things this way.

Jim

> On Dec 2, 2015, at 8:19 AM, Todd Fiala via lldb-dev  
> wrote:
> 
> Sorry for being late the the party here.
> 
> Sean Callanan and some of the other members can comment more on this, but 
> LLDB's expression parser for C/C++ is going to need access to the clang 
> include headers, so somehow lldb has to be able to find them.  Out of tree 
> llvm/clang usage is certainly possible as others have pointed out.  Using 
> that as the one way it is done, though, is likely to lead to pain.  Parts of 
> lldb's source will adjust as needed when the API surface area of LLVM or 
> clang changes.  It may not be happening quite as frequently as it had say 2 
> or 3 years ago, but it definitely happens.  So my expectation would be that 
> if you decouple lldb from llvm/clang (i.e. let them drift), sooner or later 
> you will get bitten by that.  Particularly when things like clang modules and 
> whatnot come along and actually require different logic on the lldb side to 
> deal with content generated on the clang/llvm side.  Once expression 
> evaluation is potentially compromised (due to the drift), I suspect the lldb 
> experience will degrade significantly.
> 
> On Sun, Nov 29, 2015 at 9:28 PM, Kamil Rytarowski via lldb-dev 
>  wrote:
> -BEGIN PGP SIGNED MESSAGE-
> Hash: SHA256
> 
> On 27.11.2015 00:57, Kamil Rytarowski via lldb-dev wrote:
> > On 11/23/15 10:28, Pavel Labath wrote:
> >> I believe that for purposes of building distribution packages you
> >>  should use the out-of-tree mode of building lldb. This means,
> >> you build llvm and clang separately, and then point your LLDB
> >> build to their installation path with LLDB_PATH_TO_LLVM_BUILD and
> >>  LLDB_PATH_TO_CLANG_BUILD variables. This way you can avoid
> >> building llvm/clang twice, you can have a separate package for
> >> each logical component of llvm and you can make lldb optional for
> >> your users (e.g. have only clang installed normally, if user
> >> chooses to install lldb, it will automatically pull in clang if
> >> needed). In this mode "make install" should install only the lldb
> >> components, which should be correctly linked to the
> >> already-installed llvm libraries.
> >
> >> That said, I can't guarantee that this mode will work for you
> >> out-of-the-box. We occasionally get patches to fix it up, but I
> >> don't know anyone who is using it extensively. However, I think
> >> this would be the best way forward for you and I'm prepared yo
> >> help you out if you choose to go that way.
> >
> >> What do you think about that?
> >
> >> pl
> >
> >
> > Thank you for your note on this mode. I was trying to prototype a
> > set of packages with: sources of llvm and clang, build dirs of llvm
> > and clang and installations of llvm and clang.
> >
> > Badly this approach doesn't work with pkgsrc, as this framework
> > contains various checks against using sources, headers, executables
> > or other files ou

Re: [lldb-dev] static swig bindings and xcode workspace

[Todd Fiala via lldb-dev]

On Wed, Dec 2, 2015 at 8:28 AM, Todd Fiala  wrote:

> Hi Zachary,
>
> On Mon, Nov 30, 2015 at 9:23 AM, Zachary Turner 
> wrote:
>
>> Has the xcode build been changed to use static bindings yet?
>>
>
> It is only in our downstream branches.  I stripped out support in llvm.org
> lldb per our other threads.
>
>
>>   I got to thinking that maybe it would make sense to put them alongside
>> the xcode workspace folders, just to emphasize that the static bindings
>> were an artifact of how the xcode build works.
>>
>
> We could do that.  Internally we also use that for builds other than
> Xcode, so whatever solution I use (which is currently what I had proposed
> earlier but now have only in our branches) really needs to work for more
> than Xcode.
>
>
>>   This way we just say "Xcode build uses static bindings" and "CMake
>> build needs an installed swig", and this is enforced at the directory
>> level.
>>
>>
> That's a great compromise, I appreciate your thoughts on that.  Since I
> need it for more than Xcode, right now the solution I have in our branch is
> working okay.
>
>
>> In order to do this you'd have to probably make a new toplevel folder to
>> house both the lldb.xcodeproj and lldb.xcworkspace folders, but I think
>> that would be useful for other reasons as well.  For example, I want to
>> check in a visual studio python solution for the test suite at some point,
>> and it would make sense if all of this additional stuff was in one place.
>> So perhaps something like:
>>
>> lldb
>> |__ contrib
>> |__ xcode
>> |__ LLDBWrapPython.cpp
>> |__ lldb.py
>> |__ lldb.xcodeproj
>> |__ lldb.xcworkspace
>> |__ msvc
>>
>>
> That structure may make sense.  That could live in llvm.org.  Then for
> other OSes where I want similar behavior, I could just keep those parts in
> our branch.  Ultimately I'd end up with multiple copies of the wrapper (for
> any OS we may build for internally), but I could symlink so that's not
> really any kind of issue.
>
> This might work.
>
>
>> I have been thinking about this idea of a contrib folder for a while
>> anyway, but wanted to have more reasons to make use of it before I brought
>> it up.
>>
>> Good idea?  Bad idea?  Thoughts?
>>
>
> I could see that layout making sense.  If we did something like that, I
> think I'd separate moving the lldb.xcodeproj and lldb.xcworkspace from the
> creation of the contrib folder.
>

It looks like we may have some reasons why we need the Xcode
workspace/project files at the top of the lldb source tree.  I'm not sure
we'll able to move those.  But the rest of it looks like a reasonable way
to go.


>  (i.e. I'd start with the wrapper part in there, and have the others move
> there at lower priority as a scheduling thing --- there's a bit of work to
> make the workspace/project change but should be totally doable).
>
> I think I like the idea since it reduces the number of merge issues I'd
> have to deal with.
>
> --
> -Todd
>



-- 
-Todd
___
lldb-dev mailing list
lldb-dev@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev

Re: [lldb-dev] New test summary results formatter

[Zachary Turner via lldb-dev]

Can --results-file=stdout be the default so that we don't have to specify
that?

On Wed, Dec 2, 2015 at 11:02 AM Todd Fiala via lldb-dev <
lldb-dev@lists.llvm.org> wrote:

> Also, all the text in the summary is fixed-width lined up nicely, which
> may not show in the commit message description if you're using a
> variable-width font.  On a terminal it looks nice.
>
> On Wed, Dec 2, 2015 at 11:01 AM, Todd Fiala  wrote:
>
>>
>>
>> On Wed, Dec 2, 2015 at 10:57 AM, Todd Fiala  wrote:
>>
>>> Hi all,
>>>
>>> I just put up an optional test results formatter that is a prototype of
>>> what we may move towards for our default test summary results.  It went in
>>> here:
>>>
>>> r254530
>>>
>>> and you can try it out with something like:
>>>
>>> time test/dotest.py --executable `pwd`/build/Debug/lldb
>>> --results-formatter
>>> lldbsuite.test.basic_results_formatter.BasicResultsFormatter --results-file
>>> st
>>> out
>>>
>>>
>> I cut and paste my line, but more than likely for most people you'd just
>> want this:
>>
>> test/dotest.py --results-formatter
>> lldbsuite.test.basic_results_formatter.BasicResultsFormatter --results-file
>> stdout
>>
>> The other stuff was specific to my setup.  That line assumes you run from
>> the lldb source dir root.
>>
>>
>> Let me know if this satisfies the basic needs of counts and whatnot.  It
>>> counts test method runs rather than all the oddball "file, class, etc."
>>> counts we had before.
>>>
>>> It prints out the Details section when there are details, and keeps it
>>> nice and clean when there are none.
>>>
>>> It also mentions a bit about test reruns up top, but that won't come
>>> into play until I get the multi-test-pass, single-worker/low-load mechanism
>>> in place, which will depend on newer rerun count awareness support.
>>>
>>> The change also cleans up places where the test event framework was
>>> using string codes and replaces them with symbolic constants.
>>>
>>> Let me know what you think.  I can tweak it as needed to address testbot
>>> and other needs.  Once it looks reasonable, I'd like to move over to using
>>> it by default in the parallel test runner rather than the legacy support.
>>>
>>> Thanks!
>>> --
>>> -Todd
>>>
>>
>>
>>
>> --
>> -Todd
>>
>
>
>
> --
> -Todd
> ___
> lldb-dev mailing list
> lldb-dev@lists.llvm.org
> http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev
>
___
lldb-dev mailing list
lldb-dev@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev

Re: [lldb-dev] New test summary results formatter

[Todd Fiala via lldb-dev]

I think it might be already - let me check.

The longer-term goal would be to get this without specifying anything (i.e.
does what we want by default).  If stdout is not already being used by
default when a formatter is specified, that would be an easy fix.

Checking now...

On Wed, Dec 2, 2015 at 11:04 AM, Zachary Turner  wrote:

> Can --results-file=stdout be the default so that we don't have to specify
> that?
>
> On Wed, Dec 2, 2015 at 11:02 AM Todd Fiala via lldb-dev <
> lldb-dev@lists.llvm.org> wrote:
>
>> Also, all the text in the summary is fixed-width lined up nicely, which
>> may not show in the commit message description if you're using a
>> variable-width font.  On a terminal it looks nice.
>>
>> On Wed, Dec 2, 2015 at 11:01 AM, Todd Fiala  wrote:
>>
>>>
>>>
>>> On Wed, Dec 2, 2015 at 10:57 AM, Todd Fiala 
>>> wrote:
>>>
 Hi all,

 I just put up an optional test results formatter that is a prototype of
 what we may move towards for our default test summary results.  It went in
 here:

 r254530

 and you can try it out with something like:

 time test/dotest.py --executable `pwd`/build/Debug/lldb
 --results-formatter
 lldbsuite.test.basic_results_formatter.BasicResultsFormatter --results-file
 st
 out


>>> I cut and paste my line, but more than likely for most people you'd just
>>> want this:
>>>
>>> test/dotest.py --results-formatter
>>> lldbsuite.test.basic_results_formatter.BasicResultsFormatter --results-file
>>> stdout
>>>
>>> The other stuff was specific to my setup.  That line assumes you run
>>> from the lldb source dir root.
>>>
>>>
>>> Let me know if this satisfies the basic needs of counts and whatnot.  It
 counts test method runs rather than all the oddball "file, class, etc."
 counts we had before.

 It prints out the Details section when there are details, and keeps it
 nice and clean when there are none.

 It also mentions a bit about test reruns up top, but that won't come
 into play until I get the multi-test-pass, single-worker/low-load mechanism
 in place, which will depend on newer rerun count awareness support.

 The change also cleans up places where the test event framework was
 using string codes and replaces them with symbolic constants.

 Let me know what you think.  I can tweak it as needed to address
 testbot and other needs.  Once it looks reasonable, I'd like to move over
 to using it by default in the parallel test runner rather than the legacy
 support.

 Thanks!
 --
 -Todd

>>>
>>>
>>>
>>> --
>>> -Todd
>>>
>>
>>
>>
>> --
>> -Todd
>> ___
>> lldb-dev mailing list
>> lldb-dev@lists.llvm.org
>> http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev
>>
>


-- 
-Todd
___
lldb-dev mailing list
lldb-dev@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev

Re: [lldb-dev] New test summary results formatter

[Todd Fiala via lldb-dev]

Nope, it didn't work without the results-file entry.

I can get that fixed up.  I'll look at that now.

On Wed, Dec 2, 2015 at 11:05 AM, Todd Fiala  wrote:

> I think it might be already - let me check.
>
> The longer-term goal would be to get this without specifying anything
> (i.e. does what we want by default).  If stdout is not already being used
> by default when a formatter is specified, that would be an easy fix.
>
> Checking now...
>
> On Wed, Dec 2, 2015 at 11:04 AM, Zachary Turner 
> wrote:
>
>> Can --results-file=stdout be the default so that we don't have to specify
>> that?
>>
>> On Wed, Dec 2, 2015 at 11:02 AM Todd Fiala via lldb-dev <
>> lldb-dev@lists.llvm.org> wrote:
>>
>>> Also, all the text in the summary is fixed-width lined up nicely, which
>>> may not show in the commit message description if you're using a
>>> variable-width font.  On a terminal it looks nice.
>>>
>>> On Wed, Dec 2, 2015 at 11:01 AM, Todd Fiala 
>>> wrote:
>>>


 On Wed, Dec 2, 2015 at 10:57 AM, Todd Fiala 
 wrote:

> Hi all,
>
> I just put up an optional test results formatter that is a prototype
> of what we may move towards for our default test summary results.  It went
> in here:
>
> r254530
>
> and you can try it out with something like:
>
> time test/dotest.py --executable `pwd`/build/Debug/lldb
> --results-formatter
> lldbsuite.test.basic_results_formatter.BasicResultsFormatter 
> --results-file
> st
> out
>
>
 I cut and paste my line, but more than likely for most people you'd
 just want this:

 test/dotest.py --results-formatter
 lldbsuite.test.basic_results_formatter.BasicResultsFormatter --results-file
 stdout

 The other stuff was specific to my setup.  That line assumes you run
 from the lldb source dir root.


 Let me know if this satisfies the basic needs of counts and whatnot.
> It counts test method runs rather than all the oddball "file, class, etc."
> counts we had before.
>
> It prints out the Details section when there are details, and keeps it
> nice and clean when there are none.
>
> It also mentions a bit about test reruns up top, but that won't come
> into play until I get the multi-test-pass, single-worker/low-load 
> mechanism
> in place, which will depend on newer rerun count awareness support.
>
> The change also cleans up places where the test event framework was
> using string codes and replaces them with symbolic constants.
>
> Let me know what you think.  I can tweak it as needed to address
> testbot and other needs.  Once it looks reasonable, I'd like to move over
> to using it by default in the parallel test runner rather than the legacy
> support.
>
> Thanks!
> --
> -Todd
>



 --
 -Todd

>>>
>>>
>>>
>>> --
>>> -Todd
>>> ___
>>> lldb-dev mailing list
>>> lldb-dev@lists.llvm.org
>>> http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev
>>>
>>
>
>
> --
> -Todd
>



-- 
-Todd
___
lldb-dev mailing list
lldb-dev@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev

Re: [lldb-dev] New test summary results formatter

[Zachary Turner via lldb-dev]

Also another stylistic suggestion.  I've been thinking about how to more
logically organize all the source files now that we have a package.  So it
makes sense conceptually to group all of the different result formatters
under a subpackage called formatters.  So right now you've got
lldbsuite.test.basic_results_formatter.BasicResultsFormatter but it might
make sense for this to be
lldbsuite.test.formatters.basic.BasicResultsFormatter.  If you do things
this way, it can actually result in a substantially shorter command line,
because the --results-formatter option can use lldbsuite.test.formatters as
a starting point.  So you could instead write:

test/dotest.py --results-formatter basic

dotest then looks for a `basic.py` module in the
`lldbsuite.test.formatters` package, looks for a class inside with a
@result_formatter decorator, and instantiates that.

This has the advantage of making the command line shorter *and* a more
logical source file organization.

On Wed, Dec 2, 2015 at 11:04 AM Zachary Turner  wrote:

> Can --results-file=stdout be the default so that we don't have to specify
> that?
>
> On Wed, Dec 2, 2015 at 11:02 AM Todd Fiala via lldb-dev <
> lldb-dev@lists.llvm.org> wrote:
>
>> Also, all the text in the summary is fixed-width lined up nicely, which
>> may not show in the commit message description if you're using a
>> variable-width font.  On a terminal it looks nice.
>>
>> On Wed, Dec 2, 2015 at 11:01 AM, Todd Fiala  wrote:
>>
>>>
>>>
>>> On Wed, Dec 2, 2015 at 10:57 AM, Todd Fiala 
>>> wrote:
>>>
 Hi all,

 I just put up an optional test results formatter that is a prototype of
 what we may move towards for our default test summary results.  It went in
 here:

 r254530

 and you can try it out with something like:

 time test/dotest.py --executable `pwd`/build/Debug/lldb
 --results-formatter
 lldbsuite.test.basic_results_formatter.BasicResultsFormatter --results-file
 st
 out


>>> I cut and paste my line, but more than likely for most people you'd just
>>> want this:
>>>
>>> test/dotest.py --results-formatter
>>> lldbsuite.test.basic_results_formatter.BasicResultsFormatter --results-file
>>> stdout
>>>
>>> The other stuff was specific to my setup.  That line assumes you run
>>> from the lldb source dir root.
>>>
>>>
>>> Let me know if this satisfies the basic needs of counts and whatnot.  It
 counts test method runs rather than all the oddball "file, class, etc."
 counts we had before.

 It prints out the Details section when there are details, and keeps it
 nice and clean when there are none.

 It also mentions a bit about test reruns up top, but that won't come
 into play until I get the multi-test-pass, single-worker/low-load mechanism
 in place, which will depend on newer rerun count awareness support.

 The change also cleans up places where the test event framework was
 using string codes and replaces them with symbolic constants.

 Let me know what you think.  I can tweak it as needed to address
 testbot and other needs.  Once it looks reasonable, I'd like to move over
 to using it by default in the parallel test runner rather than the legacy
 support.

 Thanks!
 --
 -Todd

>>>
>>>
>>>
>>> --
>>> -Todd
>>>
>>
>>
>>
>> --
>> -Todd
>> ___
>> lldb-dev mailing list
>> lldb-dev@lists.llvm.org
>> http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev
>>
>
___
lldb-dev mailing list
lldb-dev@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev

Re: [lldb-dev] static swig bindings and xcode workspace

[Zachary Turner via lldb-dev]

Ahh, that's a bummer if true.  Because `contrib` is a nice way to group
together all the things that all the things that have specific maintainers
and so everyone else is allowed to break.

On Wed, Dec 2, 2015 at 11:04 AM Todd Fiala  wrote:

> On Wed, Dec 2, 2015 at 8:28 AM, Todd Fiala  wrote:
>
>> Hi Zachary,
>>
>> On Mon, Nov 30, 2015 at 9:23 AM, Zachary Turner 
>> wrote:
>>
>>> Has the xcode build been changed to use static bindings yet?
>>>
>>
>> It is only in our downstream branches.  I stripped out support in
>> llvm.org lldb per our other threads.
>>
>>
>>>   I got to thinking that maybe it would make sense to put them alongside
>>> the xcode workspace folders, just to emphasize that the static bindings
>>> were an artifact of how the xcode build works.
>>>
>>
>> We could do that.  Internally we also use that for builds other than
>> Xcode, so whatever solution I use (which is currently what I had proposed
>> earlier but now have only in our branches) really needs to work for more
>> than Xcode.
>>
>>
>>>   This way we just say "Xcode build uses static bindings" and "CMake
>>> build needs an installed swig", and this is enforced at the directory
>>> level.
>>>
>>>
>> That's a great compromise, I appreciate your thoughts on that.  Since I
>> need it for more than Xcode, right now the solution I have in our branch is
>> working okay.
>>
>>
>>> In order to do this you'd have to probably make a new toplevel folder to
>>> house both the lldb.xcodeproj and lldb.xcworkspace folders, but I think
>>> that would be useful for other reasons as well.  For example, I want to
>>> check in a visual studio python solution for the test suite at some point,
>>> and it would make sense if all of this additional stuff was in one place.
>>> So perhaps something like:
>>>
>>> lldb
>>> |__ contrib
>>> |__ xcode
>>> |__ LLDBWrapPython.cpp
>>> |__ lldb.py
>>> |__ lldb.xcodeproj
>>> |__ lldb.xcworkspace
>>> |__ msvc
>>>
>>>
>> That structure may make sense.  That could live in llvm.org.  Then for
>> other OSes where I want similar behavior, I could just keep those parts in
>> our branch.  Ultimately I'd end up with multiple copies of the wrapper (for
>> any OS we may build for internally), but I could symlink so that's not
>> really any kind of issue.
>>
>> This might work.
>>
>>
>>> I have been thinking about this idea of a contrib folder for a while
>>> anyway, but wanted to have more reasons to make use of it before I brought
>>> it up.
>>>
>>> Good idea?  Bad idea?  Thoughts?
>>>
>>
>> I could see that layout making sense.  If we did something like that, I
>> think I'd separate moving the lldb.xcodeproj and lldb.xcworkspace from the
>> creation of the contrib folder.
>>
>
> It looks like we may have some reasons why we need the Xcode
> workspace/project files at the top of the lldb source tree.  I'm not sure
> we'll able to move those.  But the rest of it looks like a reasonable way
> to go.
>
>
>>  (i.e. I'd start with the wrapper part in there, and have the others move
>> there at lower priority as a scheduling thing --- there's a bit of work to
>> make the workspace/project change but should be totally doable).
>>
>> I think I like the idea since it reduces the number of merge issues I'd
>> have to deal with.
>>
>> --
>> -Todd
>>
>
>
>
> --
> -Todd
>
___
lldb-dev mailing list
lldb-dev@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev

Re: [lldb-dev] New test summary results formatter

[Todd Fiala via lldb-dev]

Yeah I'd be good with that.  I can change that as well.

-Todd

On Wed, Dec 2, 2015 at 11:10 AM, Zachary Turner  wrote:

> Also another stylistic suggestion.  I've been thinking about how to more
> logically organize all the source files now that we have a package.  So it
> makes sense conceptually to group all of the different result formatters
> under a subpackage called formatters.  So right now you've got
> lldbsuite.test.basic_results_formatter.BasicResultsFormatter but it might
> make sense for this to be
> lldbsuite.test.formatters.basic.BasicResultsFormatter.  If you do things
> this way, it can actually result in a substantially shorter command line,
> because the --results-formatter option can use lldbsuite.test.formatters as
> a starting point.  So you could instead write:
>
> test/dotest.py --results-formatter basic
>
> dotest then looks for a `basic.py` module in the
> `lldbsuite.test.formatters` package, looks for a class inside with a
> @result_formatter decorator, and instantiates that.
>
> This has the advantage of making the command line shorter *and* a more
> logical source file organization.
>
> On Wed, Dec 2, 2015 at 11:04 AM Zachary Turner  wrote:
>
>> Can --results-file=stdout be the default so that we don't have to specify
>> that?
>>
>> On Wed, Dec 2, 2015 at 11:02 AM Todd Fiala via lldb-dev <
>> lldb-dev@lists.llvm.org> wrote:
>>
>>> Also, all the text in the summary is fixed-width lined up nicely, which
>>> may not show in the commit message description if you're using a
>>> variable-width font.  On a terminal it looks nice.
>>>
>>> On Wed, Dec 2, 2015 at 11:01 AM, Todd Fiala 
>>> wrote:
>>>


 On Wed, Dec 2, 2015 at 10:57 AM, Todd Fiala 
 wrote:

> Hi all,
>
> I just put up an optional test results formatter that is a prototype
> of what we may move towards for our default test summary results.  It went
> in here:
>
> r254530
>
> and you can try it out with something like:
>
> time test/dotest.py --executable `pwd`/build/Debug/lldb
> --results-formatter
> lldbsuite.test.basic_results_formatter.BasicResultsFormatter 
> --results-file
> st
> out
>
>
 I cut and paste my line, but more than likely for most people you'd
 just want this:

 test/dotest.py --results-formatter
 lldbsuite.test.basic_results_formatter.BasicResultsFormatter --results-file
 stdout

 The other stuff was specific to my setup.  That line assumes you run
 from the lldb source dir root.


 Let me know if this satisfies the basic needs of counts and whatnot.
> It counts test method runs rather than all the oddball "file, class, etc."
> counts we had before.
>
> It prints out the Details section when there are details, and keeps it
> nice and clean when there are none.
>
> It also mentions a bit about test reruns up top, but that won't come
> into play until I get the multi-test-pass, single-worker/low-load 
> mechanism
> in place, which will depend on newer rerun count awareness support.
>
> The change also cleans up places where the test event framework was
> using string codes and replaces them with symbolic constants.
>
> Let me know what you think.  I can tweak it as needed to address
> testbot and other needs.  Once it looks reasonable, I'd like to move over
> to using it by default in the parallel test runner rather than the legacy
> support.
>
> Thanks!
> --
> -Todd
>



 --
 -Todd

>>>
>>>
>>>
>>> --
>>> -Todd
>>> ___
>>> lldb-dev mailing list
>>> lldb-dev@lists.llvm.org
>>> http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev
>>>
>>


-- 
-Todd
___
lldb-dev mailing list
lldb-dev@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev

Re: [lldb-dev] New test summary results formatter

[Todd Fiala via lldb-dev]

I'll also move those couple counts (total tests run and the rerun count)
under the summary.  I missed that when evolving the code.

On Wed, Dec 2, 2015 at 11:20 AM, Todd Fiala  wrote:

> Yeah I'd be good with that.  I can change that as well.
>
> -Todd
>
> On Wed, Dec 2, 2015 at 11:10 AM, Zachary Turner 
> wrote:
>
>> Also another stylistic suggestion.  I've been thinking about how to more
>> logically organize all the source files now that we have a package.  So it
>> makes sense conceptually to group all of the different result formatters
>> under a subpackage called formatters.  So right now you've got
>> lldbsuite.test.basic_results_formatter.BasicResultsFormatter but it
>> might make sense for this to be
>> lldbsuite.test.formatters.basic.BasicResultsFormatter.  If you do things
>> this way, it can actually result in a substantially shorter command line,
>> because the --results-formatter option can use lldbsuite.test.formatters as
>> a starting point.  So you could instead write:
>>
>> test/dotest.py --results-formatter basic
>>
>> dotest then looks for a `basic.py` module in the
>> `lldbsuite.test.formatters` package, looks for a class inside with a
>> @result_formatter decorator, and instantiates that.
>>
>> This has the advantage of making the command line shorter *and* a more
>> logical source file organization.
>>
>> On Wed, Dec 2, 2015 at 11:04 AM Zachary Turner 
>> wrote:
>>
>>> Can --results-file=stdout be the default so that we don't have to
>>> specify that?
>>>
>>> On Wed, Dec 2, 2015 at 11:02 AM Todd Fiala via lldb-dev <
>>> lldb-dev@lists.llvm.org> wrote:
>>>
 Also, all the text in the summary is fixed-width lined up nicely, which
 may not show in the commit message description if you're using a
 variable-width font.  On a terminal it looks nice.

 On Wed, Dec 2, 2015 at 11:01 AM, Todd Fiala 
 wrote:

>
>
> On Wed, Dec 2, 2015 at 10:57 AM, Todd Fiala 
> wrote:
>
>> Hi all,
>>
>> I just put up an optional test results formatter that is a prototype
>> of what we may move towards for our default test summary results.  It 
>> went
>> in here:
>>
>> r254530
>>
>> and you can try it out with something like:
>>
>> time test/dotest.py --executable `pwd`/build/Debug/lldb
>> --results-formatter
>> lldbsuite.test.basic_results_formatter.BasicResultsFormatter 
>> --results-file
>> st
>> out
>>
>>
> I cut and paste my line, but more than likely for most people you'd
> just want this:
>
> test/dotest.py --results-formatter
> lldbsuite.test.basic_results_formatter.BasicResultsFormatter 
> --results-file
> stdout
>
> The other stuff was specific to my setup.  That line assumes you run
> from the lldb source dir root.
>
>
> Let me know if this satisfies the basic needs of counts and whatnot.
>> It counts test method runs rather than all the oddball "file, class, 
>> etc."
>> counts we had before.
>>
>> It prints out the Details section when there are details, and keeps
>> it nice and clean when there are none.
>>
>> It also mentions a bit about test reruns up top, but that won't come
>> into play until I get the multi-test-pass, single-worker/low-load 
>> mechanism
>> in place, which will depend on newer rerun count awareness support.
>>
>> The change also cleans up places where the test event framework was
>> using string codes and replaces them with symbolic constants.
>>
>> Let me know what you think.  I can tweak it as needed to address
>> testbot and other needs.  Once it looks reasonable, I'd like to move over
>> to using it by default in the parallel test runner rather than the legacy
>> support.
>>
>> Thanks!
>> --
>> -Todd
>>
>
>
>
> --
> -Todd
>



 --
 -Todd
 ___
 lldb-dev mailing list
 lldb-dev@lists.llvm.org
 http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev

>>>
>
>
> --
> -Todd
>



-- 
-Todd
___
lldb-dev mailing list
lldb-dev@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev

Re: [lldb-dev] static swig bindings and xcode workspace

[Todd Fiala via lldb-dev]

I'll dig in more when I get to it, but yeah I like the concept for sure.

On Wed, Dec 2, 2015 at 11:12 AM, Zachary Turner  wrote:

> Ahh, that's a bummer if true.  Because `contrib` is a nice way to group
> together all the things that all the things that have specific maintainers
> and so everyone else is allowed to break.
>
> On Wed, Dec 2, 2015 at 11:04 AM Todd Fiala  wrote:
>
>> On Wed, Dec 2, 2015 at 8:28 AM, Todd Fiala  wrote:
>>
>>> Hi Zachary,
>>>
>>> On Mon, Nov 30, 2015 at 9:23 AM, Zachary Turner 
>>> wrote:
>>>
 Has the xcode build been changed to use static bindings yet?

>>>
>>> It is only in our downstream branches.  I stripped out support in
>>> llvm.org lldb per our other threads.
>>>
>>>
   I got to thinking that maybe it would make sense to put them
 alongside the xcode workspace folders, just to emphasize that the static
 bindings were an artifact of how the xcode build works.

>>>
>>> We could do that.  Internally we also use that for builds other than
>>> Xcode, so whatever solution I use (which is currently what I had proposed
>>> earlier but now have only in our branches) really needs to work for more
>>> than Xcode.
>>>
>>>
   This way we just say "Xcode build uses static bindings" and "CMake
 build needs an installed swig", and this is enforced at the directory
 level.


>>> That's a great compromise, I appreciate your thoughts on that.  Since I
>>> need it for more than Xcode, right now the solution I have in our branch is
>>> working okay.
>>>
>>>
 In order to do this you'd have to probably make a new toplevel folder
 to house both the lldb.xcodeproj and lldb.xcworkspace folders, but I think
 that would be useful for other reasons as well.  For example, I want to
 check in a visual studio python solution for the test suite at some point,
 and it would make sense if all of this additional stuff was in one place.
 So perhaps something like:

 lldb
 |__ contrib
 |__ xcode
 |__ LLDBWrapPython.cpp
 |__ lldb.py
 |__ lldb.xcodeproj
 |__ lldb.xcworkspace
 |__ msvc


>>> That structure may make sense.  That could live in llvm.org.  Then for
>>> other OSes where I want similar behavior, I could just keep those parts in
>>> our branch.  Ultimately I'd end up with multiple copies of the wrapper (for
>>> any OS we may build for internally), but I could symlink so that's not
>>> really any kind of issue.
>>>
>>> This might work.
>>>
>>>
 I have been thinking about this idea of a contrib folder for a while
 anyway, but wanted to have more reasons to make use of it before I brought
 it up.

 Good idea?  Bad idea?  Thoughts?

>>>
>>> I could see that layout making sense.  If we did something like that, I
>>> think I'd separate moving the lldb.xcodeproj and lldb.xcworkspace from the
>>> creation of the contrib folder.
>>>
>>
>> It looks like we may have some reasons why we need the Xcode
>> workspace/project files at the top of the lldb source tree.  I'm not sure
>> we'll able to move those.  But the rest of it looks like a reasonable way
>> to go.
>>
>>
>>>  (i.e. I'd start with the wrapper part in there, and have the others
>>> move there at lower priority as a scheduling thing --- there's a bit of
>>> work to make the workspace/project change but should be totally doable).
>>>
>>> I think I like the idea since it reduces the number of merge issues I'd
>>> have to deal with.
>>>
>>> --
>>> -Todd
>>>
>>
>>
>>
>> --
>> -Todd
>>
>


-- 
-Todd
___
lldb-dev mailing list
lldb-dev@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev

Re: [lldb-dev] New test summary results formatter

[Todd Fiala via lldb-dev]

On Wed, Dec 2, 2015 at 11:20 AM, Todd Fiala  wrote:

> Yeah I'd be good with that.  I can change that as well.
>
> -Todd
>
> On Wed, Dec 2, 2015 at 11:10 AM, Zachary Turner 
> wrote:
>
>> Also another stylistic suggestion.  I've been thinking about how to more
>> logically organize all the source files now that we have a package.  So it
>> makes sense conceptually to group all of the different result formatters
>> under a subpackage called formatters.  So right now you've got
>> lldbsuite.test.basic_results_formatter.BasicResultsFormatter but it
>> might make sense for this to be
>> lldbsuite.test.formatters.basic.BasicResultsFormatter.  If you do things
>> this way, it can actually result in a substantially shorter command line,
>> because the --results-formatter option can use lldbsuite.test.formatters as
>> a starting point.  So you could instead write:
>>
>> test/dotest.py --results-formatter basic
>>
>> dotest then looks for a `basic.py` module in the
>> `lldbsuite.test.formatters` package, looks for a class inside with a
>> @result_formatter decorator, and instantiates that.
>>
>> This has the advantage of making the command line shorter *and* a more
>> logical source file organization.
>>
>
The other thing that could allow me to do is possibly short-circuit the
results formatter specifier so that, if just the module is specified, and
if the module only has one ResultsFormatter-derived class, I can probably
rig up code that figures out the right results formatter, shortening the
required discriminator to something even shorter (i.e. module.classname
becomes just module.)


>
>> On Wed, Dec 2, 2015 at 11:04 AM Zachary Turner 
>> wrote:
>>
>>> Can --results-file=stdout be the default so that we don't have to
>>> specify that?
>>>
>>> On Wed, Dec 2, 2015 at 11:02 AM Todd Fiala via lldb-dev <
>>> lldb-dev@lists.llvm.org> wrote:
>>>
 Also, all the text in the summary is fixed-width lined up nicely, which
 may not show in the commit message description if you're using a
 variable-width font.  On a terminal it looks nice.

 On Wed, Dec 2, 2015 at 11:01 AM, Todd Fiala 
 wrote:

>
>
> On Wed, Dec 2, 2015 at 10:57 AM, Todd Fiala 
> wrote:
>
>> Hi all,
>>
>> I just put up an optional test results formatter that is a prototype
>> of what we may move towards for our default test summary results.  It 
>> went
>> in here:
>>
>> r254530
>>
>> and you can try it out with something like:
>>
>> time test/dotest.py --executable `pwd`/build/Debug/lldb
>> --results-formatter
>> lldbsuite.test.basic_results_formatter.BasicResultsFormatter 
>> --results-file
>> st
>> out
>>
>>
> I cut and paste my line, but more than likely for most people you'd
> just want this:
>
> test/dotest.py --results-formatter
> lldbsuite.test.basic_results_formatter.BasicResultsFormatter 
> --results-file
> stdout
>
> The other stuff was specific to my setup.  That line assumes you run
> from the lldb source dir root.
>
>
> Let me know if this satisfies the basic needs of counts and whatnot.
>> It counts test method runs rather than all the oddball "file, class, 
>> etc."
>> counts we had before.
>>
>> It prints out the Details section when there are details, and keeps
>> it nice and clean when there are none.
>>
>> It also mentions a bit about test reruns up top, but that won't come
>> into play until I get the multi-test-pass, single-worker/low-load 
>> mechanism
>> in place, which will depend on newer rerun count awareness support.
>>
>> The change also cleans up places where the test event framework was
>> using string codes and replaces them with symbolic constants.
>>
>> Let me know what you think.  I can tweak it as needed to address
>> testbot and other needs.  Once it looks reasonable, I'd like to move over
>> to using it by default in the parallel test runner rather than the legacy
>> support.
>>
>> Thanks!
>> --
>> -Todd
>>
>
>
>
> --
> -Todd
>



 --
 -Todd
 ___
 lldb-dev mailing list
 lldb-dev@lists.llvm.org
 http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev

>>>
>
>
> --
> -Todd
>



-- 
-Todd
___
lldb-dev mailing list
lldb-dev@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev

Re: [lldb-dev] New test summary results formatter

[Zachary Turner via lldb-dev]

Oh yea, I made up that decorator idea because I didn't know all the
formatters were derived from a common base.  But your idea is better if
everything is derived from a common base.  To be honest you could even just
generate an error if there are two ResultsFormatter derived classes in the
same module.  We should be encouraging more smaller files with single
responsibility.  One of the things I plan to do as part of some cleanup in
a week or two is to split up dotest, dosep, and lldbtest.py into a couple
different files by breaking out things like TestBase, etc into separate
files.  So that it's easier to keep a mental map of where different code is.

On Wed, Dec 2, 2015 at 11:26 AM Todd Fiala  wrote:

> On Wed, Dec 2, 2015 at 11:20 AM, Todd Fiala  wrote:
>
>> Yeah I'd be good with that.  I can change that as well.
>>
>> -Todd
>>
>> On Wed, Dec 2, 2015 at 11:10 AM, Zachary Turner 
>> wrote:
>>
>>> Also another stylistic suggestion.  I've been thinking about how to more
>>> logically organize all the source files now that we have a package.  So it
>>> makes sense conceptually to group all of the different result formatters
>>> under a subpackage called formatters.  So right now you've got
>>> lldbsuite.test.basic_results_formatter.BasicResultsFormatter but it
>>> might make sense for this to be
>>> lldbsuite.test.formatters.basic.BasicResultsFormatter.  If you do things
>>> this way, it can actually result in a substantially shorter command line,
>>> because the --results-formatter option can use lldbsuite.test.formatters as
>>> a starting point.  So you could instead write:
>>>
>>> test/dotest.py --results-formatter basic
>>>
>>> dotest then looks for a `basic.py` module in the
>>> `lldbsuite.test.formatters` package, looks for a class inside with a
>>> @result_formatter decorator, and instantiates that.
>>>
>>> This has the advantage of making the command line shorter *and* a more
>>> logical source file organization.
>>>
>>
> The other thing that could allow me to do is possibly short-circuit the
> results formatter specifier so that, if just the module is specified, and
> if the module only has one ResultsFormatter-derived class, I can probably
> rig up code that figures out the right results formatter, shortening the
> required discriminator to something even shorter (i.e. module.classname
> becomes just module.)
>
>
>>
>>> On Wed, Dec 2, 2015 at 11:04 AM Zachary Turner 
>>> wrote:
>>>
 Can --results-file=stdout be the default so that we don't have to
 specify that?

 On Wed, Dec 2, 2015 at 11:02 AM Todd Fiala via lldb-dev <
 lldb-dev@lists.llvm.org> wrote:

> Also, all the text in the summary is fixed-width lined up nicely,
> which may not show in the commit message description if you're using a
> variable-width font.  On a terminal it looks nice.
>
> On Wed, Dec 2, 2015 at 11:01 AM, Todd Fiala 
> wrote:
>
>>
>>
>> On Wed, Dec 2, 2015 at 10:57 AM, Todd Fiala 
>> wrote:
>>
>>> Hi all,
>>>
>>> I just put up an optional test results formatter that is a prototype
>>> of what we may move towards for our default test summary results.  It 
>>> went
>>> in here:
>>>
>>> r254530
>>>
>>> and you can try it out with something like:
>>>
>>> time test/dotest.py --executable `pwd`/build/Debug/lldb
>>> --results-formatter
>>> lldbsuite.test.basic_results_formatter.BasicResultsFormatter 
>>> --results-file
>>> st
>>> out
>>>
>>>
>> I cut and paste my line, but more than likely for most people you'd
>> just want this:
>>
>> test/dotest.py --results-formatter
>> lldbsuite.test.basic_results_formatter.BasicResultsFormatter 
>> --results-file
>> stdout
>>
>> The other stuff was specific to my setup.  That line assumes you run
>> from the lldb source dir root.
>>
>>
>> Let me know if this satisfies the basic needs of counts and whatnot.
>>> It counts test method runs rather than all the oddball "file, class, 
>>> etc."
>>> counts we had before.
>>>
>>> It prints out the Details section when there are details, and keeps
>>> it nice and clean when there are none.
>>>
>>> It also mentions a bit about test reruns up top, but that won't come
>>> into play until I get the multi-test-pass, single-worker/low-load 
>>> mechanism
>>> in place, which will depend on newer rerun count awareness support.
>>>
>>> The change also cleans up places where the test event framework was
>>> using string codes and replaces them with symbolic constants.
>>>
>>> Let me know what you think.  I can tweak it as needed to address
>>> testbot and other needs.  Once it looks reasonable, I'd like to move 
>>> over
>>> to using it by default in the parallel test runner rather than the 
>>> legacy
>>> support.
>>>
>>> Thanks!
>>> --
>>> -Todd
>>>
>>
>>>

Re: [lldb-dev] New test summary results formatter

[Zachary Turner via lldb-dev]

When I run this under Python 3 I get "A bytes object is used like a string"
on Line 1033 of test_results.py.  I'm going to dig into it a little bit,
but maybe you know off the top of your head the right way to fix it.

On Wed, Dec 2, 2015 at 11:32 AM Zachary Turner  wrote:

> Oh yea, I made up that decorator idea because I didn't know all the
> formatters were derived from a common base.  But your idea is better if
> everything is derived from a common base.  To be honest you could even just
> generate an error if there are two ResultsFormatter derived classes in the
> same module.  We should be encouraging more smaller files with single
> responsibility.  One of the things I plan to do as part of some cleanup in
> a week or two is to split up dotest, dosep, and lldbtest.py into a couple
> different files by breaking out things like TestBase, etc into separate
> files.  So that it's easier to keep a mental map of where different code is.
>
> On Wed, Dec 2, 2015 at 11:26 AM Todd Fiala  wrote:
>
>> On Wed, Dec 2, 2015 at 11:20 AM, Todd Fiala  wrote:
>>
>>> Yeah I'd be good with that.  I can change that as well.
>>>
>>> -Todd
>>>
>>> On Wed, Dec 2, 2015 at 11:10 AM, Zachary Turner 
>>> wrote:
>>>
 Also another stylistic suggestion.  I've been thinking about how to
 more logically organize all the source files now that we have a package.
 So it makes sense conceptually to group all of the different result
 formatters under a subpackage called formatters.  So right now you've got
 lldbsuite.test.basic_results_formatter.BasicResultsFormatter but it
 might make sense for this to be
 lldbsuite.test.formatters.basic.BasicResultsFormatter.  If you do things
 this way, it can actually result in a substantially shorter command line,
 because the --results-formatter option can use lldbsuite.test.formatters as
 a starting point.  So you could instead write:

 test/dotest.py --results-formatter basic

 dotest then looks for a `basic.py` module in the
 `lldbsuite.test.formatters` package, looks for a class inside with a
 @result_formatter decorator, and instantiates that.

 This has the advantage of making the command line shorter *and* a more
 logical source file organization.

>>>
>> The other thing that could allow me to do is possibly short-circuit the
>> results formatter specifier so that, if just the module is specified, and
>> if the module only has one ResultsFormatter-derived class, I can probably
>> rig up code that figures out the right results formatter, shortening the
>> required discriminator to something even shorter (i.e. module.classname
>> becomes just module.)
>>
>>
>>>
 On Wed, Dec 2, 2015 at 11:04 AM Zachary Turner 
 wrote:

> Can --results-file=stdout be the default so that we don't have to
> specify that?
>
> On Wed, Dec 2, 2015 at 11:02 AM Todd Fiala via lldb-dev <
> lldb-dev@lists.llvm.org> wrote:
>
>> Also, all the text in the summary is fixed-width lined up nicely,
>> which may not show in the commit message description if you're using a
>> variable-width font.  On a terminal it looks nice.
>>
>> On Wed, Dec 2, 2015 at 11:01 AM, Todd Fiala 
>> wrote:
>>
>>>
>>>
>>> On Wed, Dec 2, 2015 at 10:57 AM, Todd Fiala 
>>> wrote:
>>>
 Hi all,

 I just put up an optional test results formatter that is a
 prototype of what we may move towards for our default test summary
 results.  It went in here:

 r254530

 and you can try it out with something like:

 time test/dotest.py --executable `pwd`/build/Debug/lldb
 --results-formatter
 lldbsuite.test.basic_results_formatter.BasicResultsFormatter 
 --results-file
 st
 out


>>> I cut and paste my line, but more than likely for most people you'd
>>> just want this:
>>>
>>> test/dotest.py --results-formatter
>>> lldbsuite.test.basic_results_formatter.BasicResultsFormatter 
>>> --results-file
>>> stdout
>>>
>>> The other stuff was specific to my setup.  That line assumes you run
>>> from the lldb source dir root.
>>>
>>>
>>> Let me know if this satisfies the basic needs of counts and
 whatnot.  It counts test method runs rather than all the oddball "file,
 class, etc." counts we had before.

 It prints out the Details section when there are details, and keeps
 it nice and clean when there are none.

 It also mentions a bit about test reruns up top, but that won't
 come into play until I get the multi-test-pass, single-worker/low-load
 mechanism in place, which will depend on newer rerun count awareness
 support.

 The change also cleans up places where the test event framework was
 using string codes and repl

Re: [lldb-dev] New test summary results formatter

[Zachary Turner via lldb-dev]

Is there any way to force the single process test runner to use this same
system?  I'm trying to debug the problem, but this codepath doesn't execute
in the single process test runner, and it executes in the child process in
the multiprocess test runner.  Basically I need the following callstack to
execute in the single process test runner:

Command invoked: C:\Python35\python_d.exe
D:\src\llvm\tools\lldb\test\dotest.py -q --arch=i686 --executable
D:/src/llvmbuild/ninja_py35/bin/lldb.exe -s
D:/src/llvmbuild/ninja_py35/lldb-test-traces -u CXXFLAGS -u CFLAGS
--enable-crash-dialog -C d:\src\llvmbuild\ninja_release\bin\clang.exe
--results-port 60794 --inferior -p TestIntegerTypesExpr.py
D:\src\llvm\tools\lldb\packages\Python\lldbsuite\test --event-add-entries
worker_index=7:int
411 out of 412 test suites processed - TestIntegerTypesExpr.py
Traceback (most recent call last):
  File "D:\src\llvm\tools\lldb\test\dotest.py", line 7, in 
lldbsuite.test.run_suite()
  File "D:\src\llvm\tools\lldb\packages\Python\lldbsuite\test\dotest.py",
line 1476, in run_suite
setupTestResults()
  File "D:\src\llvm\tools\lldb\packages\Python\lldbsuite\test\dotest.py",
line 982, in setupTestResults
results_formatter_object.handle_event(initialize_event)
  File
"D:\src\llvm\tools\lldb\packages\Python\lldbsuite\test\test_results.py",
line 1033, in handle_event
"{}#{}".format(len(pickled_message), pickled_message))
TypeError: a bytes-like object is required, not 'str'

On Wed, Dec 2, 2015 at 11:40 AM Zachary Turner  wrote:

> When I run this under Python 3 I get "A bytes object is used like a
> string" on Line 1033 of test_results.py.  I'm going to dig into it a little
> bit, but maybe you know off the top of your head the right way to fix it.
>
> On Wed, Dec 2, 2015 at 11:32 AM Zachary Turner  wrote:
>
>> Oh yea, I made up that decorator idea because I didn't know all the
>> formatters were derived from a common base.  But your idea is better if
>> everything is derived from a common base.  To be honest you could even just
>> generate an error if there are two ResultsFormatter derived classes in the
>> same module.  We should be encouraging more smaller files with single
>> responsibility.  One of the things I plan to do as part of some cleanup in
>> a week or two is to split up dotest, dosep, and lldbtest.py into a couple
>> different files by breaking out things like TestBase, etc into separate
>> files.  So that it's easier to keep a mental map of where different code is.
>>
>> On Wed, Dec 2, 2015 at 11:26 AM Todd Fiala  wrote:
>>
>>> On Wed, Dec 2, 2015 at 11:20 AM, Todd Fiala 
>>> wrote:
>>>
 Yeah I'd be good with that.  I can change that as well.

 -Todd

 On Wed, Dec 2, 2015 at 11:10 AM, Zachary Turner 
 wrote:

> Also another stylistic suggestion.  I've been thinking about how to
> more logically organize all the source files now that we have a package.
> So it makes sense conceptually to group all of the different result
> formatters under a subpackage called formatters.  So right now you've got
> lldbsuite.test.basic_results_formatter.BasicResultsFormatter but it
> might make sense for this to be
> lldbsuite.test.formatters.basic.BasicResultsFormatter.  If you do things
> this way, it can actually result in a substantially shorter command line,
> because the --results-formatter option can use lldbsuite.test.formatters 
> as
> a starting point.  So you could instead write:
>
> test/dotest.py --results-formatter basic
>
> dotest then looks for a `basic.py` module in the
> `lldbsuite.test.formatters` package, looks for a class inside with a
> @result_formatter decorator, and instantiates that.
>
> This has the advantage of making the command line shorter *and* a more
> logical source file organization.
>

>>> The other thing that could allow me to do is possibly short-circuit the
>>> results formatter specifier so that, if just the module is specified, and
>>> if the module only has one ResultsFormatter-derived class, I can probably
>>> rig up code that figures out the right results formatter, shortening the
>>> required discriminator to something even shorter (i.e. module.classname
>>> becomes just module.)
>>>
>>>

> On Wed, Dec 2, 2015 at 11:04 AM Zachary Turner 
> wrote:
>
>> Can --results-file=stdout be the default so that we don't have to
>> specify that?
>>
>> On Wed, Dec 2, 2015 at 11:02 AM Todd Fiala via lldb-dev <
>> lldb-dev@lists.llvm.org> wrote:
>>
>>> Also, all the text in the summary is fixed-width lined up nicely,
>>> which may not show in the commit message description if you're using a
>>> variable-width font.  On a terminal it looks nice.
>>>
>>> On Wed, Dec 2, 2015 at 11:01 AM, Todd Fiala 
>>> wrote:
>>>


 On Wed, Dec 2, 2015 at 10:57 AM, Todd Fiala 
 wrote:

> Hi

Re: [lldb-dev] New test summary results formatter

[Todd Fiala via lldb-dev]

I think I can fix the issue without you debugging.

Getting the single pass test runner to use it isn't impossible but will
take some work.  Can you direct-send me the backtrace from the point of
failure from your system?  Thanks!

-Todd

On Wed, Dec 2, 2015 at 12:34 PM, Zachary Turner  wrote:

> Is there any way to force the single process test runner to use this same
> system?  I'm trying to debug the problem, but this codepath doesn't execute
> in the single process test runner, and it executes in the child process in
> the multiprocess test runner.  Basically I need the following callstack to
> execute in the single process test runner:
>
> Command invoked: C:\Python35\python_d.exe
> D:\src\llvm\tools\lldb\test\dotest.py -q --arch=i686 --executable
> D:/src/llvmbuild/ninja_py35/bin/lldb.exe -s
> D:/src/llvmbuild/ninja_py35/lldb-test-traces -u CXXFLAGS -u CFLAGS
> --enable-crash-dialog -C d:\src\llvmbuild\ninja_release\bin\clang.exe
> --results-port 60794 --inferior -p TestIntegerTypesExpr.py
> D:\src\llvm\tools\lldb\packages\Python\lldbsuite\test --event-add-entries
> worker_index=7:int
> 411 out of 412 test suites processed - TestIntegerTypesExpr.py
> Traceback (most recent call last):
>   File "D:\src\llvm\tools\lldb\test\dotest.py", line 7, in 
> lldbsuite.test.run_suite()
>   File "D:\src\llvm\tools\lldb\packages\Python\lldbsuite\test\dotest.py",
> line 1476, in run_suite
> setupTestResults()
>   File "D:\src\llvm\tools\lldb\packages\Python\lldbsuite\test\dotest.py",
> line 982, in setupTestResults
> results_formatter_object.handle_event(initialize_event)
>   File
> "D:\src\llvm\tools\lldb\packages\Python\lldbsuite\test\test_results.py",
> line 1033, in handle_event
> "{}#{}".format(len(pickled_message), pickled_message))
> TypeError: a bytes-like object is required, not 'str'
>
> On Wed, Dec 2, 2015 at 11:40 AM Zachary Turner  wrote:
>
>> When I run this under Python 3 I get "A bytes object is used like a
>> string" on Line 1033 of test_results.py.  I'm going to dig into it a little
>> bit, but maybe you know off the top of your head the right way to fix it.
>>
>> On Wed, Dec 2, 2015 at 11:32 AM Zachary Turner 
>> wrote:
>>
>>> Oh yea, I made up that decorator idea because I didn't know all the
>>> formatters were derived from a common base.  But your idea is better if
>>> everything is derived from a common base.  To be honest you could even just
>>> generate an error if there are two ResultsFormatter derived classes in the
>>> same module.  We should be encouraging more smaller files with single
>>> responsibility.  One of the things I plan to do as part of some cleanup in
>>> a week or two is to split up dotest, dosep, and lldbtest.py into a couple
>>> different files by breaking out things like TestBase, etc into separate
>>> files.  So that it's easier to keep a mental map of where different code is.
>>>
>>> On Wed, Dec 2, 2015 at 11:26 AM Todd Fiala  wrote:
>>>
 On Wed, Dec 2, 2015 at 11:20 AM, Todd Fiala 
 wrote:

> Yeah I'd be good with that.  I can change that as well.
>
> -Todd
>
> On Wed, Dec 2, 2015 at 11:10 AM, Zachary Turner 
> wrote:
>
>> Also another stylistic suggestion.  I've been thinking about how to
>> more logically organize all the source files now that we have a package.
>> So it makes sense conceptually to group all of the different result
>> formatters under a subpackage called formatters.  So right now you've got
>> lldbsuite.test.basic_results_formatter.BasicResultsFormatter but it
>> might make sense for this to be
>> lldbsuite.test.formatters.basic.BasicResultsFormatter.  If you do things
>> this way, it can actually result in a substantially shorter command line,
>> because the --results-formatter option can use lldbsuite.test.formatters 
>> as
>> a starting point.  So you could instead write:
>>
>> test/dotest.py --results-formatter basic
>>
>> dotest then looks for a `basic.py` module in the
>> `lldbsuite.test.formatters` package, looks for a class inside with a
>> @result_formatter decorator, and instantiates that.
>>
>> This has the advantage of making the command line shorter *and* a
>> more logical source file organization.
>>
>
 The other thing that could allow me to do is possibly short-circuit the
 results formatter specifier so that, if just the module is specified, and
 if the module only has one ResultsFormatter-derived class, I can probably
 rig up code that figures out the right results formatter, shortening the
 required discriminator to something even shorter (i.e. module.classname
 becomes just module.)


>
>> On Wed, Dec 2, 2015 at 11:04 AM Zachary Turner 
>> wrote:
>>
>>> Can --results-file=stdout be the default so that we don't have to
>>> specify that?
>>>
>>> On Wed, Dec 2, 2015 at 11:02 AM Todd Fiala via lldb-dev <
>>> lldb-dev@lists.ll

Re: [lldb-dev] New test summary results formatter

[Todd Fiala via lldb-dev]

On Wed, Dec 2, 2015 at 11:04 AM, Zachary Turner  wrote:

> Can --results-file=stdout be the default so that we don't have to specify
> that?
>
>
I've adjusted the code here:
r254546

to support dropping the --results-file=stdout part if a --results-formatter
is specified and no results-file is specified.  Good idea, thanks!


> On Wed, Dec 2, 2015 at 11:02 AM Todd Fiala via lldb-dev <
> lldb-dev@lists.llvm.org> wrote:
>
>> Also, all the text in the summary is fixed-width lined up nicely, which
>> may not show in the commit message description if you're using a
>> variable-width font.  On a terminal it looks nice.
>>
>> On Wed, Dec 2, 2015 at 11:01 AM, Todd Fiala  wrote:
>>
>>>
>>>
>>> On Wed, Dec 2, 2015 at 10:57 AM, Todd Fiala 
>>> wrote:
>>>
 Hi all,

 I just put up an optional test results formatter that is a prototype of
 what we may move towards for our default test summary results.  It went in
 here:

 r254530

 and you can try it out with something like:

 time test/dotest.py --executable `pwd`/build/Debug/lldb
 --results-formatter
 lldbsuite.test.basic_results_formatter.BasicResultsFormatter --results-file
 st
 out


>>> I cut and paste my line, but more than likely for most people you'd just
>>> want this:
>>>
>>> test/dotest.py --results-formatter
>>> lldbsuite.test.basic_results_formatter.BasicResultsFormatter --results-file
>>> stdout
>>>
>>> The other stuff was specific to my setup.  That line assumes you run
>>> from the lldb source dir root.
>>>
>>>
>>> Let me know if this satisfies the basic needs of counts and whatnot.  It
 counts test method runs rather than all the oddball "file, class, etc."
 counts we had before.

 It prints out the Details section when there are details, and keeps it
 nice and clean when there are none.

 It also mentions a bit about test reruns up top, but that won't come
 into play until I get the multi-test-pass, single-worker/low-load mechanism
 in place, which will depend on newer rerun count awareness support.

 The change also cleans up places where the test event framework was
 using string codes and replaces them with symbolic constants.

 Let me know what you think.  I can tweak it as needed to address
 testbot and other needs.  Once it looks reasonable, I'd like to move over
 to using it by default in the parallel test runner rather than the legacy
 support.

 Thanks!
 --
 -Todd

>>>
>>>
>>>
>>> --
>>> -Todd
>>>
>>
>>
>>
>> --
>> -Todd
>> ___
>> lldb-dev mailing list
>> lldb-dev@lists.llvm.org
>> http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev
>>
>


-- 
-Todd
___
lldb-dev mailing list
lldb-dev@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev

Re: [lldb-dev] New test summary results formatter

[Todd Fiala via lldb-dev]

I think I know how to fix.  Trying now.

On Wed, Dec 2, 2015 at 1:17 PM, Todd Fiala  wrote:

> I think I can fix the issue without you debugging.
>
> Getting the single pass test runner to use it isn't impossible but will
> take some work.  Can you direct-send me the backtrace from the point of
> failure from your system?  Thanks!
>
> -Todd
>
> On Wed, Dec 2, 2015 at 12:34 PM, Zachary Turner 
> wrote:
>
>> Is there any way to force the single process test runner to use this same
>> system?  I'm trying to debug the problem, but this codepath doesn't execute
>> in the single process test runner, and it executes in the child process in
>> the multiprocess test runner.  Basically I need the following callstack to
>> execute in the single process test runner:
>>
>> Command invoked: C:\Python35\python_d.exe
>> D:\src\llvm\tools\lldb\test\dotest.py -q --arch=i686 --executable
>> D:/src/llvmbuild/ninja_py35/bin/lldb.exe -s
>> D:/src/llvmbuild/ninja_py35/lldb-test-traces -u CXXFLAGS -u CFLAGS
>> --enable-crash-dialog -C d:\src\llvmbuild\ninja_release\bin\clang.exe
>> --results-port 60794 --inferior -p TestIntegerTypesExpr.py
>> D:\src\llvm\tools\lldb\packages\Python\lldbsuite\test --event-add-entries
>> worker_index=7:int
>> 411 out of 412 test suites processed - TestIntegerTypesExpr.py
>> Traceback (most recent call last):
>>   File "D:\src\llvm\tools\lldb\test\dotest.py", line 7, in 
>> lldbsuite.test.run_suite()
>>   File "D:\src\llvm\tools\lldb\packages\Python\lldbsuite\test\dotest.py",
>> line 1476, in run_suite
>> setupTestResults()
>>   File "D:\src\llvm\tools\lldb\packages\Python\lldbsuite\test\dotest.py",
>> line 982, in setupTestResults
>> results_formatter_object.handle_event(initialize_event)
>>   File
>> "D:\src\llvm\tools\lldb\packages\Python\lldbsuite\test\test_results.py",
>> line 1033, in handle_event
>> "{}#{}".format(len(pickled_message), pickled_message))
>> TypeError: a bytes-like object is required, not 'str'
>>
>> On Wed, Dec 2, 2015 at 11:40 AM Zachary Turner 
>> wrote:
>>
>>> When I run this under Python 3 I get "A bytes object is used like a
>>> string" on Line 1033 of test_results.py.  I'm going to dig into it a little
>>> bit, but maybe you know off the top of your head the right way to fix it.
>>>
>>> On Wed, Dec 2, 2015 at 11:32 AM Zachary Turner 
>>> wrote:
>>>
 Oh yea, I made up that decorator idea because I didn't know all the
 formatters were derived from a common base.  But your idea is better if
 everything is derived from a common base.  To be honest you could even just
 generate an error if there are two ResultsFormatter derived classes in the
 same module.  We should be encouraging more smaller files with single
 responsibility.  One of the things I plan to do as part of some cleanup in
 a week or two is to split up dotest, dosep, and lldbtest.py into a couple
 different files by breaking out things like TestBase, etc into separate
 files.  So that it's easier to keep a mental map of where different code 
 is.

 On Wed, Dec 2, 2015 at 11:26 AM Todd Fiala 
 wrote:

> On Wed, Dec 2, 2015 at 11:20 AM, Todd Fiala 
> wrote:
>
>> Yeah I'd be good with that.  I can change that as well.
>>
>> -Todd
>>
>> On Wed, Dec 2, 2015 at 11:10 AM, Zachary Turner 
>> wrote:
>>
>>> Also another stylistic suggestion.  I've been thinking about how to
>>> more logically organize all the source files now that we have a package.
>>> So it makes sense conceptually to group all of the different result
>>> formatters under a subpackage called formatters.  So right now you've 
>>> got
>>> lldbsuite.test.basic_results_formatter.BasicResultsFormatter but it
>>> might make sense for this to be
>>> lldbsuite.test.formatters.basic.BasicResultsFormatter.  If you do things
>>> this way, it can actually result in a substantially shorter command 
>>> line,
>>> because the --results-formatter option can use 
>>> lldbsuite.test.formatters as
>>> a starting point.  So you could instead write:
>>>
>>> test/dotest.py --results-formatter basic
>>>
>>> dotest then looks for a `basic.py` module in the
>>> `lldbsuite.test.formatters` package, looks for a class inside with a
>>> @result_formatter decorator, and instantiates that.
>>>
>>> This has the advantage of making the command line shorter *and* a
>>> more logical source file organization.
>>>
>>
> The other thing that could allow me to do is possibly short-circuit
> the results formatter specifier so that, if just the module is specified,
> and if the module only has one ResultsFormatter-derived class, I can
> probably rig up code that figures out the right results formatter,
> shortening the required discriminator to something even shorter (i.e.
> module.classname becomes just module.)
>
>
>>
>>> On Wed, Dec 2, 2015 at 11:04

Re: [lldb-dev] New test summary results formatter

[Zachary Turner via lldb-dev]

Yea I was messing around with it too.  I don't have a fix yet but I think
you will need to either encode the pickled data as utf8, or better yet,
don't write this:

"{}#{}".format(...)

because pickled data is supposed to be binary data anyway.  So use
bytes.append() instead.

Then on the other side in dotest_channels, there's a couple places where
you do something like:

self.ibuffer = ""

which would need to change to

self.ibuffer = b""

and any other similar operations on self.ibuffer which assume string data.

On Wed, Dec 2, 2015 at 1:33 PM Todd Fiala  wrote:

> I think I know how to fix.  Trying now.
>
> On Wed, Dec 2, 2015 at 1:17 PM, Todd Fiala  wrote:
>
>> I think I can fix the issue without you debugging.
>>
>> Getting the single pass test runner to use it isn't impossible but will
>> take some work.  Can you direct-send me the backtrace from the point of
>> failure from your system?  Thanks!
>>
>> -Todd
>>
>> On Wed, Dec 2, 2015 at 12:34 PM, Zachary Turner 
>> wrote:
>>
>>> Is there any way to force the single process test runner to use this
>>> same system?  I'm trying to debug the problem, but this codepath doesn't
>>> execute in the single process test runner, and it executes in the child
>>> process in the multiprocess test runner.  Basically I need the following
>>> callstack to execute in the single process test runner:
>>>
>>> Command invoked: C:\Python35\python_d.exe
>>> D:\src\llvm\tools\lldb\test\dotest.py -q --arch=i686 --executable
>>> D:/src/llvmbuild/ninja_py35/bin/lldb.exe -s
>>> D:/src/llvmbuild/ninja_py35/lldb-test-traces -u CXXFLAGS -u CFLAGS
>>> --enable-crash-dialog -C d:\src\llvmbuild\ninja_release\bin\clang.exe
>>> --results-port 60794 --inferior -p TestIntegerTypesExpr.py
>>> D:\src\llvm\tools\lldb\packages\Python\lldbsuite\test --event-add-entries
>>> worker_index=7:int
>>> 411 out of 412 test suites processed - TestIntegerTypesExpr.py
>>> Traceback (most recent call last):
>>>   File "D:\src\llvm\tools\lldb\test\dotest.py", line 7, in 
>>> lldbsuite.test.run_suite()
>>>   File
>>> "D:\src\llvm\tools\lldb\packages\Python\lldbsuite\test\dotest.py", line
>>> 1476, in run_suite
>>> setupTestResults()
>>>   File
>>> "D:\src\llvm\tools\lldb\packages\Python\lldbsuite\test\dotest.py", line
>>> 982, in setupTestResults
>>> results_formatter_object.handle_event(initialize_event)
>>>   File
>>> "D:\src\llvm\tools\lldb\packages\Python\lldbsuite\test\test_results.py",
>>> line 1033, in handle_event
>>> "{}#{}".format(len(pickled_message), pickled_message))
>>> TypeError: a bytes-like object is required, not 'str'
>>>
>>> On Wed, Dec 2, 2015 at 11:40 AM Zachary Turner 
>>> wrote:
>>>
 When I run this under Python 3 I get "A bytes object is used like a
 string" on Line 1033 of test_results.py.  I'm going to dig into it a little
 bit, but maybe you know off the top of your head the right way to fix it.

 On Wed, Dec 2, 2015 at 11:32 AM Zachary Turner 
 wrote:

> Oh yea, I made up that decorator idea because I didn't know all the
> formatters were derived from a common base.  But your idea is better if
> everything is derived from a common base.  To be honest you could even 
> just
> generate an error if there are two ResultsFormatter derived classes in the
> same module.  We should be encouraging more smaller files with single
> responsibility.  One of the things I plan to do as part of some cleanup in
> a week or two is to split up dotest, dosep, and lldbtest.py into a couple
> different files by breaking out things like TestBase, etc into separate
> files.  So that it's easier to keep a mental map of where different code 
> is.
>
> On Wed, Dec 2, 2015 at 11:26 AM Todd Fiala 
> wrote:
>
>> On Wed, Dec 2, 2015 at 11:20 AM, Todd Fiala 
>> wrote:
>>
>>> Yeah I'd be good with that.  I can change that as well.
>>>
>>> -Todd
>>>
>>> On Wed, Dec 2, 2015 at 11:10 AM, Zachary Turner 
>>> wrote:
>>>
 Also another stylistic suggestion.  I've been thinking about how to
 more logically organize all the source files now that we have a 
 package.
 So it makes sense conceptually to group all of the different result
 formatters under a subpackage called formatters.  So right now you've 
 got
 lldbsuite.test.basic_results_formatter.BasicResultsFormatter but
 it might make sense for this to be
 lldbsuite.test.formatters.basic.BasicResultsFormatter.  If you do 
 things
 this way, it can actually result in a substantially shorter command 
 line,
 because the --results-formatter option can use 
 lldbsuite.test.formatters as
 a starting point.  So you could instead write:

 test/dotest.py --results-formatter basic

 dotest then looks for a `basic.py` module in the
 `lldbsuite.test.formatters` package, looks fo

Re: [lldb-dev] New test summary results formatter

[Todd Fiala via lldb-dev]

Can you try this?

diff --git a/packages/Python/lldbsuite/test/test_results.py
b/packages/Python/lldbsuite/test/test_results.py
index 12fb2e5..547603e 100644
--- a/packages/Python/lldbsuite/test/test_results.py
+++ b/packages/Python/lldbsuite/test/test_results.py
@@ -1029,8 +1029,8 @@ class RawPickledFormatter(ResultsFormatter):

 # Send it as
{serialized_length_of_serialized_bytes}#{serialized_bytes}
 pickled_message = cPickle.dumps(test_event)
-self.out_file.send(
-"{}#{}".format(len(pickled_message), pickled_message))
+self.out_file.send("{}#".format(len(pickled_message)))
+self.out_file.send(pickled_message)

On Wed, Dec 2, 2015 at 1:36 PM, Zachary Turner  wrote:

> Yea I was messing around with it too.  I don't have a fix yet but I think
> you will need to either encode the pickled data as utf8, or better yet,
> don't write this:
>
> "{}#{}".format(...)
>
> because pickled data is supposed to be binary data anyway.  So use
> bytes.append() instead.
>
> Then on the other side in dotest_channels, there's a couple places where
> you do something like:
>
> self.ibuffer = ""
>
> which would need to change to
>
> self.ibuffer = b""
>
> and any other similar operations on self.ibuffer which assume string data.
>
> On Wed, Dec 2, 2015 at 1:33 PM Todd Fiala  wrote:
>
>> I think I know how to fix.  Trying now.
>>
>> On Wed, Dec 2, 2015 at 1:17 PM, Todd Fiala  wrote:
>>
>>> I think I can fix the issue without you debugging.
>>>
>>> Getting the single pass test runner to use it isn't impossible but will
>>> take some work.  Can you direct-send me the backtrace from the point of
>>> failure from your system?  Thanks!
>>>
>>> -Todd
>>>
>>> On Wed, Dec 2, 2015 at 12:34 PM, Zachary Turner 
>>> wrote:
>>>
 Is there any way to force the single process test runner to use this
 same system?  I'm trying to debug the problem, but this codepath doesn't
 execute in the single process test runner, and it executes in the child
 process in the multiprocess test runner.  Basically I need the following
 callstack to execute in the single process test runner:

 Command invoked: C:\Python35\python_d.exe
 D:\src\llvm\tools\lldb\test\dotest.py -q --arch=i686 --executable
 D:/src/llvmbuild/ninja_py35/bin/lldb.exe -s
 D:/src/llvmbuild/ninja_py35/lldb-test-traces -u CXXFLAGS -u CFLAGS
 --enable-crash-dialog -C d:\src\llvmbuild\ninja_release\bin\clang.exe
 --results-port 60794 --inferior -p TestIntegerTypesExpr.py
 D:\src\llvm\tools\lldb\packages\Python\lldbsuite\test --event-add-entries
 worker_index=7:int
 411 out of 412 test suites processed - TestIntegerTypesExpr.py
 Traceback (most recent call last):
   File "D:\src\llvm\tools\lldb\test\dotest.py", line 7, in 
 lldbsuite.test.run_suite()
   File
 "D:\src\llvm\tools\lldb\packages\Python\lldbsuite\test\dotest.py", line
 1476, in run_suite
 setupTestResults()
   File
 "D:\src\llvm\tools\lldb\packages\Python\lldbsuite\test\dotest.py", line
 982, in setupTestResults
 results_formatter_object.handle_event(initialize_event)
   File
 "D:\src\llvm\tools\lldb\packages\Python\lldbsuite\test\test_results.py",
 line 1033, in handle_event
 "{}#{}".format(len(pickled_message), pickled_message))
 TypeError: a bytes-like object is required, not 'str'

 On Wed, Dec 2, 2015 at 11:40 AM Zachary Turner 
 wrote:

> When I run this under Python 3 I get "A bytes object is used like a
> string" on Line 1033 of test_results.py.  I'm going to dig into it a 
> little
> bit, but maybe you know off the top of your head the right way to fix it.
>
> On Wed, Dec 2, 2015 at 11:32 AM Zachary Turner 
> wrote:
>
>> Oh yea, I made up that decorator idea because I didn't know all the
>> formatters were derived from a common base.  But your idea is better if
>> everything is derived from a common base.  To be honest you could even 
>> just
>> generate an error if there are two ResultsFormatter derived classes in 
>> the
>> same module.  We should be encouraging more smaller files with single
>> responsibility.  One of the things I plan to do as part of some cleanup 
>> in
>> a week or two is to split up dotest, dosep, and lldbtest.py into a couple
>> different files by breaking out things like TestBase, etc into separate
>> files.  So that it's easier to keep a mental map of where different code 
>> is.
>>
>> On Wed, Dec 2, 2015 at 11:26 AM Todd Fiala 
>> wrote:
>>
>>> On Wed, Dec 2, 2015 at 11:20 AM, Todd Fiala 
>>> wrote:
>>>
 Yeah I'd be good with that.  I can change that as well.

 -Todd

 On Wed, Dec 2, 2015 at 11:10 AM, Zachary Turner >>> > wrote:

> Also another stylistic suggestion.  I've been thinking about how
> to more logically o

Re: [lldb-dev] New test summary results formatter

[Todd Fiala via lldb-dev]

Can you try making those changes in the other spots?  There's a handful of
code you have probably not ever run if you haven't selected running with a
test results formatter.

If not, I can try to address them tonight or tomorrow night when I can get
to some kind of python 3 setup.

On Wed, Dec 2, 2015 at 1:36 PM, Zachary Turner  wrote:

> Yea I was messing around with it too.  I don't have a fix yet but I think
> you will need to either encode the pickled data as utf8, or better yet,
> don't write this:
>
> "{}#{}".format(...)
>
> because pickled data is supposed to be binary data anyway.  So use
> bytes.append() instead.
>
> Then on the other side in dotest_channels, there's a couple places where
> you do something like:
>
> self.ibuffer = ""
>
> which would need to change to
>
> self.ibuffer = b""
>
> and any other similar operations on self.ibuffer which assume string data.
>
> On Wed, Dec 2, 2015 at 1:33 PM Todd Fiala  wrote:
>
>> I think I know how to fix.  Trying now.
>>
>> On Wed, Dec 2, 2015 at 1:17 PM, Todd Fiala  wrote:
>>
>>> I think I can fix the issue without you debugging.
>>>
>>> Getting the single pass test runner to use it isn't impossible but will
>>> take some work.  Can you direct-send me the backtrace from the point of
>>> failure from your system?  Thanks!
>>>
>>> -Todd
>>>
>>> On Wed, Dec 2, 2015 at 12:34 PM, Zachary Turner 
>>> wrote:
>>>
 Is there any way to force the single process test runner to use this
 same system?  I'm trying to debug the problem, but this codepath doesn't
 execute in the single process test runner, and it executes in the child
 process in the multiprocess test runner.  Basically I need the following
 callstack to execute in the single process test runner:

 Command invoked: C:\Python35\python_d.exe
 D:\src\llvm\tools\lldb\test\dotest.py -q --arch=i686 --executable
 D:/src/llvmbuild/ninja_py35/bin/lldb.exe -s
 D:/src/llvmbuild/ninja_py35/lldb-test-traces -u CXXFLAGS -u CFLAGS
 --enable-crash-dialog -C d:\src\llvmbuild\ninja_release\bin\clang.exe
 --results-port 60794 --inferior -p TestIntegerTypesExpr.py
 D:\src\llvm\tools\lldb\packages\Python\lldbsuite\test --event-add-entries
 worker_index=7:int
 411 out of 412 test suites processed - TestIntegerTypesExpr.py
 Traceback (most recent call last):
   File "D:\src\llvm\tools\lldb\test\dotest.py", line 7, in 
 lldbsuite.test.run_suite()
   File
 "D:\src\llvm\tools\lldb\packages\Python\lldbsuite\test\dotest.py", line
 1476, in run_suite
 setupTestResults()
   File
 "D:\src\llvm\tools\lldb\packages\Python\lldbsuite\test\dotest.py", line
 982, in setupTestResults
 results_formatter_object.handle_event(initialize_event)
   File
 "D:\src\llvm\tools\lldb\packages\Python\lldbsuite\test\test_results.py",
 line 1033, in handle_event
 "{}#{}".format(len(pickled_message), pickled_message))
 TypeError: a bytes-like object is required, not 'str'

 On Wed, Dec 2, 2015 at 11:40 AM Zachary Turner 
 wrote:

> When I run this under Python 3 I get "A bytes object is used like a
> string" on Line 1033 of test_results.py.  I'm going to dig into it a 
> little
> bit, but maybe you know off the top of your head the right way to fix it.
>
> On Wed, Dec 2, 2015 at 11:32 AM Zachary Turner 
> wrote:
>
>> Oh yea, I made up that decorator idea because I didn't know all the
>> formatters were derived from a common base.  But your idea is better if
>> everything is derived from a common base.  To be honest you could even 
>> just
>> generate an error if there are two ResultsFormatter derived classes in 
>> the
>> same module.  We should be encouraging more smaller files with single
>> responsibility.  One of the things I plan to do as part of some cleanup 
>> in
>> a week or two is to split up dotest, dosep, and lldbtest.py into a couple
>> different files by breaking out things like TestBase, etc into separate
>> files.  So that it's easier to keep a mental map of where different code 
>> is.
>>
>> On Wed, Dec 2, 2015 at 11:26 AM Todd Fiala 
>> wrote:
>>
>>> On Wed, Dec 2, 2015 at 11:20 AM, Todd Fiala 
>>> wrote:
>>>
 Yeah I'd be good with that.  I can change that as well.

 -Todd

 On Wed, Dec 2, 2015 at 11:10 AM, Zachary Turner >>> > wrote:

> Also another stylistic suggestion.  I've been thinking about how
> to more logically organize all the source files now that we have a
> package.  So it makes sense conceptually to group all of the different
> result formatters under a subpackage called formatters.  So right now
> you've got lldbsuite.test.basic_results_formatter.
> BasicResultsFormatter but it might make sense for this to be
> lldbsuite.test.formatters.basic.BasicRe

Re: [lldb-dev] New test summary results formatter

[Todd Fiala via lldb-dev]

On Wed, Dec 2, 2015 at 1:42 PM, Todd Fiala  wrote:

> Can you try making those changes in the other spots?  There's a handful of
> code you have probably not ever run if you haven't selected running with a
> test results formatter.
>
>
I'm actually going to make the ibuffer ones now since it's easier for me to
verify that it doesn't break the non-Windows side since I'm right in there
now.  If that doesn't do it, we'll need to see what else doesn't work.


> If not, I can try to address them tonight or tomorrow night when I can get
> to some kind of python 3 setup.
>
> On Wed, Dec 2, 2015 at 1:36 PM, Zachary Turner  wrote:
>
>> Yea I was messing around with it too.  I don't have a fix yet but I think
>> you will need to either encode the pickled data as utf8, or better yet,
>> don't write this:
>>
>> "{}#{}".format(...)
>>
>> because pickled data is supposed to be binary data anyway.  So use
>> bytes.append() instead.
>>
>> Then on the other side in dotest_channels, there's a couple places where
>> you do something like:
>>
>> self.ibuffer = ""
>>
>> which would need to change to
>>
>> self.ibuffer = b""
>>
>> and any other similar operations on self.ibuffer which assume string data.
>>
>> On Wed, Dec 2, 2015 at 1:33 PM Todd Fiala  wrote:
>>
>>> I think I know how to fix.  Trying now.
>>>
>>> On Wed, Dec 2, 2015 at 1:17 PM, Todd Fiala  wrote:
>>>
 I think I can fix the issue without you debugging.

 Getting the single pass test runner to use it isn't impossible but will
 take some work.  Can you direct-send me the backtrace from the point of
 failure from your system?  Thanks!

 -Todd

 On Wed, Dec 2, 2015 at 12:34 PM, Zachary Turner 
 wrote:

> Is there any way to force the single process test runner to use this
> same system?  I'm trying to debug the problem, but this codepath doesn't
> execute in the single process test runner, and it executes in the child
> process in the multiprocess test runner.  Basically I need the following
> callstack to execute in the single process test runner:
>
> Command invoked: C:\Python35\python_d.exe
> D:\src\llvm\tools\lldb\test\dotest.py -q --arch=i686 --executable
> D:/src/llvmbuild/ninja_py35/bin/lldb.exe -s
> D:/src/llvmbuild/ninja_py35/lldb-test-traces -u CXXFLAGS -u CFLAGS
> --enable-crash-dialog -C d:\src\llvmbuild\ninja_release\bin\clang.exe
> --results-port 60794 --inferior -p TestIntegerTypesExpr.py
> D:\src\llvm\tools\lldb\packages\Python\lldbsuite\test --event-add-entries
> worker_index=7:int
> 411 out of 412 test suites processed - TestIntegerTypesExpr.py
> Traceback (most recent call last):
>   File "D:\src\llvm\tools\lldb\test\dotest.py", line 7, in 
> lldbsuite.test.run_suite()
>   File
> "D:\src\llvm\tools\lldb\packages\Python\lldbsuite\test\dotest.py", line
> 1476, in run_suite
> setupTestResults()
>   File
> "D:\src\llvm\tools\lldb\packages\Python\lldbsuite\test\dotest.py", line
> 982, in setupTestResults
> results_formatter_object.handle_event(initialize_event)
>   File
> "D:\src\llvm\tools\lldb\packages\Python\lldbsuite\test\test_results.py",
> line 1033, in handle_event
> "{}#{}".format(len(pickled_message), pickled_message))
> TypeError: a bytes-like object is required, not 'str'
>
> On Wed, Dec 2, 2015 at 11:40 AM Zachary Turner 
> wrote:
>
>> When I run this under Python 3 I get "A bytes object is used like a
>> string" on Line 1033 of test_results.py.  I'm going to dig into it a 
>> little
>> bit, but maybe you know off the top of your head the right way to fix it.
>>
>> On Wed, Dec 2, 2015 at 11:32 AM Zachary Turner 
>> wrote:
>>
>>> Oh yea, I made up that decorator idea because I didn't know all the
>>> formatters were derived from a common base.  But your idea is better if
>>> everything is derived from a common base.  To be honest you could even 
>>> just
>>> generate an error if there are two ResultsFormatter derived classes in 
>>> the
>>> same module.  We should be encouraging more smaller files with single
>>> responsibility.  One of the things I plan to do as part of some cleanup 
>>> in
>>> a week or two is to split up dotest, dosep, and lldbtest.py into a 
>>> couple
>>> different files by breaking out things like TestBase, etc into separate
>>> files.  So that it's easier to keep a mental map of where different 
>>> code is.
>>>
>>> On Wed, Dec 2, 2015 at 11:26 AM Todd Fiala 
>>> wrote:
>>>
 On Wed, Dec 2, 2015 at 11:20 AM, Todd Fiala 
 wrote:

> Yeah I'd be good with that.  I can change that as well.
>
> -Todd
>
> On Wed, Dec 2, 2015 at 11:10 AM, Zachary Turner <
> ztur...@google.com> wrote:
>
>> Also another stylistic suggestion.  I've been thinkin

Re: [lldb-dev] New test summary results formatter

[Todd Fiala via lldb-dev]

Those fixes are up here:
r254550

Let me know what you see after that, Zachary.

On Wed, Dec 2, 2015 at 1:45 PM, Todd Fiala  wrote:

>
>
> On Wed, Dec 2, 2015 at 1:42 PM, Todd Fiala  wrote:
>
>> Can you try making those changes in the other spots?  There's a handful
>> of code you have probably not ever run if you haven't selected running with
>> a test results formatter.
>>
>>
> I'm actually going to make the ibuffer ones now since it's easier for me
> to verify that it doesn't break the non-Windows side since I'm right in
> there now.  If that doesn't do it, we'll need to see what else doesn't work.
>
>
>> If not, I can try to address them tonight or tomorrow night when I can
>> get to some kind of python 3 setup.
>>
>> On Wed, Dec 2, 2015 at 1:36 PM, Zachary Turner 
>> wrote:
>>
>>> Yea I was messing around with it too.  I don't have a fix yet but I
>>> think you will need to either encode the pickled data as utf8, or better
>>> yet, don't write this:
>>>
>>> "{}#{}".format(...)
>>>
>>> because pickled data is supposed to be binary data anyway.  So use
>>> bytes.append() instead.
>>>
>>> Then on the other side in dotest_channels, there's a couple places where
>>> you do something like:
>>>
>>> self.ibuffer = ""
>>>
>>> which would need to change to
>>>
>>> self.ibuffer = b""
>>>
>>> and any other similar operations on self.ibuffer which assume string
>>> data.
>>>
>>> On Wed, Dec 2, 2015 at 1:33 PM Todd Fiala  wrote:
>>>
 I think I know how to fix.  Trying now.

 On Wed, Dec 2, 2015 at 1:17 PM, Todd Fiala 
 wrote:

> I think I can fix the issue without you debugging.
>
> Getting the single pass test runner to use it isn't impossible but
> will take some work.  Can you direct-send me the backtrace from the point
> of failure from your system?  Thanks!
>
> -Todd
>
> On Wed, Dec 2, 2015 at 12:34 PM, Zachary Turner 
> wrote:
>
>> Is there any way to force the single process test runner to use this
>> same system?  I'm trying to debug the problem, but this codepath doesn't
>> execute in the single process test runner, and it executes in the child
>> process in the multiprocess test runner.  Basically I need the following
>> callstack to execute in the single process test runner:
>>
>> Command invoked: C:\Python35\python_d.exe
>> D:\src\llvm\tools\lldb\test\dotest.py -q --arch=i686 --executable
>> D:/src/llvmbuild/ninja_py35/bin/lldb.exe -s
>> D:/src/llvmbuild/ninja_py35/lldb-test-traces -u CXXFLAGS -u CFLAGS
>> --enable-crash-dialog -C d:\src\llvmbuild\ninja_release\bin\clang.exe
>> --results-port 60794 --inferior -p TestIntegerTypesExpr.py
>> D:\src\llvm\tools\lldb\packages\Python\lldbsuite\test --event-add-entries
>> worker_index=7:int
>> 411 out of 412 test suites processed - TestIntegerTypesExpr.py
>> Traceback (most recent call last):
>>   File "D:\src\llvm\tools\lldb\test\dotest.py", line 7, in 
>> lldbsuite.test.run_suite()
>>   File
>> "D:\src\llvm\tools\lldb\packages\Python\lldbsuite\test\dotest.py", line
>> 1476, in run_suite
>> setupTestResults()
>>   File
>> "D:\src\llvm\tools\lldb\packages\Python\lldbsuite\test\dotest.py", line
>> 982, in setupTestResults
>> results_formatter_object.handle_event(initialize_event)
>>   File
>> "D:\src\llvm\tools\lldb\packages\Python\lldbsuite\test\test_results.py",
>> line 1033, in handle_event
>> "{}#{}".format(len(pickled_message), pickled_message))
>> TypeError: a bytes-like object is required, not 'str'
>>
>> On Wed, Dec 2, 2015 at 11:40 AM Zachary Turner 
>> wrote:
>>
>>> When I run this under Python 3 I get "A bytes object is used like a
>>> string" on Line 1033 of test_results.py.  I'm going to dig into it a 
>>> little
>>> bit, but maybe you know off the top of your head the right way to fix 
>>> it.
>>>
>>> On Wed, Dec 2, 2015 at 11:32 AM Zachary Turner 
>>> wrote:
>>>
 Oh yea, I made up that decorator idea because I didn't know all the
 formatters were derived from a common base.  But your idea is better if
 everything is derived from a common base.  To be honest you could even 
 just
 generate an error if there are two ResultsFormatter derived classes in 
 the
 same module.  We should be encouraging more smaller files with single
 responsibility.  One of the things I plan to do as part of some 
 cleanup in
 a week or two is to split up dotest, dosep, and lldbtest.py into a 
 couple
 different files by breaking out things like TestBase, etc into separate
 files.  So that it's easier to keep a mental map of where different 
 code is.

 On Wed, Dec 2, 2015 at 11:26 AM Todd Fiala 
 wrote:

> On Wed, Dec 2, 2015 at 11:20 AM, Todd Fiala 
> wrote:
>>>

Re: [lldb-dev] New test summary results formatter

[Zachary Turner via lldb-dev]

I think the best solution is going to be to use struct.pack.  Like this on
the writing side:

import struct
msg = cPickle.dumps(test_event)
packet = struct.pack("!I%ds" % len(msg), len(msg), msg)
self.out_file.send(packet)

and like this on the reading side:

self.packet_bytes_remaining = struct.unpack("!I",
self.header_contents)[0]
self.header_contents = b""
self.reading_header = False
return data[(index+1):]

The current solution still doesn't work because it's calling socket.send()
with a str instead of a bytes.  So it has to be bytes all the way through.

On Wed, Dec 2, 2015 at 1:48 PM Todd Fiala  wrote:

> Those fixes are up here:
> r254550
>
> Let me know what you see after that, Zachary.
>
> On Wed, Dec 2, 2015 at 1:45 PM, Todd Fiala  wrote:
>
>>
>>
>> On Wed, Dec 2, 2015 at 1:42 PM, Todd Fiala  wrote:
>>
>>> Can you try making those changes in the other spots?  There's a handful
>>> of code you have probably not ever run if you haven't selected running with
>>> a test results formatter.
>>>
>>>
>> I'm actually going to make the ibuffer ones now since it's easier for me
>> to verify that it doesn't break the non-Windows side since I'm right in
>> there now.  If that doesn't do it, we'll need to see what else doesn't work.
>>
>>
>>> If not, I can try to address them tonight or tomorrow night when I can
>>> get to some kind of python 3 setup.
>>>
>>> On Wed, Dec 2, 2015 at 1:36 PM, Zachary Turner 
>>> wrote:
>>>
 Yea I was messing around with it too.  I don't have a fix yet but I
 think you will need to either encode the pickled data as utf8, or better
 yet, don't write this:

 "{}#{}".format(...)

 because pickled data is supposed to be binary data anyway.  So use
 bytes.append() instead.

 Then on the other side in dotest_channels, there's a couple places
 where you do something like:

 self.ibuffer = ""

 which would need to change to

 self.ibuffer = b""

 and any other similar operations on self.ibuffer which assume string
 data.

 On Wed, Dec 2, 2015 at 1:33 PM Todd Fiala  wrote:

> I think I know how to fix.  Trying now.
>
> On Wed, Dec 2, 2015 at 1:17 PM, Todd Fiala 
> wrote:
>
>> I think I can fix the issue without you debugging.
>>
>> Getting the single pass test runner to use it isn't impossible but
>> will take some work.  Can you direct-send me the backtrace from the point
>> of failure from your system?  Thanks!
>>
>> -Todd
>>
>> On Wed, Dec 2, 2015 at 12:34 PM, Zachary Turner 
>> wrote:
>>
>>> Is there any way to force the single process test runner to use this
>>> same system?  I'm trying to debug the problem, but this codepath doesn't
>>> execute in the single process test runner, and it executes in the child
>>> process in the multiprocess test runner.  Basically I need the following
>>> callstack to execute in the single process test runner:
>>>
>>> Command invoked: C:\Python35\python_d.exe
>>> D:\src\llvm\tools\lldb\test\dotest.py -q --arch=i686 --executable
>>> D:/src/llvmbuild/ninja_py35/bin/lldb.exe -s
>>> D:/src/llvmbuild/ninja_py35/lldb-test-traces -u CXXFLAGS -u CFLAGS
>>> --enable-crash-dialog -C d:\src\llvmbuild\ninja_release\bin\clang.exe
>>> --results-port 60794 --inferior -p TestIntegerTypesExpr.py
>>> D:\src\llvm\tools\lldb\packages\Python\lldbsuite\test 
>>> --event-add-entries
>>> worker_index=7:int
>>> 411 out of 412 test suites processed - TestIntegerTypesExpr.py
>>> Traceback (most recent call last):
>>>   File "D:\src\llvm\tools\lldb\test\dotest.py", line 7, in 
>>> lldbsuite.test.run_suite()
>>>   File
>>> "D:\src\llvm\tools\lldb\packages\Python\lldbsuite\test\dotest.py", line
>>> 1476, in run_suite
>>> setupTestResults()
>>>   File
>>> "D:\src\llvm\tools\lldb\packages\Python\lldbsuite\test\dotest.py", line
>>> 982, in setupTestResults
>>> results_formatter_object.handle_event(initialize_event)
>>>   File
>>> "D:\src\llvm\tools\lldb\packages\Python\lldbsuite\test\test_results.py",
>>> line 1033, in handle_event
>>> "{}#{}".format(len(pickled_message), pickled_message))
>>> TypeError: a bytes-like object is required, not 'str'
>>>
>>> On Wed, Dec 2, 2015 at 11:40 AM Zachary Turner 
>>> wrote:
>>>
 When I run this under Python 3 I get "A bytes object is used like a
 string" on Line 1033 of test_results.py.  I'm going to dig into it a 
 little
 bit, but maybe you know off the top of your head the right way to fix 
 it.

 On Wed, Dec 2, 2015 at 11:32 AM Zachary Turner 
 wrote:

> Oh yea, I made up that decorator idea because I didn't know all
> the formatters were derived from

Re: [lldb-dev] New test summary results formatter

[Zachary Turner via lldb-dev]

BTW, this is what I do in the swig service, and I think it's the idiomatic
way of sending variable length data over a socket.

On Wed, Dec 2, 2015 at 2:08 PM Zachary Turner  wrote:

> I think the best solution is going to be to use struct.pack.  Like this on
> the writing side:
>
> import struct
> msg = cPickle.dumps(test_event)
> packet = struct.pack("!I%ds" % len(msg), len(msg), msg)
> self.out_file.send(packet)
>
> and like this on the reading side:
>
> self.packet_bytes_remaining = struct.unpack("!I",
> self.header_contents)[0]
> self.header_contents = b""
> self.reading_header = False
> return data[(index+1):]
>
> The current solution still doesn't work because it's calling socket.send()
> with a str instead of a bytes.  So it has to be bytes all the way through.
>
> On Wed, Dec 2, 2015 at 1:48 PM Todd Fiala  wrote:
>
>> Those fixes are up here:
>> r254550
>>
>> Let me know what you see after that, Zachary.
>>
>> On Wed, Dec 2, 2015 at 1:45 PM, Todd Fiala  wrote:
>>
>>>
>>>
>>> On Wed, Dec 2, 2015 at 1:42 PM, Todd Fiala  wrote:
>>>
 Can you try making those changes in the other spots?  There's a handful
 of code you have probably not ever run if you haven't selected running with
 a test results formatter.


>>> I'm actually going to make the ibuffer ones now since it's easier for me
>>> to verify that it doesn't break the non-Windows side since I'm right in
>>> there now.  If that doesn't do it, we'll need to see what else doesn't work.
>>>
>>>
 If not, I can try to address them tonight or tomorrow night when I can
 get to some kind of python 3 setup.

 On Wed, Dec 2, 2015 at 1:36 PM, Zachary Turner 
 wrote:

> Yea I was messing around with it too.  I don't have a fix yet but I
> think you will need to either encode the pickled data as utf8, or better
> yet, don't write this:
>
> "{}#{}".format(...)
>
> because pickled data is supposed to be binary data anyway.  So use
> bytes.append() instead.
>
> Then on the other side in dotest_channels, there's a couple places
> where you do something like:
>
> self.ibuffer = ""
>
> which would need to change to
>
> self.ibuffer = b""
>
> and any other similar operations on self.ibuffer which assume string
> data.
>
> On Wed, Dec 2, 2015 at 1:33 PM Todd Fiala 
> wrote:
>
>> I think I know how to fix.  Trying now.
>>
>> On Wed, Dec 2, 2015 at 1:17 PM, Todd Fiala 
>> wrote:
>>
>>> I think I can fix the issue without you debugging.
>>>
>>> Getting the single pass test runner to use it isn't impossible but
>>> will take some work.  Can you direct-send me the backtrace from the 
>>> point
>>> of failure from your system?  Thanks!
>>>
>>> -Todd
>>>
>>> On Wed, Dec 2, 2015 at 12:34 PM, Zachary Turner 
>>> wrote:
>>>
 Is there any way to force the single process test runner to use
 this same system?  I'm trying to debug the problem, but this codepath
 doesn't execute in the single process test runner, and it executes in 
 the
 child process in the multiprocess test runner.  Basically I need the
 following callstack to execute in the single process test runner:

 Command invoked: C:\Python35\python_d.exe
 D:\src\llvm\tools\lldb\test\dotest.py -q --arch=i686 --executable
 D:/src/llvmbuild/ninja_py35/bin/lldb.exe -s
 D:/src/llvmbuild/ninja_py35/lldb-test-traces -u CXXFLAGS -u CFLAGS
 --enable-crash-dialog -C d:\src\llvmbuild\ninja_release\bin\clang.exe
 --results-port 60794 --inferior -p TestIntegerTypesExpr.py
 D:\src\llvm\tools\lldb\packages\Python\lldbsuite\test 
 --event-add-entries
 worker_index=7:int
 411 out of 412 test suites processed - TestIntegerTypesExpr.py
 Traceback (most recent call last):
   File "D:\src\llvm\tools\lldb\test\dotest.py", line 7, in 
 lldbsuite.test.run_suite()
   File
 "D:\src\llvm\tools\lldb\packages\Python\lldbsuite\test\dotest.py", line
 1476, in run_suite
 setupTestResults()
   File
 "D:\src\llvm\tools\lldb\packages\Python\lldbsuite\test\dotest.py", line
 982, in setupTestResults
 results_formatter_object.handle_event(initialize_event)
   File
 "D:\src\llvm\tools\lldb\packages\Python\lldbsuite\test\test_results.py",
 line 1033, in handle_event
 "{}#{}".format(len(pickled_message), pickled_message))
 TypeError: a bytes-like object is required, not 'str'

 On Wed, Dec 2, 2015 at 11:40 AM Zachary Turner 
 wrote:

> When I run this under Python 3 I get "A bytes object is used like
> a string" on Line 1033 of test_results.py

Re: [lldb-dev] New test summary results formatter

[Zachary Turner via lldb-dev]

 think I got something working, I'll upload a patch in a few.

On Wed, Dec 2, 2015 at 2:10 PM Zachary Turner  wrote:

> BTW, this is what I do in the swig service, and I think it's the idiomatic
> way of sending variable length data over a socket.
>
> On Wed, Dec 2, 2015 at 2:08 PM Zachary Turner  wrote:
>
>> I think the best solution is going to be to use struct.pack.  Like this
>> on the writing side:
>>
>> import struct
>> msg = cPickle.dumps(test_event)
>> packet = struct.pack("!I%ds" % len(msg), len(msg), msg)
>> self.out_file.send(packet)
>>
>> and like this on the reading side:
>>
>> self.packet_bytes_remaining = struct.unpack("!I",
>> self.header_contents)[0]
>> self.header_contents = b""
>> self.reading_header = False
>> return data[(index+1):]
>>
>> The current solution still doesn't work because it's calling
>> socket.send() with a str instead of a bytes.  So it has to be bytes all the
>> way through.
>>
>> On Wed, Dec 2, 2015 at 1:48 PM Todd Fiala  wrote:
>>
>>> Those fixes are up here:
>>> r254550
>>>
>>> Let me know what you see after that, Zachary.
>>>
>>> On Wed, Dec 2, 2015 at 1:45 PM, Todd Fiala  wrote:
>>>


 On Wed, Dec 2, 2015 at 1:42 PM, Todd Fiala 
 wrote:

> Can you try making those changes in the other spots?  There's a
> handful of code you have probably not ever run if you haven't selected
> running with a test results formatter.
>
>
 I'm actually going to make the ibuffer ones now since it's easier for
 me to verify that it doesn't break the non-Windows side since I'm right in
 there now.  If that doesn't do it, we'll need to see what else doesn't 
 work.


> If not, I can try to address them tonight or tomorrow night when I can
> get to some kind of python 3 setup.
>
> On Wed, Dec 2, 2015 at 1:36 PM, Zachary Turner 
> wrote:
>
>> Yea I was messing around with it too.  I don't have a fix yet but I
>> think you will need to either encode the pickled data as utf8, or better
>> yet, don't write this:
>>
>> "{}#{}".format(...)
>>
>> because pickled data is supposed to be binary data anyway.  So use
>> bytes.append() instead.
>>
>> Then on the other side in dotest_channels, there's a couple places
>> where you do something like:
>>
>> self.ibuffer = ""
>>
>> which would need to change to
>>
>> self.ibuffer = b""
>>
>> and any other similar operations on self.ibuffer which assume string
>> data.
>>
>> On Wed, Dec 2, 2015 at 1:33 PM Todd Fiala 
>> wrote:
>>
>>> I think I know how to fix.  Trying now.
>>>
>>> On Wed, Dec 2, 2015 at 1:17 PM, Todd Fiala 
>>> wrote:
>>>
 I think I can fix the issue without you debugging.

 Getting the single pass test runner to use it isn't impossible but
 will take some work.  Can you direct-send me the backtrace from the 
 point
 of failure from your system?  Thanks!

 -Todd

 On Wed, Dec 2, 2015 at 12:34 PM, Zachary Turner >>> > wrote:

> Is there any way to force the single process test runner to use
> this same system?  I'm trying to debug the problem, but this codepath
> doesn't execute in the single process test runner, and it executes in 
> the
> child process in the multiprocess test runner.  Basically I need the
> following callstack to execute in the single process test runner:
>
> Command invoked: C:\Python35\python_d.exe
> D:\src\llvm\tools\lldb\test\dotest.py -q --arch=i686 --executable
> D:/src/llvmbuild/ninja_py35/bin/lldb.exe -s
> D:/src/llvmbuild/ninja_py35/lldb-test-traces -u CXXFLAGS -u CFLAGS
> --enable-crash-dialog -C d:\src\llvmbuild\ninja_release\bin\clang.exe
> --results-port 60794 --inferior -p TestIntegerTypesExpr.py
> D:\src\llvm\tools\lldb\packages\Python\lldbsuite\test 
> --event-add-entries
> worker_index=7:int
> 411 out of 412 test suites processed - TestIntegerTypesExpr.py
> Traceback (most recent call last):
>   File "D:\src\llvm\tools\lldb\test\dotest.py", line 7, in 
> lldbsuite.test.run_suite()
>   File
> "D:\src\llvm\tools\lldb\packages\Python\lldbsuite\test\dotest.py", 
> line
> 1476, in run_suite
> setupTestResults()
>   File
> "D:\src\llvm\tools\lldb\packages\Python\lldbsuite\test\dotest.py", 
> line
> 982, in setupTestResults
> results_formatter_object.handle_event(initialize_event)
>   File
> "D:\src\llvm\tools\lldb\packages\Python\lldbsuite\test\test_results.py",
> line 1033, in handle_event
> "{}#{}".format(len(pickled_message), pickled_message)

[lldb-dev] Running lldb tests?

[Chuck Ries via lldb-dev]

I am trying to run the lldb tests with dotest.py but it is failing with
chuckr-mac-build:test chuckr$ python ./dotest.py
Traceback (most recent call last):
  File "./dotest.py", line 30, in 
import progress
ImportError: No module named progress

It looks like some work was done to move to python 3 and I am using Python 
2.7.10. The info at http://lldb.llvm.org/test.html does not seem to be up to 
date. Is there any fix to get the test script working?

ChuckR
___
lldb-dev mailing list
lldb-dev@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev

Re: [lldb-dev] Running lldb tests?

[Zachary Turner via lldb-dev]

Couple things:

1) Python 2.7.10 is still the officially supported way to run tests for all
platforms
2) Python 3 is only supported for Windows and it's experimental.
3) At some point I will drop support for 2.x on Windows, but 2.x will
always be the officially supported way to run tests on every other platform.

Are you running lldb/test/dotest.py or
lldb/packages/lldbsuite/test/dotest.py?  You should be running the former.
The `progress` module is located in
lldb/third_party/Python/module/progress.  That should be in your `sys.path`
because when you run `lldb/test/dotest.py` the first line calls `import
use_lldb_suite`, which modifies `sys.path` to contain this and a few other
directories.

Hopefully that gives you a starting point to diagnose what's wrong.

On Wed, Dec 2, 2015 at 2:35 PM Chuck Ries via lldb-dev <
lldb-dev@lists.llvm.org> wrote:

> I am trying to run the lldb tests with dotest.py but it is failing with
>
> chuckr-mac-build:test chuckr$ python ./dotest.py
>
> Traceback (most recent call last):
>
>   File "./dotest.py", line 30, in 
>
> import progress
>
> ImportError: No module named progress
>
>
>
> It looks like some work was done to move to python 3 and I am using Python
> 2.7.10. The info at http://lldb.llvm.org/test.html does not seem to be up
> to date. Is there any fix to get the test script working?
>
>
>
> ChuckR
> ___
> lldb-dev mailing list
> lldb-dev@lists.llvm.org
> http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev
>
___
lldb-dev mailing list
lldb-dev@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev

Re: [lldb-dev] Running lldb tests?

[Chuck Ries via lldb-dev]

I was confused because all the tests moved to the packages directory. Do I need 
to run the dotest.py from the lldb/test/ directory but point it to the 
individual tests within the packages directory?

From: Zachary Turner [mailto:ztur...@google.com]
Sent: Wednesday, December 2, 2015 2:41 PM
To: Chuck Ries ; lldb-dev@lists.llvm.org
Subject: Re: [lldb-dev] Running lldb tests?

Couple things:

1) Python 2.7.10 is still the officially supported way to run tests for all 
platforms
2) Python 3 is only supported for Windows and it's experimental.
3) At some point I will drop support for 2.x on Windows, but 2.x will always be 
the officially supported way to run tests on every other platform.

Are you running lldb/test/dotest.py or lldb/packages/lldbsuite/test/dotest.py?  
You should be running the former.  The `progress` module is located in 
lldb/third_party/Python/module/progress.  That should be in your `sys.path` 
because when you run `lldb/test/dotest.py` the first line calls `import 
use_lldb_suite`, which modifies `sys.path` to contain this and a few other 
directories.

Hopefully that gives you a starting point to diagnose what's wrong.

On Wed, Dec 2, 2015 at 2:35 PM Chuck Ries via lldb-dev 
mailto:lldb-dev@lists.llvm.org>> wrote:
I am trying to run the lldb tests with dotest.py but it is failing with
chuckr-mac-build:test chuckr$ python ./dotest.py
Traceback (most recent call last):
  File "./dotest.py", line 30, in 
import progress
ImportError: No module named progress

It looks like some work was done to move to python 3 and I am using Python 
2.7.10. The info at 
http://lldb.llvm.org/test.html
 does not seem to be up to date. Is there any fix to get the test script 
working?

ChuckR
___
lldb-dev mailing list
lldb-dev@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev
___
lldb-dev mailing list
lldb-dev@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev

Re: [lldb-dev] Running lldb tests?

[Zachary Turner via lldb-dev]

You shouldn't have to point it anywhere.  Just run lldb/test/dotest.py the
same way you're currently running it and it should work if you use the
right script.  That use_lldb_suite also adds the location of the test
folder to sys.path so it can find it.

The reason all this happened and everything was moved was to make a "real"
python package out of the core of the test suite, which allows lots of
functionality to be more easily reused both within the test suite itself,
as well as within other scripts that are not part of the test suite.

On Wed, Dec 2, 2015 at 2:43 PM Chuck Ries  wrote:

> I was confused because all the tests moved to the packages directory. Do I
> need to run the dotest.py from the lldb/test/ directory but point it to the
> individual tests within the packages directory?
>
>
>
> *From:* Zachary Turner [mailto:ztur...@google.com]
> *Sent:* Wednesday, December 2, 2015 2:41 PM
> *To:* Chuck Ries ; lldb-dev@lists.llvm.org
> *Subject:* Re: [lldb-dev] Running lldb tests?
>
>
>
> Couple things:
>
>
>
> 1) Python 2.7.10 is still the officially supported way to run tests for
> all platforms
>
> 2) Python 3 is only supported for Windows and it's experimental.
>
> 3) At some point I will drop support for 2.x on Windows, but 2.x will
> always be the officially supported way to run tests on every other platform.
>
>
>
> Are you running lldb/test/dotest.py or
> lldb/packages/lldbsuite/test/dotest.py?  You should be running the former.
> The `progress` module is located in
> lldb/third_party/Python/module/progress.  That should be in your `sys.path`
> because when you run `lldb/test/dotest.py` the first line calls `import
> use_lldb_suite`, which modifies `sys.path` to contain this and a few other
> directories.
>
>
>
> Hopefully that gives you a starting point to diagnose what's wrong.
>
>
>
> On Wed, Dec 2, 2015 at 2:35 PM Chuck Ries via lldb-dev <
> lldb-dev@lists.llvm.org> wrote:
>
> I am trying to run the lldb tests with dotest.py but it is failing with
>
> chuckr-mac-build:test chuckr$ python ./dotest.py
>
> Traceback (most recent call last):
>
>   File "./dotest.py", line 30, in 
>
> import progress
>
> ImportError: No module named progress
>
>
>
> It looks like some work was done to move to python 3 and I am using Python
> 2.7.10. The info at http://lldb.llvm.org/test.html
> 
> does not seem to be up to date. Is there any fix to get the test script
> working?
>
>
>
> ChuckR
>
> ___
> lldb-dev mailing list
> lldb-dev@lists.llvm.org
> http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev
> 
>
>
___
lldb-dev mailing list
lldb-dev@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev

Re: [lldb-dev] New test summary results formatter

[Todd Fiala via lldb-dev]

All good.

Yeah Python 2/3 string/byte handling is one of the trickier areas of
cross-Python version compatibility.

On Wed, Dec 2, 2015 at 2:18 PM, Zachary Turner  wrote:

>  think I got something working, I'll upload a patch in a few.
>
> On Wed, Dec 2, 2015 at 2:10 PM Zachary Turner  wrote:
>
>> BTW, this is what I do in the swig service, and I think it's the
>> idiomatic way of sending variable length data over a socket.
>>
>> On Wed, Dec 2, 2015 at 2:08 PM Zachary Turner  wrote:
>>
>>> I think the best solution is going to be to use struct.pack.  Like this
>>> on the writing side:
>>>
>>> import struct
>>> msg = cPickle.dumps(test_event)
>>> packet = struct.pack("!I%ds" % len(msg), len(msg), msg)
>>> self.out_file.send(packet)
>>>
>>> and like this on the reading side:
>>>
>>> self.packet_bytes_remaining = struct.unpack("!I",
>>> self.header_contents)[0]
>>> self.header_contents = b""
>>> self.reading_header = False
>>> return data[(index+1):]
>>>
>>> The current solution still doesn't work because it's calling
>>> socket.send() with a str instead of a bytes.  So it has to be bytes all the
>>> way through.
>>>
>>> On Wed, Dec 2, 2015 at 1:48 PM Todd Fiala  wrote:
>>>
 Those fixes are up here:
 r254550

 Let me know what you see after that, Zachary.

 On Wed, Dec 2, 2015 at 1:45 PM, Todd Fiala 
 wrote:

>
>
> On Wed, Dec 2, 2015 at 1:42 PM, Todd Fiala 
> wrote:
>
>> Can you try making those changes in the other spots?  There's a
>> handful of code you have probably not ever run if you haven't selected
>> running with a test results formatter.
>>
>>
> I'm actually going to make the ibuffer ones now since it's easier for
> me to verify that it doesn't break the non-Windows side since I'm right in
> there now.  If that doesn't do it, we'll need to see what else doesn't 
> work.
>
>
>> If not, I can try to address them tonight or tomorrow night when I
>> can get to some kind of python 3 setup.
>>
>> On Wed, Dec 2, 2015 at 1:36 PM, Zachary Turner 
>> wrote:
>>
>>> Yea I was messing around with it too.  I don't have a fix yet but I
>>> think you will need to either encode the pickled data as utf8, or better
>>> yet, don't write this:
>>>
>>> "{}#{}".format(...)
>>>
>>> because pickled data is supposed to be binary data anyway.  So use
>>> bytes.append() instead.
>>>
>>> Then on the other side in dotest_channels, there's a couple places
>>> where you do something like:
>>>
>>> self.ibuffer = ""
>>>
>>> which would need to change to
>>>
>>> self.ibuffer = b""
>>>
>>> and any other similar operations on self.ibuffer which assume string
>>> data.
>>>
>>> On Wed, Dec 2, 2015 at 1:33 PM Todd Fiala 
>>> wrote:
>>>
 I think I know how to fix.  Trying now.

 On Wed, Dec 2, 2015 at 1:17 PM, Todd Fiala 
 wrote:

> I think I can fix the issue without you debugging.
>
> Getting the single pass test runner to use it isn't impossible but
> will take some work.  Can you direct-send me the backtrace from the 
> point
> of failure from your system?  Thanks!
>
> -Todd
>
> On Wed, Dec 2, 2015 at 12:34 PM, Zachary Turner <
> ztur...@google.com> wrote:
>
>> Is there any way to force the single process test runner to use
>> this same system?  I'm trying to debug the problem, but this codepath
>> doesn't execute in the single process test runner, and it executes 
>> in the
>> child process in the multiprocess test runner.  Basically I need the
>> following callstack to execute in the single process test runner:
>>
>> Command invoked: C:\Python35\python_d.exe
>> D:\src\llvm\tools\lldb\test\dotest.py -q --arch=i686 --executable
>> D:/src/llvmbuild/ninja_py35/bin/lldb.exe -s
>> D:/src/llvmbuild/ninja_py35/lldb-test-traces -u CXXFLAGS -u CFLAGS
>> --enable-crash-dialog -C d:\src\llvmbuild\ninja_release\bin\clang.exe
>> --results-port 60794 --inferior -p TestIntegerTypesExpr.py
>> D:\src\llvm\tools\lldb\packages\Python\lldbsuite\test 
>> --event-add-entries
>> worker_index=7:int
>> 411 out of 412 test suites processed - TestIntegerTypesExpr.py
>> Traceback (most recent call last):
>>   File "D:\src\llvm\tools\lldb\test\dotest.py", line 7, in
>> 
>> lldbsuite.test.run_suite()
>>   File
>> "D:\src\llvm\tools\lldb\packages\Python\lldbsuite\test\dotest.py", 
>> line
>> 1476, in run_suite
>> setupTestResults()
>>   File
>> "D:\src\llvm\tools\lldb\packages\Python\lldbsuite\te

Re: [lldb-dev] Running lldb tests?

[Jim Ingham via lldb-dev]

Should we change the name of lldb/packages/Python/lldbsuite/dotest.py to 
something else to avoid this confusion?

Jim

> On Dec 2, 2015, at 2:48 PM, Zachary Turner via lldb-dev 
>  wrote:
> 
> You shouldn't have to point it anywhere.  Just run lldb/test/dotest.py the 
> same way you're currently running it and it should work if you use the right 
> script.  That use_lldb_suite also adds the location of the test folder to 
> sys.path so it can find it.
> 
> The reason all this happened and everything was moved was to make a "real" 
> python package out of the core of the test suite, which allows lots of 
> functionality to be more easily reused both within the test suite itself, as 
> well as within other scripts that are not part of the test suite.
> 
> On Wed, Dec 2, 2015 at 2:43 PM Chuck Ries  wrote:
> I was confused because all the tests moved to the packages directory. Do I 
> need to run the dotest.py from the lldb/test/ directory but point it to the 
> individual tests within the packages directory?
> 
>  
> 
> From: Zachary Turner [mailto:ztur...@google.com] 
> Sent: Wednesday, December 2, 2015 2:41 PM
> To: Chuck Ries ; lldb-dev@lists.llvm.org
> Subject: Re: [lldb-dev] Running lldb tests?
> 
>  
> 
> Couple things:
> 
>  
> 
> 1) Python 2.7.10 is still the officially supported way to run tests for all 
> platforms
> 
> 2) Python 3 is only supported for Windows and it's experimental.  
> 
> 3) At some point I will drop support for 2.x on Windows, but 2.x will always 
> be the officially supported way to run tests on every other platform.
> 
>  
> 
> Are you running lldb/test/dotest.py or 
> lldb/packages/lldbsuite/test/dotest.py?  You should be running the former.  
> The `progress` module is located in lldb/third_party/Python/module/progress.  
> That should be in your `sys.path` because when you run `lldb/test/dotest.py` 
> the first line calls `import use_lldb_suite`, which modifies `sys.path` to 
> contain this and a few other directories.
> 
>  
> 
> Hopefully that gives you a starting point to diagnose what's wrong.
> 
>  
> 
> On Wed, Dec 2, 2015 at 2:35 PM Chuck Ries via lldb-dev 
>  wrote:
> 
> I am trying to run the lldb tests with dotest.py but it is failing with
> 
> chuckr-mac-build:test chuckr$ python ./dotest.py
> 
> Traceback (most recent call last):
> 
>   File "./dotest.py", line 30, in 
> 
> import progress
> 
> ImportError: No module named progress
> 
>  
> 
> It looks like some work was done to move to python 3 and I am using Python 
> 2.7.10. The info at http://lldb.llvm.org/test.html does not seem to be up to 
> date. Is there any fix to get the test script working?
> 
>  
> 
> ChuckR
> 
> ___
> lldb-dev mailing list
> lldb-dev@lists.llvm.org
> http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev
> 
> ___
> lldb-dev mailing list
> lldb-dev@lists.llvm.org
> http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev

___
lldb-dev mailing list
lldb-dev@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev

Re: [lldb-dev] Running lldb tests?

[Zachary Turner via lldb-dev]

Yea.  And I should also make the if __name__ == "__main__" check actually
work by putting it above the import statements.  It's supposed to print out
a message and tell you what to do when you accidentally run that, but it
doesn't get that far because it fails to import a module.

On Wed, Dec 2, 2015 at 3:15 PM Jim Ingham  wrote:

> Should we change the name of lldb/packages/Python/lldbsuite/dotest.py to
> something else to avoid this confusion?
>
> Jim
>
> > On Dec 2, 2015, at 2:48 PM, Zachary Turner via lldb-dev <
> lldb-dev@lists.llvm.org> wrote:
> >
> > You shouldn't have to point it anywhere.  Just run lldb/test/dotest.py
> the same way you're currently running it and it should work if you use the
> right script.  That use_lldb_suite also adds the location of the test
> folder to sys.path so it can find it.
> >
> > The reason all this happened and everything was moved was to make a
> "real" python package out of the core of the test suite, which allows lots
> of functionality to be more easily reused both within the test suite
> itself, as well as within other scripts that are not part of the test suite.
> >
> > On Wed, Dec 2, 2015 at 2:43 PM Chuck Ries 
> wrote:
> > I was confused because all the tests moved to the packages directory. Do
> I need to run the dotest.py from the lldb/test/ directory but point it to
> the individual tests within the packages directory?
> >
> >
> >
> > From: Zachary Turner [mailto:ztur...@google.com]
> > Sent: Wednesday, December 2, 2015 2:41 PM
> > To: Chuck Ries ; lldb-dev@lists.llvm.org
> > Subject: Re: [lldb-dev] Running lldb tests?
> >
> >
> >
> > Couple things:
> >
> >
> >
> > 1) Python 2.7.10 is still the officially supported way to run tests for
> all platforms
> >
> > 2) Python 3 is only supported for Windows and it's experimental.
> >
> > 3) At some point I will drop support for 2.x on Windows, but 2.x will
> always be the officially supported way to run tests on every other platform.
> >
> >
> >
> > Are you running lldb/test/dotest.py or
> lldb/packages/lldbsuite/test/dotest.py?  You should be running the former.
> The `progress` module is located in
> lldb/third_party/Python/module/progress.  That should be in your `sys.path`
> because when you run `lldb/test/dotest.py` the first line calls `import
> use_lldb_suite`, which modifies `sys.path` to contain this and a few other
> directories.
> >
> >
> >
> > Hopefully that gives you a starting point to diagnose what's wrong.
> >
> >
> >
> > On Wed, Dec 2, 2015 at 2:35 PM Chuck Ries via lldb-dev <
> lldb-dev@lists.llvm.org> wrote:
> >
> > I am trying to run the lldb tests with dotest.py but it is failing with
> >
> > chuckr-mac-build:test chuckr$ python ./dotest.py
> >
> > Traceback (most recent call last):
> >
> >   File "./dotest.py", line 30, in 
> >
> > import progress
> >
> > ImportError: No module named progress
> >
> >
> >
> > It looks like some work was done to move to python 3 and I am using
> Python 2.7.10. The info at http://lldb.llvm.org/test.html does not seem
> to be up to date. Is there any fix to get the test script working?
> >
> >
> >
> > ChuckR
> >
> > ___
> > lldb-dev mailing list
> > lldb-dev@lists.llvm.org
> > http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev
> >
> > ___
> > lldb-dev mailing list
> > lldb-dev@lists.llvm.org
> > http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev
>
>
___
lldb-dev mailing list
lldb-dev@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev

Re: [lldb-dev] New test summary results formatter

[Zachary Turner via lldb-dev]

Now that it's working in Python 3 and I've had a chance to see the output,
I think I like it much better.  These are the things I noticed (good and
bad):

1) Output of each test is printed on the fly as the test is run.  Much
better than staring at a black screen for 5 minutes and then seeing a bunch
of junk fly by.
2) Summary output is organized by test function, not by file.  This gives
better counts.
3) Behavior seems different somehow.  My successes / failures / unexpected
successes seem different when using this formatter.  I'll have to look into
this to figure out if it's just my imagination or something's actually
wrong.

Will this abstraction of a result formatter provide some way to make a
formatter intended for machine consumption?  Right now lots of people
scrape logs for the purposes of build bots, and it would be better if we
could just dump the results as json or something.

Only suggestion I have at this point is that in the summary printout the
lines are too long.  Long test name (module.classname.function) and long
filename.  Perhaps the filename could be converted to a relative path and
the classname could be dropped (there's only one class per file anyway, so
the classname is just wasted space)

On Wed, Dec 2, 2015 at 3:06 PM Todd Fiala  wrote:

> All good.
>
> Yeah Python 2/3 string/byte handling is one of the trickier areas of
> cross-Python version compatibility.
>
> On Wed, Dec 2, 2015 at 2:18 PM, Zachary Turner  wrote:
>
>>  think I got something working, I'll upload a patch in a few.
>>
>> On Wed, Dec 2, 2015 at 2:10 PM Zachary Turner  wrote:
>>
>>> BTW, this is what I do in the swig service, and I think it's the
>>> idiomatic way of sending variable length data over a socket.
>>>
>>> On Wed, Dec 2, 2015 at 2:08 PM Zachary Turner 
>>> wrote:
>>>
 I think the best solution is going to be to use struct.pack.  Like this
 on the writing side:

 import struct
 msg = cPickle.dumps(test_event)
 packet = struct.pack("!I%ds" % len(msg), len(msg), msg)
 self.out_file.send(packet)

 and like this on the reading side:

 self.packet_bytes_remaining = struct.unpack("!I",
 self.header_contents)[0]
 self.header_contents = b""
 self.reading_header = False
 return data[(index+1):]

 The current solution still doesn't work because it's calling
 socket.send() with a str instead of a bytes.  So it has to be bytes all the
 way through.

 On Wed, Dec 2, 2015 at 1:48 PM Todd Fiala  wrote:

> Those fixes are up here:
> r254550
>
> Let me know what you see after that, Zachary.
>
> On Wed, Dec 2, 2015 at 1:45 PM, Todd Fiala 
> wrote:
>
>>
>>
>> On Wed, Dec 2, 2015 at 1:42 PM, Todd Fiala 
>> wrote:
>>
>>> Can you try making those changes in the other spots?  There's a
>>> handful of code you have probably not ever run if you haven't selected
>>> running with a test results formatter.
>>>
>>>
>> I'm actually going to make the ibuffer ones now since it's easier for
>> me to verify that it doesn't break the non-Windows side since I'm right 
>> in
>> there now.  If that doesn't do it, we'll need to see what else doesn't 
>> work.
>>
>>
>>> If not, I can try to address them tonight or tomorrow night when I
>>> can get to some kind of python 3 setup.
>>>
>>> On Wed, Dec 2, 2015 at 1:36 PM, Zachary Turner 
>>> wrote:
>>>
 Yea I was messing around with it too.  I don't have a fix yet but I
 think you will need to either encode the pickled data as utf8, or 
 better
 yet, don't write this:

 "{}#{}".format(...)

 because pickled data is supposed to be binary data anyway.  So use
 bytes.append() instead.

 Then on the other side in dotest_channels, there's a couple places
 where you do something like:

 self.ibuffer = ""

 which would need to change to

 self.ibuffer = b""

 and any other similar operations on self.ibuffer which assume
 string data.

 On Wed, Dec 2, 2015 at 1:33 PM Todd Fiala 
 wrote:

> I think I know how to fix.  Trying now.
>
> On Wed, Dec 2, 2015 at 1:17 PM, Todd Fiala 
> wrote:
>
>> I think I can fix the issue without you debugging.
>>
>> Getting the single pass test runner to use it isn't impossible
>> but will take some work.  Can you direct-send me the backtrace from 
>> the
>> point of failure from your system?  Thanks!
>>
>> -Todd
>>
>> On Wed, Dec 2, 2015 at 12:34 PM, Zachary Turner <
>> ztur...@google.com> wrote:
>>
>>> Is there any way to force the si

Re: [lldb-dev] New test summary results formatter

[Zachary Turner via lldb-dev]

In other words, instead of printing this:

FAIL:
TestDataFormatterPythonSynth.PythonSynthDataFormatterTestCase.test_rdar10960550_with_run_command_dwo
(D:\src\llvm\tools\lldb\packages\Python\lldbsuite\test\functionalities\data-formatter\data-formatter-python-synth\TestDataFormatterPythonSynth.py)

Print this:

FAIL: TestDataFormatterPythonSynth.test_rdar10960550_with_run_command_dwo
(functionalities\data-formatter\data-formatter-python-synth\TestDataFormatterPythonSynth.py)

On Wed, Dec 2, 2015 at 3:59 PM Zachary Turner  wrote:

> Now that it's working in Python 3 and I've had a chance to see the output,
> I think I like it much better.  These are the things I noticed (good and
> bad):
>
> 1) Output of each test is printed on the fly as the test is run.  Much
> better than staring at a black screen for 5 minutes and then seeing a bunch
> of junk fly by.
> 2) Summary output is organized by test function, not by file.  This gives
> better counts.
> 3) Behavior seems different somehow.  My successes / failures / unexpected
> successes seem different when using this formatter.  I'll have to look into
> this to figure out if it's just my imagination or something's actually
> wrong.
>
> Will this abstraction of a result formatter provide some way to make a
> formatter intended for machine consumption?  Right now lots of people
> scrape logs for the purposes of build bots, and it would be better if we
> could just dump the results as json or something.
>
> Only suggestion I have at this point is that in the summary printout the
> lines are too long.  Long test name (module.classname.function) and long
> filename.  Perhaps the filename could be converted to a relative path and
> the classname could be dropped (there's only one class per file anyway, so
> the classname is just wasted space)
>
> On Wed, Dec 2, 2015 at 3:06 PM Todd Fiala  wrote:
>
>> All good.
>>
>> Yeah Python 2/3 string/byte handling is one of the trickier areas of
>> cross-Python version compatibility.
>>
>> On Wed, Dec 2, 2015 at 2:18 PM, Zachary Turner 
>> wrote:
>>
>>>  think I got something working, I'll upload a patch in a few.
>>>
>>> On Wed, Dec 2, 2015 at 2:10 PM Zachary Turner 
>>> wrote:
>>>
 BTW, this is what I do in the swig service, and I think it's the
 idiomatic way of sending variable length data over a socket.

 On Wed, Dec 2, 2015 at 2:08 PM Zachary Turner 
 wrote:

> I think the best solution is going to be to use struct.pack.  Like
> this on the writing side:
>
> import struct
> msg = cPickle.dumps(test_event)
> packet = struct.pack("!I%ds" % len(msg), len(msg), msg)
> self.out_file.send(packet)
>
> and like this on the reading side:
>
> self.packet_bytes_remaining = struct.unpack("!I",
> self.header_contents)[0]
> self.header_contents = b""
> self.reading_header = False
> return data[(index+1):]
>
> The current solution still doesn't work because it's calling
> socket.send() with a str instead of a bytes.  So it has to be bytes all 
> the
> way through.
>
> On Wed, Dec 2, 2015 at 1:48 PM Todd Fiala 
> wrote:
>
>> Those fixes are up here:
>> r254550
>>
>> Let me know what you see after that, Zachary.
>>
>> On Wed, Dec 2, 2015 at 1:45 PM, Todd Fiala 
>> wrote:
>>
>>>
>>>
>>> On Wed, Dec 2, 2015 at 1:42 PM, Todd Fiala 
>>> wrote:
>>>
 Can you try making those changes in the other spots?  There's a
 handful of code you have probably not ever run if you haven't selected
 running with a test results formatter.


>>> I'm actually going to make the ibuffer ones now since it's easier
>>> for me to verify that it doesn't break the non-Windows side since I'm 
>>> right
>>> in there now.  If that doesn't do it, we'll need to see what else 
>>> doesn't
>>> work.
>>>
>>>
 If not, I can try to address them tonight or tomorrow night when I
 can get to some kind of python 3 setup.

 On Wed, Dec 2, 2015 at 1:36 PM, Zachary Turner 
 wrote:

> Yea I was messing around with it too.  I don't have a fix yet but
> I think you will need to either encode the pickled data as utf8, or 
> better
> yet, don't write this:
>
> "{}#{}".format(...)
>
> because pickled data is supposed to be binary data anyway.  So use
> bytes.append() instead.
>
> Then on the other side in dotest_channels, there's a couple places
> where you do something like:
>
> self.ibuffer = ""
>
> which would need to change to
>
> self.ibuffer = b""
>
> and any other similar operations on self.ibuffer which assume
> string data.
>
>

Re: [lldb-dev] reply: reply： lldb debug jit-compiled code with llvm on windows

[Oleksiy Vyalov via lldb-dev]

Could you re-build lldb with http://reviews.llvm.org/D15172 and try to hit
a breakpoint in jit code?

I'm using following setup to compile and run JIT code:
./build_release/bin/clang -c -O0 -g -emit-llvm ./jit_test.cpp
./build_release/bin/lli -debugger-tune=lldb -jit-kind=mcjit
~/projects/lldb/test/jit_test.bc

On Mon, Nov 30, 2015 at 3:47 AM, Tamas Berghammer via lldb-dev <
lldb-dev@lists.llvm.org> wrote:

>
>
> On Mon, Nov 30, 2015 at 10:18 AM haifeng_q  wrote:
>
>> Question 1:
>> On the windows, I use the code implements a function (see
>> debug_target.cpp) of JIT (see debug_target_process.cpp), but when
>> generating debug information, no production .symtab section for information
>> that leads LLDB get JIT debugging information .symtab failure , and then
>> set a breakpoint to fail.
>>  LLDB command: lldb_result.txt
>>  JIT compilation results: debug_target.ll
>>
>>  Question 2:
>>  How JIT debugging supported on Linux?
>>
>
> I theory when a new function is JIT-ed then __jit_debug_register_code
> function is called where LLDB have a breakpoint set. When that breakpoint
> is hit then LLDB reads the JIT-ed elf file based on the information in 
> __it_debug_descriptor
> and processes all debug info in it.
>
> In practice when I last tried JIT debugging with lldb and lli (few weeks
> ago) it get the notification for the new JIT-ed elf file but it processed
> only the eh_frame from it even though symtab and full debug info was also
> provided. Most likely there is some issue around the JIT breakpoint
> handling or around the elf file parsing code in LLDB what needs
> some investigation.
>
>>
>> thanks!
>>
>> -- The original message --
>> *From:* "Zachary Turner";;
>> *Data:* 2015年11月21日 AM 0:10
>> *Receive:* "Tamas Berghammer"; " "<
>> haifen...@foxmail.com>; "lldb-dev";
>> *Title:* Re: [lldb-dev] reply： lldb debug jit-compiled code with llvm on
>> windows
>>
>> Can you also try clang-cl and see if it works?
>>
>> On Fri, Nov 20, 2015 at 3:02 AM Tamas Berghammer via lldb-dev <
>> lldb-dev@lists.llvm.org> wrote:
>>
>>> I don't know how JIT debugging should work on WIndows with MSVC but I
>>> don't think LLDB support it in any way. What I wrote should be true on
>>> Linux (and on some related) systems. You might be able to get the same
>>> results on Windows if you use lli (LLVM based JIT runner) but I have no
>>> knowledge if it will work or not.
>>>
>>> On Fri, Nov 20, 2015 at 8:56 AM haifeng_q  wrote:
>>>
 My analysis of the code, the reasons are:

 Since the debugging process is MSVC compiler, there is no DWARF debugging
 information. So lldb get __jit_debug_register_code and
 __it_debug_descriptor symbol debugging process fails, and
 __jit_debug_register_code not support MSVC.
 I do not know whether correct?

 -- original message--
 *From:*"Tamas Berghammer";tbergham...@google.com;
 Date*:*2015年11月19日 PM 8:37
 *receive:* " "; "lldb-dev"<
 lldb-dev@lists.llvm.org>;
 *Subject:* Re: [lldb-dev] lldb debug jit-compiled code with llvm on
 windows

 In theory you don't have to do anything special to debug some JIT-ed
 code as everything should just work (based on the gdb jit interface). In
 practice I tried it out a few days ago and it wasn't working at all even in
 the case when the application is launched under LLDB (not with attach).
 LLDB was understanding the eh_frame for the JIT-ed code but didn't found
 the debug info for unknown reason. We should investigate it and try to fix
 it sometime. We (lldb for android developers) plan to do it sometime but if
 you are interested in it then please feel free to take a look and let us
 know if you have any question.

 Tamas

 On Thu, Nov 19, 2015 at 8:40 AM haifeng_q via lldb-dev <
 lldb-dev@lists.llvm.org> wrote:

> hi,
> process A generate function Func1 code with llvm jit compiler, and calls
> Func1. modeled on "Kaleidoscope: Adding Debug Information" add debug
> information. how to use LLDB to attach process A to debug this
> function, add a breakpoint in the function?
>
> thanks!
> ___
> lldb-dev mailing list
> lldb-dev@lists.llvm.org
> http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev
>
 ___
>>> lldb-dev mailing list
>>> lldb-dev@lists.llvm.org
>>> http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev
>>>
>>
> ___
> lldb-dev mailing list
> lldb-dev@lists.llvm.org
> http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev
>
>


-- 
Oleksiy Vyalov | Software Engineer | ovya...@google.com
___
lldb-dev mailing list
lldb-dev@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev

Re: [lldb-dev] New test summary results formatter

[Todd Fiala via lldb-dev]

Thanks, Zachary.

On Wed, Dec 2, 2015 at 3:59 PM, Zachary Turner  wrote:

> Now that it's working in Python 3 and I've had a chance to see the output,
> I think I like it much better.  These are the things I noticed (good and
> bad):
>
> 1) Output of each test is printed on the fly as the test is run.  Much
> better than staring at a black screen for 5 minutes and then seeing a bunch
> of junk fly by.
> 2) Summary output is organized by test function, not by file.  This gives
> better counts.
> 3) Behavior seems different somehow.  My successes / failures / unexpected
> successes seem different when using this formatter.  I'll have to look into
> this to figure out if it's just my imagination or something's actually
> wrong.
>

Let me know if you see anything wrong.  We've been using the Xunit
formatter with our CI for 3-ish months at this point (which runs on the
test event architecture), and have been happy with the results.  Our CI
cleanup still gets timeouts and exceptional exits from the legacy summary,
but I have bugzilla bugs on routing those through the test event system as
well.  That means ResultFormatter implementations can handle them, which is
ultimately what we want.


>
> Will this abstraction of a result formatter provide some way to make a
> formatter intended for machine consumption?
>

Sure.  What did you have in mind?  Some kind of structured output?  (Right
now the Xunit formatter is exactly that - our CI uses the Xunit output and
ingests it into Jenkins, which has an Xunit-based plugin that can interpret
and display the results quite nicely).


> Right now lots of people scrape logs for the purposes of build bots, and
> it would be better if we could just dump the results as json or something.
>
> Only suggestion I have at this point is that in the summary printout the
> lines are too long.
>

Agreed.


> Long test name (module.classname.function) and long filename.  Perhaps the
> filename could be converted to a relative path
>

This was something I was planning on doing.  I like the idea.


> and the classname could be dropped (there's only one class per file
> anyway, so the classname is just wasted space)
>

Part of the reason I included that is I've hit several times where copy and
paste errors lead to the same class name, method name or even file name
being used for a test.  I think, though, that most of those are addressed
by having the path (relative is fine) to the python test file.  I think we
can probably get by with classname.methodname (relative test path).  (From
your other email, I think you nuke the classname and keep the module name,
but I'd probably do the reverse, keeping the class name and getting rid of
the module name since it can be derived from the filename).

Definitely let me know if you find any issues aside from lack of timeout
and exceptional (POSIX-land crashed process) exits.  As you noted and I
described, the counts are all now test method based.

Feel free to play with the Xunit one as well if you're looking for
something machine readable.  And if you're running a test run on Linux, the
curses-based one gives nice output with some user control over what test
results show in the lower portion of the display.  The upper portion shows
you what each of the workers is doing. --curses is the shortcut to get to
it.

-Todd


> On Wed, Dec 2, 2015 at 3:06 PM Todd Fiala  wrote:
>
>> All good.
>>
>> Yeah Python 2/3 string/byte handling is one of the trickier areas of
>> cross-Python version compatibility.
>>
>> On Wed, Dec 2, 2015 at 2:18 PM, Zachary Turner 
>> wrote:
>>
>>>  think I got something working, I'll upload a patch in a few.
>>>
>>> On Wed, Dec 2, 2015 at 2:10 PM Zachary Turner 
>>> wrote:
>>>
 BTW, this is what I do in the swig service, and I think it's the
 idiomatic way of sending variable length data over a socket.

 On Wed, Dec 2, 2015 at 2:08 PM Zachary Turner 
 wrote:

> I think the best solution is going to be to use struct.pack.  Like
> this on the writing side:
>
> import struct
> msg = cPickle.dumps(test_event)
> packet = struct.pack("!I%ds" % len(msg), len(msg), msg)
> self.out_file.send(packet)
>
> and like this on the reading side:
>
> self.packet_bytes_remaining = struct.unpack("!I",
> self.header_contents)[0]
> self.header_contents = b""
> self.reading_header = False
> return data[(index+1):]
>
> The current solution still doesn't work because it's calling
> socket.send() with a str instead of a bytes.  So it has to be bytes all 
> the
> way through.
>
> On Wed, Dec 2, 2015 at 1:48 PM Todd Fiala 
> wrote:
>
>> Those fixes are up here:
>> r254550
>>
>> Let me know what you see after that, Zachary.
>>
>> On Wed, Dec 2, 2015 at 1:45 PM, Todd Fiala 
>> wrote:
>>
>>>
>>>
>>> On Wed, Dec

Re: [lldb-dev] New test summary results formatter

[Zachary Turner via lldb-dev]

On Wed, Dec 2, 2015 at 9:44 PM Todd Fiala  wrote:

>
>
>> and the classname could be dropped (there's only one class per file
>> anyway, so the classname is just wasted space)
>>
>
> Part of the reason I included that is I've hit several times where copy
> and paste errors lead to the same class name, method name or even file name
> being used for a test.  I think, though, that most of those are addressed
> by having the path (relative is fine) to the python test file.  I think we
> can probably get by with classname.methodname (relative test path).  (From
> your other email, I think you nuke the classname and keep the module name,
> but I'd probably do the reverse, keeping the class name and getting rid of
> the module name since it can be derived from the filename).
>
I don't think the filename can be the same anymore, as things will break if
two filenames are the same.  We could go one step further and enforce this
in the part where it scans for all the tests.  If it finds two test files
with the same name we could just generate an error.  I think that's a good
idea anyway, because if two test files have the same name, then the tests
inside must be similar enough to warrant merging them into the same file.

If no two filenames are the same, and if there's only 1 class per file,
then filename + method name should uniquely identify a single test, and so
you could omit the class name and show a relative path to the filename.

>
___
lldb-dev mailing list
lldb-dev@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev

Re: [lldb-dev] New test summary results formatter

[Todd Fiala via lldb-dev]

On Wed, Dec 2, 2015 at 9:48 PM, Zachary Turner  wrote:

>
>
> On Wed, Dec 2, 2015 at 9:44 PM Todd Fiala  wrote:
>
>>
>>
>>> and the classname could be dropped (there's only one class per file
>>> anyway, so the classname is just wasted space)
>>>
>>
>> Part of the reason I included that is I've hit several times where copy
>> and paste errors lead to the same class name, method name or even file name
>> being used for a test.  I think, though, that most of those are addressed
>> by having the path (relative is fine) to the python test file.  I think we
>> can probably get by with classname.methodname (relative test path).  (From
>> your other email, I think you nuke the classname and keep the module name,
>> but I'd probably do the reverse, keeping the class name and getting rid of
>> the module name since it can be derived from the filename).
>>
> I don't think the filename can be the same anymore, as things will break
> if two filenames are the same.
>

Maybe, but that wasn't my experience as of fairly recently.  When tracking
failures sometime within the last month, I tracked something down in a
downstream branch with two same-named files that (with the legacy output)
made it hard to track down what was actually failing given the limited info
of the legacy test summary output.  Maybe that has changed since then, but
I'm not aware of anything that would have prohibited that.


>   We could go one step further and enforce this in the part where it scans
> for all the tests.
>

I think I can come up with a valid counterargument to doing that.  I could
imagine some python .py files being organized hierarchically, where some of
the context of what is being tested clearly comes from the directory
structure.

Something like (I'm making this up):

lang/c/const/TestConst.py
lang/c++/const/TestConst.py

where it seems totally reasonable to me to have things testing const
support (in this example) but being very different things for C and C++,
being totally uniqued by path rather than the .py file.  I'd prefer not to
require something like this to say:
lang/c/const/TestConstC.py
lang/c++/const/TestConstC++.py

as it is redundant (at least via the path hierarchy).

The other reason I could see avoiding that
unique-test-basenames-across-test-suite restriction is that it can become
somewhat of an unnecessary burden on downstream branches.  Imagine somebody
has a branch and has a test that happens to be running fine, then somebody
in llvm.org lldb adds a test with the same name.  Downstream breaks.  We
could choose to not care about that, but given that a lot of our tests will
revolve around language features accessed/provided by the debugger, and a
number of language features pull out of a limited set of feature names
(e.g. const above), I could see us sometimes hitting this.

Just one take on it.  I'm not particularly wedded to it (I probably would
avoid the confusion by doing something exactly like what I said above with
regards to tacking on the language to the test name), but I have hit this
in similar form across different language tests.


>   If it finds two test files with the same name we could just generate an
> error.  I think that's a good idea anyway, because if two test files have
> the same name, then the tests inside must be similar enough to warrant
> merging them into the same file.
>

Maybe, but not in the real cases I saw across different languages.  I think
for other areas of the debugger, this isn't an issue.  So maybe language
feature tests just have to know to append their language (whether it be C,
C++, ObjC, etc.)


>
> If no two filenames are the same, and if there's only 1 class per file,
> then filename + method name should uniquely identify a single test, and so
> you could omit the class name and show a relative path to the filename.
>
>>
I think we currently have some tests that have multiple test classes in the
test file.  We could certainly verify that in TOT, and we could certainly
undo that which seems reasonable.

I'd be interested in what other people think here on restricting test names
to be unique across the repo.  I could be convinced either way on allowing
two tests with the same name, but I'd probably avoid layering on a
restriction if it is entirely artificial and requires longer test names
that are otherwise uniqued by path.

Thanks for the feedback!
-- 
-Todd
___
lldb-dev mailing list
lldb-dev@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev