Re: [lldb-dev] [RFC]The future of pexpect

2019-02-21 Thread Pavel Labath via lldb-dev

On 21/02/2019 00:03, Davide Italiano wrote:

I found out that there are tests that effectively require
interactivity. Some of the lldb-mi ones are an example.
A common use-case is that of sending SIGTERM in a loop to make sure
`lldb-mi` doesn't crash and handle the signal correctly.

This functionality is really hard to replicate in lit_as is_.
Any ideas on how we could handle this case?


How hard is it to import a new version of pexpect which supports python3 
and stuff?


I'm not sure how the situation is on darwin, but I'd expect (:P) that 
most linux systems either already have it installed, or have an easy way 
to do so. So we may not even be able to get away with just using the 
system one and skipping tests when it's not present.


BTW, for lldb-mi I would actually argue that it should *not* use pexpect 
:D. Interactivity is one thing, and I'm very much in favour of keeping 
that ability, but pexpect is not a prerequisite for that. For me, the 
main advantage of pexpect is that it emulates a real terminal. However, 
lldb-mi does not need that stuff. It doesn't have any command line 
editing capabilities or similar. It's expecting to communicate with an 
IDE over a pipe, and that's it.


Given that, it should be fairly easy to rewrite the lldb-mi tests to 
work on top of the standard python "subprocess" library. While we're 
doing that, we might actually fix some of the issues that have been 
bugging everyone in the lldb-mi tests. At least for me, the most 
annoying thing was that when lldb-mi fails to produce the expected 
output, the test does not fail immediately, but instead the 
implementation of self.expect("^whatever") waits until the timeout 
expires, optimistically hoping that it will find some output that match 
the pattern.


If we change this to something like self.expect_reply("^whatever"), and 
make the "expect_reply" function smart enough to know that lldb-mi's 
response should come as a single line, and if the first line doesn't 
match, it should abort, this problem would be fixed. While we're at it, 
we could also tune the failure message so that it's more helpful than 
the current implementation. Plus, that would solve the issue of not 
being able to run lldb-mi tests on windows.


Anyway, that's what I'd do. I was actually planning to look into that 
soon, but then I roped myself into writing a yaml (de)serialization tool 
for minidumps, so I have no idea when I will get back to that. I hope 
some of this is helpful nonetheless.


cheers,
pavel
___
lldb-dev mailing list
lldb-dev@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev


Re: [lldb-dev] [RFC]The future of pexpect

2019-02-21 Thread Pavel Labath via lldb-dev

On 21/02/2019 15:35, Pavel Labath via lldb-dev wrote:
So we may not even be able to get away with just using the system one 
and skipping tests when it's not present.


s/may not/may/
___
lldb-dev mailing list
lldb-dev@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev


Re: [lldb-dev] [8.0.0 Release] Please write release notes

2019-02-21 Thread Raphael Isemann via lldb-dev
I'm not sure what to do with potential LLDB release notes. Should I
just add a LLDB section to the LLVM release notes or do we actually
have a separate file for LLDB like clang/libcxx/lld do?

- Raphael

Am Di., 19. Feb. 2019 um 16:26 Uhr schrieb Hans Wennborg via lldb-dev
:
>
> Hello everyone,
>
> This is your biannual release notes nag email.
>
> We're getting closer to the end of the release schedule for 8.0.0, but
> the release notes are still very thin for many parts.
>
> When the release happens, the first thing people look at are these
> notes, so it's a great opportunity to mention the work that's been
> done in the past six months.
>
> If you know anything that's worth adding, please commit to the release
> notes directly on the branch, or send me a patch, or even a plain
> email with some text and I'll happily commit it for you.
>
> Thanks,
> Hans
> ___
> lldb-dev mailing list
> lldb-dev@lists.llvm.org
> https://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev
___
lldb-dev mailing list
lldb-dev@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev


Re: [lldb-dev] [8.0.0 Release] Please write release notes

2019-02-21 Thread Hans Wennborg via lldb-dev
I don't think lldb has a separate release notes document. Adding an
lldb section to the llvm ones sounds good to me.

On Thu, Feb 21, 2019 at 5:19 PM Raphael Isemann  wrote:
>
> I'm not sure what to do with potential LLDB release notes. Should I
> just add a LLDB section to the LLVM release notes or do we actually
> have a separate file for LLDB like clang/libcxx/lld do?
>
> - Raphael
>
> Am Di., 19. Feb. 2019 um 16:26 Uhr schrieb Hans Wennborg via lldb-dev
> :
> >
> > Hello everyone,
> >
> > This is your biannual release notes nag email.
> >
> > We're getting closer to the end of the release schedule for 8.0.0, but
> > the release notes are still very thin for many parts.
> >
> > When the release happens, the first thing people look at are these
> > notes, so it's a great opportunity to mention the work that's been
> > done in the past six months.
> >
> > If you know anything that's worth adding, please commit to the release
> > notes directly on the branch, or send me a patch, or even a plain
> > email with some text and I'll happily commit it for you.
> >
> > Thanks,
> > Hans
> > ___
> > lldb-dev mailing list
> > lldb-dev@lists.llvm.org
> > https://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev
___
lldb-dev mailing list
lldb-dev@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev


Re: [lldb-dev] [RFC]The future of pexpect

2019-02-21 Thread Ted Woodward via lldb-dev


> -Original Message-
> From: lldb-dev  On Behalf Of Pavel Labath
> via lldb-dev
> Sent: Thursday, February 21, 2019 8:35 AM
> To: Davide Italiano 
> Cc: LLDB 
> Subject: [EXT] Re: [lldb-dev] [RFC]The future of pexpect
> 
> On 21/02/2019 00:03, Davide Italiano wrote:
> > I found out that there are tests that effectively require
> > interactivity. Some of the lldb-mi ones are an example.
> > A common use-case is that of sending SIGTERM in a loop to make sure
> > `lldb-mi` doesn't crash and handle the signal correctly.
> >
> > This functionality is really hard to replicate in lit_as is_.
> > Any ideas on how we could handle this case?
> 
> How hard is it to import a new version of pexpect which supports python3 and
> stuff?
> 
> I'm not sure how the situation is on darwin, but I'd expect (:P) that most 
> linux
> systems either already have it installed, or have an easy way to do so. So we
> may not even be able to get away with just using the system one and skipping
> tests when it's not present.
> 
> BTW, for lldb-mi I would actually argue that it should *not* use pexpect :D.
> Interactivity is one thing, and I'm very much in favour of keeping that 
> ability,
> but pexpect is not a prerequisite for that. For me, the main advantage of
> pexpect is that it emulates a real terminal. However, lldb-mi does not need
> that stuff. It doesn't have any command line editing capabilities or similar. 
> It's
> expecting to communicate with an IDE over a pipe, and that's it.
> 
> Given that, it should be fairly easy to rewrite the lldb-mi tests to work on 
> top
> of the standard python "subprocess" library. While we're doing that, we might
> actually fix some of the issues that have been bugging everyone in the lldb-mi
> tests. At least for me, the most annoying thing was that when lldb-mi fails to
> produce the expected output, the test does not fail immediately, but instead
> the implementation of self.expect("^whatever") waits until the timeout
> expires, optimistically hoping that it will find some output that match the
> pattern.
> 
> If we change this to something like self.expect_reply("^whatever"), and make
> the "expect_reply" function smart enough to know that lldb-mi's response
> should come as a single line, and if the first line doesn't match, it should 
> abort,
> this problem would be fixed. While we're at it, we could also tune the failure
> message so that it's more helpful than the current implementation. Plus, that
> would solve the issue of not being able to run lldb-mi tests on windows.

This would be OK, I think, as long as "expect_reply" has the option to do a 
partial match,
or a regex match. Some of the lldb-mi tests only look for certain parts of the 
reply.

Also, until Python2 is declared dead and not supported at all by lldb, we 
should be able
to run this under 2 or 3.

> Anyway, that's what I'd do. I was actually planning to look into that soon, 
> but
> then I roped myself into writing a yaml (de)serialization tool for minidumps, 
> so
> I have no idea when I will get back to that. I hope some of this is helpful
> nonetheless.
> 
> cheers,
> pavel
> ___
> lldb-dev mailing list
> lldb-dev@lists.llvm.org
> https://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev
___
lldb-dev mailing list
lldb-dev@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev


Re: [lldb-dev] SB API is not working properly with OSPython plugin

2019-02-21 Thread Alexander Polyakov via lldb-dev
It seems that the process plugin uses the Process::SetPrivateState at the
right time. If you look at the log, you will see that the process is
already in the '*private* *stopped'* state when the OS plugin is invoked.

(lldb) c
lldb Process::Resume -- locking run lock
lldb Process::PrivateResume() m_stop_id = 1, public state:
stopped private state: stopped
lldb Process::SetPrivateState (running)
intern-state Process::ShouldBroadcastEvent (0x1abea90) => new state:
running, last broadcast state: running - YES
intern-state Process::HandlePrivateEvent (pid = 1) broadcasting new
state running (old state stopped) to public
intern-state Process::PushProcessIOHandler pushing IO handler
intern-state Process::HandlePrivateEvent updated m_iohandler_sync to 1
lldb Process thinks the process has resumed.
intern-state timeout = , event_sp)...
lldb waited from m_iohandler_sync to change from 0. New value
is 1.
dbg.evt-handler  Process::SetPublicState (state = running, restarted = 0)
Process 1 resuming
lldb Process::SetPrivateState (stopped)
lldb Process::SetPrivateState (stopped) stop_id = 2
error: error: process must be stopped.
intern-state Process::ShouldBroadcastEvent (0x7f3e9c007450) => new
state: stopped, last broadcast state: stopped - YES
intern-state Process::HandlePrivateEvent (pid = 1) broadcasting new
state stopped (old state running) to public
intern-state timeout = , event_sp)...
dbg.evt-handler  Process::SetPublicState (state = stopped, restarted = 0)
dbg.evt-handler  Process::SetPublicState (stopped) -- unlocking run lock


On Fri, Feb 15, 2019 at 1:38 AM Jim Ingham  wrote:

> Your plugin should have set the private state to stopped when it figures
> out however it does that the process has stopped.  The API is
> Process::SetPrivateState.  Is that happening at the right time?
>
> Jim
>
>
> On Feb 14, 2019, at 1:50 PM, Alexander Polyakov 
> wrote:
>
> I found out that the plugin works well with an x86 application, so I think
> that the problem is in my process plugin. Maybe you know a place where to
> start looking for an issue?
>
> On Thu, Feb 14, 2019 at 10:56 PM Jim Ingham  wrote:
>
>> The simplest thing possible to reproduce the failure.  So some OS_Plugin
>> implementation which tries to look up a global like this and fails, and
>> some program source I can run it under that provides said global.  That way
>> I can easily watch it fails as you describe when the plugin gets activated,
>> and see why it isn’t allowing this call on private stop.
>>
>> Jim
>>
>>
>> On Feb 14, 2019, at 11:50 AM, Alexander Polyakov 
>> wrote:
>>
>> Sure, could you describe in more detail which example may help you?
>>
>> чт, 14 февр. 2019 г. в 22:36, Jim Ingham :
>>
>>> That’s a little complicated…
>>>
>>> lldb has two levels of stop state - private stops and public stops.
>>> When the process gets notification from the underlying process plugin that
>>> the process stopped, it raises a private stop event.  That gets handled by
>>> the ShouldStop mechanism on the private state thread in lldb, and then if
>>> the stop is judged interesting to the user, it gets rebroadcast as a public
>>> stop.
>>>
>>> For instance, when you issue a “step” command, lldb will stop and start
>>> the process multiple times as it walks through the source line.  But only
>>> the last of those stops are relevant to the user of LLDB, so all the other
>>> ones exist only as private stops.
>>>
>>> The SB API’s for the most part should only consider a “publicly stopped”
>>> process accessible.  After all, you wouldn’t want some API to succeed
>>> sometimes if you happen to catch it in the middle of a private stop.
>>>
>>> But the OperatingSystem plugin needs to get called right after a private
>>> stop, so it can provide threads for the ShouldStop mechanism.  We should
>>> really have some formal mechanism whereby things like the OS plugin can
>>> request elevated rights in the SB API’s, so that they can run at private
>>> stop time.  IIRC, we instead have a hack where SB API calls that run on the
>>> private state thread are blanket allowed to run at private stop.  The xnu
>>> Operating System plugin successfully gets global values to look up its
>>> threads.  So I’m not sure why that isn’t working for you.
>>>
>>> Can you cook up a simple example showing the failure and I’ll have a
>>> look?
>>>
>>> Jim
>>>
>>>
>>> On Feb 14, 2019, at 11:10 AM, Alexander Polyakov 
>>> wrote:
>>>
>>> It is, the error is: *error: error: process must be stopped.*
>>>
>>> I thought that the plugin (get_thread_info in my case) is invoked when
>>> the process is already stopped, but it's not. Is it ok?
>>>
>>> On Thu, Feb 14, 2019 at 9:53 PM Jim Ingham  wrote:
>>>
 All SBValues have an error in them (SBValue.GetError).  Does that say
 anything interesting?

 Jim




 On Feb 14, 2019, at 10:08 AM, Alexander Polyakov via lldb-

Re: [lldb-dev] SB API is not working properly with OSPython plugin

2019-02-21 Thread Tatyana Krasnukha via lldb-dev
> lldb Process::SetPrivateState (stopped) stop_id = 2
> error: error: process must be stopped.

These two lines are printed from different threads, you cannot be sure the real 
order of execution is the same.

The plugin should subscribe on public state change events and wait until one 
comes (correct me if I’m wrong about that).

From: lldb-dev  On Behalf Of Alexander 
Polyakov via lldb-dev
Sent: Thursday, February 21, 2019 9:54 PM
To: Jim Ingham 
Cc: LLDB 
Subject: Re: [lldb-dev] SB API is not working properly with OSPython plugin

It seems that the process plugin uses the Process::SetPrivateState at the right 
time. If you look at the log, you will see that the process is already in the 
'private stopped' state when the OS plugin is invoked.
(lldb) c
lldb Process::Resume -- locking run lock
lldb Process::PrivateResume() m_stop_id = 1, public state: stopped 
private state: stopped
lldb Process::SetPrivateState (running)
intern-state Process::ShouldBroadcastEvent (0x1abea90) => new state: 
running, last broadcast state: running - YES
intern-state Process::HandlePrivateEvent (pid = 1) broadcasting new state 
running (old state stopped) to public
intern-state Process::PushProcessIOHandler pushing IO handler
intern-state Process::HandlePrivateEvent updated m_iohandler_sync to 1
lldb Process thinks the process has resumed.
intern-state timeout = , event_sp)...
lldb waited from m_iohandler_sync to change from 0. New value is 1.
dbg.evt-handler  Process::SetPublicState (state = running, restarted = 0)
Process 1 resuming
lldb Process::SetPrivateState (stopped)
lldb Process::SetPrivateState (stopped) stop_id = 2
error: error: process must be stopped.
intern-state Process::ShouldBroadcastEvent (0x7f3e9c007450) => new state: 
stopped, last broadcast state: stopped - YES
intern-state Process::HandlePrivateEvent (pid = 1) broadcasting new state 
stopped (old state running) to public
intern-state timeout = , event_sp)...
dbg.evt-handler  Process::SetPublicState (state = stopped, restarted = 0)
dbg.evt-handler  Process::SetPublicState (stopped) -- unlocking run lock

On Fri, Feb 15, 2019 at 1:38 AM Jim Ingham 
mailto:jing...@apple.com>> wrote:
Your plugin should have set the private state to stopped when it figures out 
however it does that the process has stopped.  The API is 
Process::SetPrivateState.  Is that happening at the right time?

Jim



On Feb 14, 2019, at 1:50 PM, Alexander Polyakov 
mailto:polyakov@gmail.com>> wrote:

I found out that the plugin works well with an x86 application, so I think that 
the problem is in my process plugin. Maybe you know a place where to start 
looking for an issue?

On Thu, Feb 14, 2019 at 10:56 PM Jim Ingham 
mailto:jing...@apple.com>> wrote:
The simplest thing possible to reproduce the failure.  So some OS_Plugin 
implementation which tries to look up a global like this and fails, and some 
program source I can run it under that provides said global.  That way I can 
easily watch it fails as you describe when the plugin gets activated, and see 
why it isn’t allowing this call on private stop.

Jim



On Feb 14, 2019, at 11:50 AM, Alexander Polyakov 
mailto:polyakov@gmail.com>> wrote:

Sure, could you describe in more detail which example may help you?

чт, 14 февр. 2019 г. в 22:36, Jim Ingham 
mailto:jing...@apple.com>>:
That’s a little complicated…

lldb has two levels of stop state - private stops and public stops.  When the 
process gets notification from the underlying process plugin that the process 
stopped, it raises a private stop event.  That gets handled by the ShouldStop 
mechanism on the private state thread in lldb, and then if the stop is judged 
interesting to the user, it gets rebroadcast as a public stop.

For instance, when you issue a “step” command, lldb will stop and start the 
process multiple times as it walks through the source line.  But only the last 
of those stops are relevant to the user of LLDB, so all the other ones exist 
only as private stops.

The SB API’s for the most part should only consider a “publicly stopped” 
process accessible.  After all, you wouldn’t want some API to succeed sometimes 
if you happen to catch it in the middle of a private stop.

But the OperatingSystem plugin needs to get called right after a private stop, 
so it can provide threads for the ShouldStop mechanism.  We should really have 
some formal mechanism whereby things like the OS plugin can request elevated 
rights in the SB API’s, so that they can run at private stop time.  IIRC, we 
instead have a hack where SB API calls that run on the private state thread are 
blanket allowed to run at private stop.  The xnu Operating System plugin 
successfully gets global values to look up its threads.  So I’m not sure why 
that isn’t working for you.

Can you cook up a simple example showing the failure and I’ll have a look?

Jim



On Fe

Re: [lldb-dev] SB API is not working properly with OSPython plugin

2019-02-21 Thread Jim Ingham via lldb-dev


> On Feb 21, 2019, at 11:22 AM, Tatyana Krasnukha via lldb-dev 
>  wrote:
> 
> > lldb Process::SetPrivateState (stopped) stop_id = 2
> > error: error: process must be stopped.
>  
> These two lines are printed from different threads, you cannot be sure the 
> real order of execution is the same.
>  
> The plugin should subscribe on public state change events and wait until one 
> comes (correct me if I’m wrong about that).

That's not right.  When the process stops, but before lldb broadcasts a public 
stop event, it will query the OS Plugin directly to build up the thread list.  
It needs to do that before it declares a public stop because it uses the thread 
list to reason about whether to declare a public stop or not.  So the OS Plugin 
(and BTW the Thread Step Plan plugin is in the same boat) have to run after a 
private stop but before public one.  There isn't a principled way to do that.  
The best we have at present is a hack that says "if you are running on the 
private state thread, then you get to do things between private & public stop 
as if there were a public stop".  Cleaning that up is item 5 on the lldb 
Projects page if anyone is interested...

Jim

>  
> From: lldb-dev  On Behalf Of Alexander 
> Polyakov via lldb-dev
> Sent: Thursday, February 21, 2019 9:54 PM
> To: Jim Ingham 
> Cc: LLDB 
> Subject: Re: [lldb-dev] SB API is not working properly with OSPython plugin
>  
> It seems that the process plugin uses the Process::SetPrivateState at the 
> right time. If you look at the log, you will see that the process is already 
> in the 'private stopped' state when the OS plugin is invoked.
> 
> (lldb) c
> lldb Process::Resume -- locking run lock
> lldb Process::PrivateResume() m_stop_id = 1, public state: 
> stopped private state: stopped
> lldb Process::SetPrivateState (running)
> intern-state Process::ShouldBroadcastEvent (0x1abea90) => new state: 
> running, last broadcast state: running - YES
> intern-state Process::HandlePrivateEvent (pid = 1) broadcasting new state 
> running (old state stopped) to public
> intern-state Process::PushProcessIOHandler pushing IO handler
> intern-state Process::HandlePrivateEvent updated m_iohandler_sync to 1
> lldb Process thinks the process has resumed.
> intern-state timeout = , event_sp)...
> lldb waited from m_iohandler_sync to change from 0. New value is 
> 1.
> dbg.evt-handler  Process::SetPublicState (state = running, restarted = 0)
> Process 1 resuming
> lldb Process::SetPrivateState (stopped)
> lldb Process::SetPrivateState (stopped) stop_id = 2
> error: error: process must be stopped.
> intern-state Process::ShouldBroadcastEvent (0x7f3e9c007450) => new state: 
> stopped, last broadcast state: stopped - YES
> intern-state Process::HandlePrivateEvent (pid = 1) broadcasting new state 
> stopped (old state running) to public
> intern-state timeout = , event_sp)...
> dbg.evt-handler  Process::SetPublicState (state = stopped, restarted = 0)
> dbg.evt-handler  Process::SetPublicState (stopped) -- unlocking run lock
> 
>  
> On Fri, Feb 15, 2019 at 1:38 AM Jim Ingham  wrote:
> Your plugin should have set the private state to stopped when it figures out 
> however it does that the process has stopped.  The API is 
> Process::SetPrivateState.  Is that happening at the right time?
>  
> Jim
>  
> 
> 
> On Feb 14, 2019, at 1:50 PM, Alexander Polyakov  
> wrote:
>  
> I found out that the plugin works well with an x86 application, so I think 
> that the problem is in my process plugin. Maybe you know a place where to 
> start looking for an issue?
>  
> On Thu, Feb 14, 2019 at 10:56 PM Jim Ingham  wrote:
> The simplest thing possible to reproduce the failure.  So some OS_Plugin 
> implementation which tries to look up a global like this and fails, and some 
> program source I can run it under that provides said global.  That way I can 
> easily watch it fails as you describe when the plugin gets activated, and see 
> why it isn’t allowing this call on private stop.
>  
> Jim
>  
> 
> 
> On Feb 14, 2019, at 11:50 AM, Alexander Polyakov  
> wrote:
>  
> Sure, could you describe in more detail which example may help you?
>  
> чт, 14 февр. 2019 г. в 22:36, Jim Ingham :
> That’s a little complicated…  
>  
> lldb has two levels of stop state - private stops and public stops.  When the 
> process gets notification from the underlying process plugin that the process 
> stopped, it raises a private stop event.  That gets handled by the ShouldStop 
> mechanism on the private state thread in lldb, and then if the stop is judged 
> interesting to the user, it gets rebroadcast as a public stop.
>  
> For instance, when you issue a “step” command, lldb will stop and start the 
> process multiple times as it walks through the source line.  But only the 
> last of those stops are relevant to the user of LLDB, so all the other ones 
> exist onl

[lldb-dev] [Bug 26102] msbuild: MSB5004: The solution file has two projects named 'lldb tests'

2019-02-21 Thread via lldb-dev
https://bugs.llvm.org/show_bug.cgi?id=26102

Adrian McCarthy  changed:

   What|Removed |Added

 Resolution|--- |WORKSFORME
 Status|NEW |RESOLVED

--- Comment #2 from Adrian McCarthy  ---
I rebuilt my solution yesterday but don't see this problem.  I suspect it's
obsolete.  Perhaps this was a transitional problem when a bunch of the LLDB
testing was re-architected.

-- 
You are receiving this mail because:
You are the assignee for the bug.___
lldb-dev mailing list
lldb-dev@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev


Re: [lldb-dev] SB API is not working properly with OSPython plugin

2019-02-21 Thread Tatyana Krasnukha via lldb-dev
That clarified things, thanks!

I think, this is the reason:

ProcessRunLock &Process::GetRunLock() {
  if (m_private_state_thread.EqualsThread(Host::GetCurrentThread()))
return m_private_run_lock;
  else
return m_public_run_lock;
}

In our case the current thread is not m_private_state_thread. I create a 
separate thread for waiting a processor to halt and then SetPrivateState. It 
seems, that was an inappropriate approach.

-Original Message-
From: jing...@apple.com  
Sent: Thursday, February 21, 2019 11:58 PM
To: Tatyana Krasnukha 
Cc: Alexander Polyakov ; LLDB 
Subject: Re: [lldb-dev] SB API is not working properly with OSPython plugin



> On Feb 21, 2019, at 11:22 AM, Tatyana Krasnukha via lldb-dev 
>  wrote:
> 
> > lldb Process::SetPrivateState (stopped) stop_id = 2
> > error: error: process must be stopped.
>  
> These two lines are printed from different threads, you cannot be sure the 
> real order of execution is the same.
>  
> The plugin should subscribe on public state change events and wait until one 
> comes (correct me if I’m wrong about that).

That's not right.  When the process stops, but before lldb broadcasts a public 
stop event, it will query the OS Plugin directly to build up the thread list.  
It needs to do that before it declares a public stop because it uses the thread 
list to reason about whether to declare a public stop or not.  So the OS Plugin 
(and BTW the Thread Step Plan plugin is in the same boat) have to run after a 
private stop but before public one.  There isn't a principled way to do that.  
The best we have at present is a hack that says "if you are running on the 
private state thread, then you get to do things between private & public stop 
as if there were a public stop".  Cleaning that up is item 5 on the lldb 
Projects page if anyone is interested...

Jim

>  
> From: lldb-dev  On Behalf Of 
> Alexander Polyakov via lldb-dev
> Sent: Thursday, February 21, 2019 9:54 PM
> To: Jim Ingham 
> Cc: LLDB 
> Subject: Re: [lldb-dev] SB API is not working properly with OSPython 
> plugin
>  
> It seems that the process plugin uses the Process::SetPrivateState at the 
> right time. If you look at the log, you will see that the process is already 
> in the 'private stopped' state when the OS plugin is invoked.
> 
> (lldb) c
> lldb Process::Resume -- locking run lock
> lldb Process::PrivateResume() m_stop_id = 1, public state: 
> stopped private state: stopped
> lldb Process::SetPrivateState (running)
> intern-state Process::ShouldBroadcastEvent (0x1abea90) => new state: 
> running, last broadcast state: running - YES
> intern-state Process::HandlePrivateEvent (pid = 1) broadcasting new state 
> running (old state stopped) to public
> intern-state Process::PushProcessIOHandler pushing IO handler
> intern-state Process::HandlePrivateEvent updated m_iohandler_sync to 1
> lldb Process thinks the process has resumed.
> intern-state timeout = , event_sp)...
> lldb waited from m_iohandler_sync to change from 0. New value is 
> 1.
> dbg.evt-handler  Process::SetPublicState (state = running, restarted = 
> 0) Process 1 resuming
> lldb Process::SetPrivateState (stopped)
> lldb Process::SetPrivateState (stopped) stop_id = 2
> error: error: process must be stopped.
> intern-state Process::ShouldBroadcastEvent (0x7f3e9c007450) => new state: 
> stopped, last broadcast state: stopped - YES
> intern-state Process::HandlePrivateEvent (pid = 1) broadcasting new state 
> stopped (old state running) to public
> intern-state timeout = , event_sp)...
> dbg.evt-handler  Process::SetPublicState (state = stopped, restarted = 
> 0) dbg.evt-handler  Process::SetPublicState (stopped) -- unlocking run 
> lock
> 
>  
> On Fri, Feb 15, 2019 at 1:38 AM Jim Ingham  wrote:
> Your plugin should have set the private state to stopped when it figures out 
> however it does that the process has stopped.  The API is 
> Process::SetPrivateState.  Is that happening at the right time?
>  
> Jim
>  
> 
> 
> On Feb 14, 2019, at 1:50 PM, Alexander Polyakov  
> wrote:
>  
> I found out that the plugin works well with an x86 application, so I think 
> that the problem is in my process plugin. Maybe you know a place where to 
> start looking for an issue?
>  
> On Thu, Feb 14, 2019 at 10:56 PM Jim Ingham  wrote:
> The simplest thing possible to reproduce the failure.  So some OS_Plugin 
> implementation which tries to look up a global like this and fails, and some 
> program source I can run it under that provides said global.  That way I can 
> easily watch it fails as you describe when the plugin gets activated, and see 
> why it isn’t allowing this call on private stop.
>  
> Jim
>  
> 
> 
> On Feb 14, 2019, at 11:50 AM, Alexander Polyakov  
> wrote:
>  
> Sure, could you describe in more detail which example may help you?
>  
> чт, 14 февр. 2019 г. в 22:36, Jim Ingham :
> T

Re: [lldb-dev] SB API is not working properly with OSPython plugin

2019-02-21 Thread Jim Ingham via lldb-dev
What's triggering one of the OS Plugin methods to get run on this separate 
thread?  I would expect SetPrivateState would just cause the private stop event 
to get broadcast to the private state thread, and then that would wake up and 
then it would be the one to call the OS Plugin to do it's job.  That's how the 
GDBRemote plugin works, for instance.

Jim


> On Feb 21, 2019, at 2:54 PM, Tatyana Krasnukha 
>  wrote:
> 
> That clarified things, thanks!
> 
> I think, this is the reason:
> 
> ProcessRunLock &Process::GetRunLock() {
>  if (m_private_state_thread.EqualsThread(Host::GetCurrentThread()))
>return m_private_run_lock;
>  else
>return m_public_run_lock;
> }
> 
> In our case the current thread is not m_private_state_thread. I create a 
> separate thread for waiting a processor to halt and then SetPrivateState. It 
> seems, that was an inappropriate approach.
> 
> -Original Message-
> From: jing...@apple.com  
> Sent: Thursday, February 21, 2019 11:58 PM
> To: Tatyana Krasnukha 
> Cc: Alexander Polyakov ; LLDB 
> 
> Subject: Re: [lldb-dev] SB API is not working properly with OSPython plugin
> 
> 
> 
>> On Feb 21, 2019, at 11:22 AM, Tatyana Krasnukha via lldb-dev 
>>  wrote:
>> 
>>> lldb Process::SetPrivateState (stopped) stop_id = 2
>>> error: error: process must be stopped.
>> 
>> These two lines are printed from different threads, you cannot be sure the 
>> real order of execution is the same.
>> 
>> The plugin should subscribe on public state change events and wait until one 
>> comes (correct me if I’m wrong about that).
> 
> That's not right.  When the process stops, but before lldb broadcasts a 
> public stop event, it will query the OS Plugin directly to build up the 
> thread list.  It needs to do that before it declares a public stop because it 
> uses the thread list to reason about whether to declare a public stop or not. 
>  So the OS Plugin (and BTW the Thread Step Plan plugin is in the same boat) 
> have to run after a private stop but before public one.  There isn't a 
> principled way to do that.  The best we have at present is a hack that says 
> "if you are running on the private state thread, then you get to do things 
> between private & public stop as if there were a public stop".  Cleaning that 
> up is item 5 on the lldb Projects page if anyone is interested...
> 
> Jim
> 
>> 
>> From: lldb-dev  On Behalf Of 
>> Alexander Polyakov via lldb-dev
>> Sent: Thursday, February 21, 2019 9:54 PM
>> To: Jim Ingham 
>> Cc: LLDB 
>> Subject: Re: [lldb-dev] SB API is not working properly with OSPython 
>> plugin
>> 
>> It seems that the process plugin uses the Process::SetPrivateState at the 
>> right time. If you look at the log, you will see that the process is already 
>> in the 'private stopped' state when the OS plugin is invoked.
>> 
>> (lldb) c
>> lldb Process::Resume -- locking run lock
>> lldb Process::PrivateResume() m_stop_id = 1, public state: 
>> stopped private state: stopped
>> lldb Process::SetPrivateState (running)
>> intern-state Process::ShouldBroadcastEvent (0x1abea90) => new state: 
>> running, last broadcast state: running - YES
>> intern-state Process::HandlePrivateEvent (pid = 1) broadcasting new 
>> state running (old state stopped) to public
>> intern-state Process::PushProcessIOHandler pushing IO handler
>> intern-state Process::HandlePrivateEvent updated m_iohandler_sync to 1
>> lldb Process thinks the process has resumed.
>> intern-state timeout = , event_sp)...
>> lldb waited from m_iohandler_sync to change from 0. New value is 
>> 1.
>> dbg.evt-handler  Process::SetPublicState (state = running, restarted = 
>> 0) Process 1 resuming
>> lldb Process::SetPrivateState (stopped)
>> lldb Process::SetPrivateState (stopped) stop_id = 2
>> error: error: process must be stopped.
>> intern-state Process::ShouldBroadcastEvent (0x7f3e9c007450) => new 
>> state: stopped, last broadcast state: stopped - YES
>> intern-state Process::HandlePrivateEvent (pid = 1) broadcasting new 
>> state stopped (old state running) to public
>> intern-state timeout = , event_sp)...
>> dbg.evt-handler  Process::SetPublicState (state = stopped, restarted = 
>> 0) dbg.evt-handler  Process::SetPublicState (stopped) -- unlocking run 
>> lock
>> 
>> 
>> On Fri, Feb 15, 2019 at 1:38 AM Jim Ingham  wrote:
>> Your plugin should have set the private state to stopped when it figures out 
>> however it does that the process has stopped.  The API is 
>> Process::SetPrivateState.  Is that happening at the right time?
>> 
>> Jim
>> 
>> 
>> 
>> On Feb 14, 2019, at 1:50 PM, Alexander Polyakov  
>> wrote:
>> 
>> I found out that the plugin works well with an x86 application, so I think 
>> that the problem is in my process plugin. Maybe you know a place where to 
>> start looking for an issue?
>> 
>> On Thu, Feb 14, 2019 at 10:56 PM Jim Ingham  wrote:
>> The simplest t