[lldb-dev] environment vars not passed through to linux targets
Hi, I'm remote debugging to linux (platform select remote-linux), and then launching a process on the target. > file > process launch -w /some/dir When I do this, the process is launched but it gets an empty list of environment variables. I've confirmed that lldb and the gdb server both have the vars, but not the target process. I can manually pass environment variables to the target using the '--environment' flag, and this works. But in general I need these to be inherited. Based on online comments about this working on mac, this appears to be a linux-specific bug? Is this expected or known? Thanks, Chris Book ___ lldb-dev mailing list lldb-dev@lists.llvm.org http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev
Re: [lldb-dev] environment vars not passed through to linux targets
Yes, exactly. So things like LD_LIBRARY_PATH, PATH and others that are set based on the target linux env. The client doesn't have this information available to manually set them all. On Wed, Jul 5, 2017 at 11:44 PM Stephane Sezer wrote: > Which set of env vars did you want the child process to inherit? The env > vars of the platform process on the remote? > > On Wed, Jul 5, 2017 at 8:47 AM Christopher Book via lldb-dev < > lldb-dev@lists.llvm.org> wrote: > >> Hi, I'm remote debugging to linux (platform select remote-linux), and >> then launching a process on the target. >> > file >> > process launch -w /some/dir >> >> When I do this, the process is launched but it gets an empty list of >> environment variables. I've confirmed that lldb and the gdb server both >> have the vars, but not the target process. >> >> I can manually pass environment variables to the target using the >> '--environment' flag, and this works. But in general I need these to be >> inherited. >> >> Based on online comments about this working on mac, this appears to be a >> linux-specific bug? Is this expected or known? >> >> Thanks, >> Chris Book >> > ___ >> lldb-dev mailing list >> lldb-dev@lists.llvm.org >> http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev >> > -- > -- > Stephane Sezer > ___ lldb-dev mailing list lldb-dev@lists.llvm.org http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev
[lldb-dev] process launch without install
Hi, I'm remote debugging to linux (from windows), and I'm trying to launch a binary that already exists on both the client and target machines. Regardless of what I do, it always wants to transfer the (very large) executable to the remote machine and run it. Is there a way to avoid this? I'm running this, with lldb launched from the client working dir that contains the binary: > platform select remote-linux > platform settings -w /remote/working/dir > platform connect connect://remotehost:44333 > target create -r binaryname binaryname > process launch -s I always end up with a large transfer, due to the install. From the logs: Target::lldb_private::Target::Launch() called for C:\my\local\working\dir\binaryname Platform::Install (src='C:\my\local\working\dir\binaryname', dst='/remote/working/dir/binaryname') Regardless of what I do, I always end up with a copy to the remote working dir. If I specify the full path to the remote binary, it creates a new copy in the remote working dir. In the above example I have the binary already in the working dir and the transfer still occurs. Is there a way to avoid the copy when the binary exists on both sides? Thanks, Chris ___ lldb-dev mailing list lldb-dev@lists.llvm.org http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev
[lldb-dev] review for crash patch to gdb remote
Hi, I posted a patch last week for a crash in gdb-remote I was seeing when debugging a specific binary. https://reviews.llvm.org/D36620 Is there someone that I should add to review it? Thanks, Chris Book ___ lldb-dev mailing list lldb-dev@lists.llvm.org http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev
[lldb-dev] race condition using gdb-remote over ssh port forwarding
Greetings, I've been using liblldb to remotely debug to a linux server with
port forwarding. To do this, I start lldb-server to with --listen
specifying a localhost port, as well as with min-gdbserver-port and
--max-gdbserver-port to specify a specific port for use by 'gdb remote'.
Both ports are forwarded to the remote PC, where liblldb connects to
localhost.
This generally works fine, but there is a race condition. When the client
tells lldb-server to start gdb-remote, the port is returned to the client
which may try to connect before the gdb-remote process is actually
listening. Without port-forwarding, this is okay because the client has
retry logic:
ProcessGDBRemote::ConnectToDebugserver
...
retry_count++;
if (retry_count >= max_retry_count)
break;
usleep(10);
But with port-forwarding, the initial connection is always accepted by the
port-forwarder, and only then does it try to establish a connection to the
remote port. It has no way to not accept the incoming local connection
until it tries the remote end.
lldb has some logic to detect this further in the function, by using a
handshake to ensure the connection is actually made:
// We always seem to be able to open a connection to a local port
// so we need to make sure we can then send data to it. If we can't
// then we aren't actually connected to anything, so try and do the
// handshake with the remote GDB server and make sure that goes
// alright.
if (!m_gdb_comm.HandshakeWithServer(&error)) {
m_gdb_comm.Disconnect();
if (error.Success())
error.SetErrorString("not connected to remote gdb server");
return error;
}
But the problem here is that no retry is performed on failure. The caller
to the 'attach' API also can't retry because the gdb server is terminated
on the error.
I would like to submit a patch, but first check to see if this solution
would be acceptable:
- Include the handshake within the connection retry loop.
- This means fully disconnecting the re-establishing the connection in the
loop if the handshake fails.
- Changing the timeout check to be based on a total absolute time instead
of 50 iterations with a 100ms sleep.
Thoughts?
Alternatives could be:
- Have lldb-server delay responding to the 'start gdb server' request until
it could tell (somehow) that the process is listening.
- A sleep of some kind on the client side after starting the server but
before trying to connect.
Thanks,
Chris
___
lldb-dev mailing list
lldb-dev@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev
Re: [lldb-dev] race condition using gdb-remote over ssh port forwarding
I have been using both a custom port-forwarding program as well as
OpenSSH. The custom solution is a bit slower because it goes through a
proxy, so the problem doesn't manifest. I've been trying to switch to
OpenSSH which is fast enough to consistent hit the race condition.
I would expect other solutions to be similar to OpenSSH because I don't
think there's a way to have the client connection fail based on the result
of a remote connection. Our own forwarding solution was built using this
method because of this.
I agree that fixing this on the server seems like a better solution, since
the client only gets a port when it has one to connect to. But I don't
know the best way to make this happen from lldb-server after launching the
gdb instance.
On Mon, Nov 27, 2017 at 4:48 PM Greg Clayton wrote:
> When I wrote the code I was assuming that when the platform lldb-server
> responds with the port that the gdb-remote would be ready to receive
> packets right away, so I can see how and why this is happening. Seems like
> we have the retry stuff under control when we don't get a connection right
> away, but we should fix this such that when we hand the port back to LLDB
> from platform lldb-server, it should be listening and ready to accept a
> connection right away with no delays needed, though this can wait until
> later since it currently works.
>
> What kind of port forwarding are you using? The main issue is I would
> assume that when someone tries to connect to a port on the port forwarder
> that it would fail to connect if it isn't able to connect on the other
> side. So this really comes down to a question of what a standard port
> forwarder's contract should really be.
>
> If anyone has extensive experience in port forward tech, please chime in.
>
> Short answer: not sure what the right solution is as it depends on what
> proper port forwarding etiquette is.
>
> Greg
>
>
>
>
>
> > On Nov 27, 2017, at 12:33 PM, Christopher Book via lldb-dev <
> lldb-dev@lists.llvm.org> wrote:
> >
> > Greetings, I've been using liblldb to remotely debug to a linux server
> with port forwarding. To do this, I start lldb-server to with --listen
> specifying a localhost port, as well as with min-gdbserver-port and
> --max-gdbserver-port to specify a specific port for use by 'gdb remote'.
> Both ports are forwarded to the remote PC, where liblldb connects to
> localhost.
> >
> > This generally works fine, but there is a race condition. When the
> client tells lldb-server to start gdb-remote, the port is returned to the
> client which may try to connect before the gdb-remote process is actually
> listening. Without port-forwarding, this is okay because the client has
> retry logic:
> >
> > ProcessGDBRemote::ConnectToDebugserver
> > ...
> >retry_count++;
> > if (retry_count >= max_retry_count)
> > break;
> > usleep(10);
> >
> > But with port-forwarding, the initial connection is always accepted by
> the port-forwarder, and only then does it try to establish a connection to
> the remote port. It has no way to not accept the incoming local connection
> until it tries the remote end.
> >
> > lldb has some logic to detect this further in the function, by using a
> handshake to ensure the connection is actually made:
> >
> > // We always seem to be able to open a connection to a local port
> > // so we need to make sure we can then send data to it. If we can't
> > // then we aren't actually connected to anything, so try and do the
> > // handshake with the remote GDB server and make sure that goes
> > // alright.
> > if (!m_gdb_comm.HandshakeWithServer(&error)) {
> > m_gdb_comm.Disconnect();
> > if (error.Success())
> > error.SetErrorString("not connected to remote gdb server");
> > return error;
> > }
> >
> > But the problem here is that no retry is performed on failure. The
> caller to the 'attach' API also can't retry because the gdb server is
> terminated on the error.
> >
> > I would like to submit a patch, but first check to see if this solution
> would be acceptable:
> > - Include the handshake within the connection retry loop.
> > - This means fully disconnecting the re-establishing the connection in
> the loop if the handshake fails.
> > - Changing the timeout check to be based on a total absolute time
> instead of 50 iterations with a 100ms sleep.
> >
> > Thoughts?
> >
> > Alternatives could be:
> > - Have lldb-server delay responding to the 'start gdb se
Re: [lldb-dev] race condition using gdb-remote over ssh port forwarding
Hi Pavel, I think you are on the right track in that it already does seem
to wait for the gdbserver to be up using a pipe.
In StartDebugserverProcess it seems like there is a pipe created to tell
when the server is running. Here is a comment:
// socket_pipe is used by debug server to communicate back either
// TCP port or domain socket name which it listens on.
// The second purpose of the pipe to serve as a synchronization point -
// once data is written to the pipe, debug server is up and running.
However, it looks like specifying --min-gdbserver-port and
--max-gdbserver-port cause the pipe to not be used for some reason.
Here is the gdbserver invocation when I don't specify the gdb min and max
port:
> lldb-server gdbserver tcp://:0 --native-regs --pipe 6
And here is the gdbserver invocation when I do specify the gdb port:
> lldb-server gdbserver tcp://: --native-regs
Its not obvious to me from looking at the code why this is being skipped.
But I will debug further why this argument is not used in this case.
Thanks,
Chris
On Tue, Nov 28, 2017 at 5:25 AM Pavel Labath wrote:
> On 27 November 2017 at 20:33, Christopher Book via lldb-dev
> wrote:
> > Greetings, I've been using liblldb to remotely debug to a linux server
> with
> > port forwarding. To do this, I start lldb-server to with --listen
> > specifying a localhost port, as well as with min-gdbserver-port and
> > --max-gdbserver-port to specify a specific port for use by 'gdb remote'.
> > Both ports are forwarded to the remote PC, where liblldb connects to
> > localhost.
> >
> > This generally works fine, but there is a race condition. When the
> client
> > tells lldb-server to start gdb-remote, the port is returned to the client
> > which may try to connect before the gdb-remote process is actually
> > listening.
> Are you sure this is what's happening? Looking at lldb-server source
> code, I see that it starts listening on lldb-gdbserver.cpp:320
> error = acceptor_up->Listen(1);
> if (error.Fail()) {
> fprintf(stderr, "failed to listen: %s\n", error.AsCString());
> exit(1);
> }
> and the port number is written out only later, on line 334:
> error = acceptor_up->Listen(1);
> if (error.Fail()) {
> fprintf(stderr, "failed to listen: %s\n", error.AsCString());
> exit(1);
> }
>
> After listen returns the connection by your port forwarder should not
> be rejected, and it can only connect once it knows the port to connect
> to.
>
>
> That's not to say that connecting through port forwarders is easy. For
> android debugging, we also have to setup port forwarding with adb, and
> we need to do some quite elaborate dances to make sure that the
> connection is reliable and race free (see
> PlatformAndroidRemoteGDBServer::MakeConnectURL) -- however this is to
> resolve a different issue (allocating the port on the local side from
> which to do the forwarding), which does not sound like the problem you
> describe.
>
>
> > I would like to submit a patch, but first check to see if this solution
> > would be acceptable:
> > - Include the handshake within the connection retry loop.
> > - This means fully disconnecting the re-establishing the connection in
> the
> > loop if the handshake fails.
> > - Changing the timeout check to be based on a total absolute time
> instead of
> > 50 iterations with a 100ms sleep.
> >
> > Thoughts?
> >
> > Alternatives could be:
> > - Have lldb-server delay responding to the 'start gdb server' request
> until
> > it could tell (somehow) that the process is listening.
> > - A sleep of some kind on the client side after starting the server but
> > before trying to connect.
> >
>
> The most port-forwarder-friendly solution (and one that I've been
> wanting to implement for a while, but never got around to it) I can
> think of is to not require a second port for the llgs connection. It
> could work approximately like this:
> - we add a new packet type to the lldb-server platform instance: (e.g.
> qExecGDBServer)
> - upon receiving this packet, the platform instance would exec the
> gdb-server instance, and pass it the existing socket file descriptor
> (via a command line argument or whatever)
> - the gdb-server instance would start up with the connection primed
> and ready, and would not need to do any listening and writing back the
> port number, etc.
> - the lldb client would end up with it's "platform" connection being
> connected to llgs, and would have to create a new platform connection
> (or it could create a scratch platform
Re: [lldb-dev] race condition using gdb-remote over ssh port forwarding
I made a mistake and was not looking at the latest source. Looks like this exact issue was already discovered and fixed: https://reviews.llvm.org/D30255 I'm using 4.0.1, so it appears I need to upgrade. Chris On Wed, Nov 29, 2017 at 5:45 AM Pavel Labath wrote: > On 28 November 2017 at 20:54, Christopher Book wrote: > > Hi Pavel, I think you are on the right track in that it already does > seem to > > wait for the gdbserver to be up using a pipe. > > > > In StartDebugserverProcess it seems like there is a pipe created to tell > > when the server is running. Here is a comment: > > > > // socket_pipe is used by debug server to communicate back either > > // TCP port or domain socket name which it listens on. > > // The second purpose of the pipe to serve as a synchronization > point - > > // once data is written to the pipe, debug server is up and running. > > > > However, it looks like specifying --min-gdbserver-port and > > --max-gdbserver-port cause the pipe to not be used for some reason. > > > > Here is the gdbserver invocation when I don't specify the gdb min and max > > port: > >> lldb-server gdbserver tcp://:0 --native-regs --pipe 6 > > > > And here is the gdbserver invocation when I do specify the gdb port: > >> lldb-server gdbserver tcp://: --native-regs > > > > Its not obvious to me from looking at the code why this is being skipped. > > But I will debug further why this argument is not used in this case. > > > > Aha, interesting. That would explain why there is a race. We should > add the pipe synchronization even in case we know what is the port > going to be (though I still like the "exec" idea). > ___ lldb-dev mailing list lldb-dev@lists.llvm.org http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev
