Re: [lldb-dev] [BUG?] Confusion between translation units?

[Ramkumar Ramachandra via lldb-dev]

Greg Clayton wrote:
> Not really. If you can send me a concrete example of something that isn't 
> working, it might help.

Reporting progress.

1. `im loo -t` only ever finds one "best match", and the plist seems
to have no effect on it.

2. The plist parsing code does get executed. I checked with the
debugger. I also checked symlink versus non-symlinked paths in the
plist, which I'd missed last time. dwarfdump does not seem to read it,
and `im loo` says nothing different, so I have no idea if anything
really got remapped.

3. I managed to catch this "unstable" case: you try something, it
prints a huge error, you try something else in a different frame with
the same type, it prints fine, you come back to the original thing and
there's a smaller error.

frame #0: 0x000101c3ce8c libmwcgir_vm_rt.dylib`(anonymous
namespace)::CgJITMemManager::endFunctionBody(this=0x00010e8637b0,
F=0x00010a62ccc0, FunctionStart="�?^\n\x01",
FunctionEnd="...")
+ 28 at CgJITMemoryManager.cpp:437
   434(void)FunctionStart;
   435(void)FunctionEnd;
   436
-> 437assert(FunctionEnd > FunctionStart);
   438assert(FunctionStart == (uint8_T *)(CurBlock+1) &&
   439   "Mismatched function start/end!");
   440
(lldb) p F
error: field 'BasicBlocks' declared with incompatible types in
different translation units ('BasicBlockListType' (aka
'llvm::iplist
>') vs. 'BasicBlockListType' (aka 'llvm::iplist >'))
error: field 'ArgumentList' declared with incompatible types in
different translation units ('ArgumentListType' (aka
'llvm::iplist >')
vs. 'ArgumentListType' (aka 'llvm::iplist >'))
error: field 'BasicBlocks' declared with incompatible types in
different translation units ('BasicBlockListType' (aka
'llvm::iplist
>') vs. 'BasicBlockListType' (aka 'llvm::iplist >'))
error: field 'Context' declared with incompatible types in different
translation units ('llvm::LLVMContext &' vs. 'llvm::LLVMContext &')
error: field 'GlobalList' declared with incompatible types in
different translation units ('GlobalListType' (aka
'llvm::iplist >') vs. 'GlobalListType' (aka
'llvm::iplist >'))
error: field 'FunctionList' declared with incompatible types in
different translation units ('FunctionListType' (aka
'llvm::iplist >')
vs. 'FunctionListType' (aka 'llvm::iplist >'))
error: field 'AliasList' declared with incompatible types in different
translation units ('AliasListType' (aka
'llvm::iplist
>') vs. 'AliasListType' (aka 'llvm::iplist >'))
error: field 'NamedMDList' declared with incompatible types in
different translation units ('NamedMDListType' (aka
'llvm::ilist') vs. 'NamedMDListType' (aka
'llvm::ilist'))
error: field 'GlobalScopeAsm' declared with incompatible types in
different translation units ('string' (aka
'std::__1::basic_string,
std::__1::allocator >') vs. 'string' (aka
'std::__1::basic_string,
std::__1::allocator >'))
error: field 'ComdatSymTab' declared with incompatible types in
different translation units ('ComdatSymTabType' (aka
'llvm::StringMap') vs.
'ComdatSymTabType' (aka 'llvm::StringMap'))
error: field 'ModuleID' declared with incompatible types in different
translation units ('string' (aka 'std::__1::basic_string, std::__1::allocator >') vs.
'string' (aka 'std::__1::basic_string, std::__1::allocator >'))
error: field 'TargetTriple' declared with incompatible types in
different translation units ('string' (aka
'std::__1::basic_string,
std::__1::allocator >') vs. 'string' (aka
'std::__1::basic_string,
std::__1::allocator >'))
error: field 'DataLayoutStr' declared with incompatible types in
different translation units ('string' (aka
'std::__1::basic_string,
std::__1::allocator >') vs. 'string' (aka
'std::__1::basic_string,
std::__1::allocator >'))
error: field 'Parent' declared with incompatible types in different
translation units ('llvm::Module *' vs. 'llvm::Module *')
error: field 'Parent' declared with incompatible types in different
translation units ('l

Re: [lldb-dev] Accessing DWARF information from C++

[Greg Clayton via lldb-dev]

> On Oct 30, 2015, at 3:35 AM, Stefan Kratochwil via lldb-dev 
>  wrote:
> 
> Hi Greg,
> 
> thanks for your reply.
> 
> I am developing a dynamic software updating tool for dynamically linked C 
> libraries (in short: I want to patch dynamic libraries in-memory).
> 
> My goal is to achieve that without any code instrumentation. The main problem 
> is the state transformation between the old and the new version - here I need 
> as much type (and location) information as possible.
> 
> I am currently unaware of what types of information I _really_ need, so for 
> now, getting as much information as possible is my way to go. I guess that 
> the most important types of information will be:
> - size of primitives
> - size of pointers
> - type hierarchies and
> - the in memory-structure of compound data types
> 
> At the moment I am concentrating on the Linux / UNIX domain, so DWARF is my 
> preferred format.

This is all available through the LLDB API, so I would suggest using LLDB and 
its debug info agnostic APIs. Let me know if you need to know how to find 
certain kinds of information from our API and I can help get you up and running.

Greg Clayton

___
lldb-dev mailing list
lldb-dev@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev

Re: [lldb-dev] [BUG?] Confusion between translation units?

[Greg Clayton via lldb-dev]

> On Nov 3, 2015, at 12:07 PM, Ramkumar Ramachandra  wrote:
> 
> Greg Clayton wrote:
>> Not really. If you can send me a concrete example of something that isn't 
>> working, it might help.
> 
> Reporting progress.
> 
> 1. `im loo -t` only ever finds one "best match", and the plist seems
> to have no effect on it.

Use the --all option and it should print all matches.

> 2. The plist parsing code does get executed. I checked with the
> debugger. I also checked symlink versus non-symlinked paths in the
> plist, which I'd missed last time. dwarfdump does not seem to read it,
> and `im loo` says nothing different, so I have no idea if anything
> really got remapped.


> 
> 3. I managed to catch this "unstable" case: you try something, it
> prints a huge error, you try something else in a different frame with
> the same type, it prints fine, you come back to the original thing and
> there's a smaller error.


So one thing know in LLDB: don't use the expression parser if you don't have to 
when you run into problems. "frame variable" will show you what you want to see 
if you are just inspecting a variable or members of a variable:

(lldb) frame variable F
(lldb) frame variable F.Linkage
(lldb) frame variable a->b.c->e[12]

"frame variable" knows how to use ".", "->", "[]" for array access or using a 
pointer as an array. It also knows how to prefix the string with "*" or "&":

(lldb) frame variable &F
(lldb) frame variable *a->b.my_ptr

So you should really use frame variable for your viewing needs. It doesn't do 
any expressions, but it will allow you to view your variable. You should use 
the expression parser when you need to call a function, do match, flow control 
or anything that isn't viewing variables.


> 
>frame #0: 0x000101c3ce8c libmwcgir_vm_rt.dylib`(anonymous
> namespace)::CgJITMemManager::endFunctionBody(this=0x00010e8637b0,
> F=0x00010a62ccc0, FunctionStart="�?^\n\x01",
> FunctionEnd="...")
> + 28 at CgJITMemoryManager.cpp:437
>   434(void)FunctionStart;
>   435(void)FunctionEnd;
>   436
> -> 437assert(FunctionEnd > FunctionStart);
>   438assert(FunctionStart == (uint8_T *)(CurBlock+1) &&
>   439   "Mismatched function start/end!");
>   440
> (lldb) p F
> error: field 'BasicBlocks' declared with incompatible types in
> different translation units ('BasicBlockListType' (aka
> 'llvm::iplist
>> ') vs. 'BasicBlockListType' (aka 'llvm::iplist llvm::ilist_traits >'))
> error: field 'ArgumentList' declared with incompatible types in
> different translation units ('ArgumentListType' (aka
> 'llvm::iplist >')
> vs. 'ArgumentListType' (aka 'llvm::iplist llvm::ilist_traits >'))
> error: field 'BasicBlocks' declared with incompatible types in
> different translation units ('BasicBlockListType' (aka
> 'llvm::iplist
>> ') vs. 'BasicBlockListType' (aka 'llvm::iplist llvm::ilist_traits >'))
> error: field 'Context' declared with incompatible types in different
> translation units ('llvm::LLVMContext &' vs. 'llvm::LLVMContext &')
> error: field 'GlobalList' declared with incompatible types in
> different translation units ('GlobalListType' (aka
> 'llvm::iplist llvm::ilist_traits >') vs. 'GlobalListType' (aka
> 'llvm::iplist llvm::ilist_traits >'))
> error: field 'FunctionList' declared with incompatible types in
> different translation units ('FunctionListType' (aka
> 'llvm::iplist >')
> vs. 'FunctionListType' (aka 'llvm::iplist llvm::ilist_traits >'))
> error: field 'AliasList' declared with incompatible types in different
> translation units ('AliasListType' (aka
> 'llvm::iplist
>> ') vs. 'AliasListType' (aka 'llvm::iplist llvm::ilist_traits >'))
> error: field 'NamedMDList' declared with incompatible types in
> different translation units ('NamedMDListType' (aka
> 'llvm::ilist') vs. 'NamedMDListType' (aka
> 'llvm::ilist'))
> error: field 'GlobalScopeAsm' declared with incompatible types in
> different translation units ('stri

Re: [lldb-dev] [BUG?] Confusion between translation units?

[Ramkumar Ramachandra via lldb-dev]

Greg Clayton wrote:
> Be sure to use "frame variable" more as it will never run into these type 
> importation problems when you need to view a variable or members of a 
> variable.
>
> It would be interesting to see the output of "frame variable" from frame 0 
> and frame 4 and see if they differ at all.

`fr va` seems to fix the immediate problem, thanks! I'll let you know
if I run into issues with it.

We really should fix the expression parser, no? Why is it behaving strangely?
___
lldb-dev mailing list
lldb-dev@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev

[lldb-dev] [Bug 25396] New: Unclear error message when loading PE executable on OS X

[via lldb-dev]

https://llvm.org/bugs/show_bug.cgi?id=25396

Bug ID: 25396
   Summary: Unclear error message when loading PE executable on OS
X
   Product: lldb
   Version: unspecified
  Hardware: PC
OS: MacOS X
Status: NEW
  Severity: normal
  Priority: P
 Component: All Bugs
  Assignee: lldb-dev@lists.llvm.org
  Reporter: whitequ...@whitequark.org
CC: llvm-b...@lists.llvm.org
Classification: Unclassified

lldb currently says: "error: the platform is not currently connected".

This could be improved.

-- 
You are receiving this mail because:
You are the assignee for the bug.
___
lldb-dev mailing list
lldb-dev@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev

Re: [lldb-dev] [BUG?] Confusion between translation units?

[Greg Clayton via lldb-dev]

> On Nov 3, 2015, at 1:12 PM, Ramkumar Ramachandra  wrote:
> 
> Greg Clayton wrote:
>> Be sure to use "frame variable" more as it will never run into these type 
>> importation problems when you need to view a variable or members of a 
>> variable.
>> 
>> It would be interesting to see the output of "frame variable" from frame 0 
>> and frame 4 and see if they differ at all.
> 
> `fr va` seems to fix the immediate problem, thanks! I'll let you know
> if I run into issues with it.
> 
> We really should fix the expression parser, no?

Yes, but we need to identify the problem.

> Why is it behaving strangely?

There are many clang::ASTContext objects in LLDB. 1 per module, 1 per target 
and 1 for each expression. We import types from one AST to another. When 
running expressions we import one type for "foo" from liba.so into the 
expression AST, then we might try to import another from libb.so into the 
expression AST. If the "foo" type is already in the expression AST, then 
we need to verify the two types are the same. When this fails, then the 
expression fails.

So we need to determine which two versions of "foo" are different and 
where they differ so we can understand how to fix this.

Follow my previous suggestions: enable lldb expr logging and also compare the 
output of the two "frame variable F" for frame 0 and frame 4 in your previous 
example. Since "frame variable" always uses the type _only_ from liba.so or 
libb.so, we might be able to see a difference in the output of "frame variable 
F" for the two frames. If we don't see any difference, we need to dump the type 
from "image lookup --all -t "foo" and see if we see any differences there. 
If we don't see differences we need to then step through the code that is 
attempting to import "foo" the second time and see where it thinks the 
types differ.

Greg Clayton

___
lldb-dev mailing list
lldb-dev@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev

Re: [lldb-dev] Attaching to a stopped (cored) process hangs lldb-server

[Greg Clayton via lldb-dev]

One different approach is to have your tool write all STDIN to a file (the core 
file comes into the tool as STDIN bytes) and then hand LLDB the core file and 
do any needed backtracing and data gathering from the core file instead of 
actually attaching to the process for real. All executable and shared library 
object files (ELF files) from the core file are still on disk so you can get 
symbols and use the debug info, so LLDB should be able to load all frames up 
and symbolicate up the crash location. It should be just as good as having the 
process around without any bad side affects. Core files are less useful if they 
must be archived and symbolicated later because the executable files might not 
be around anymore since things like test suites might produce binaries for 
testing and remove them after the test is run or crashed.

What do you think about this approach?

Greg Clayton


> On Nov 2, 2015, at 5:54 PM, Mark Chandler via lldb-dev 
>  wrote:
> 
> So im trying to write a core handler program and use lldb to attach and dump 
> important information about it. This works if a use my tool to attach to an 
> existing one but I found that lldb-server will hang in a waitpid call if the 
> kernel has invoked the tool after another process has cored.
>  
> Example:
> · /proc/sys/kernel/core_pattern is set to |/opt/core_tool
> · Run a.out and it segfaults
> · Kernel invokes core_tool that uses lldb AttachToProcess and a.out 
> is in state “S+”
> · lldb-server hangs in 
> source\Plugins\Process\Linux\NativeProcessLinux.cpp:867
> · if I remove the waitpid it doesn’t hang but fails to attach
>  
> Looks like gdb had a similar problem as well: 
> http://www.sourceware.org/ml/gdb-patches/2008-04/msg00224.html
> Any ideas on how to fix this?
>  
> Mark Chandler
> Battle.Net Engineering Systems | Blizzard Entertainment
> (P) 949-955-1380 x15353
>  
> ___
> lldb-dev mailing list
> lldb-dev@lists.llvm.org
> http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev

___
lldb-dev mailing list
lldb-dev@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev

Re: [lldb-dev] Attaching to a stopped (cored) process hangs lldb-server

[Mark Chandler via lldb-dev]

Not able to do that as the servers have no hard drives (use ram disk and net 
boot) and the tool is trying to avoid a core storm that takes down the network 
file share. I found out what is causing it to hang, there is a call to waitpid 
in NativeLinuxProcess.cpp that waits forever. As the process is already 
stopped, I disabled that and it looks to be working

Mark Chandler
Battle.Net Engineering Systems | Blizzard Entertainment
(P) 949-955-1380 x15353

-Original Message-
From: Greg Clayton [mailto:gclay...@apple.com] 
Sent: Tuesday, November 03, 2015 4:34 PM
To: Mark Chandler 
Cc: lldb-dev@lists.llvm.org
Subject: Re: [lldb-dev] Attaching to a stopped (cored) process hangs lldb-server

One different approach is to have your tool write all STDIN to a file (the core 
file comes into the tool as STDIN bytes) and then hand LLDB the core file and 
do any needed backtracing and data gathering from the core file instead of 
actually attaching to the process for real. All executable and shared library 
object files (ELF files) from the core file are still on disk so you can get 
symbols and use the debug info, so LLDB should be able to load all frames up 
and symbolicate up the crash location. It should be just as good as having the 
process around without any bad side affects. Core files are less useful if they 
must be archived and symbolicated later because the executable files might not 
be around anymore since things like test suites might produce binaries for 
testing and remove them after the test is run or crashed.

What do you think about this approach?

Greg Clayton


> On Nov 2, 2015, at 5:54 PM, Mark Chandler via lldb-dev 
>  wrote:
> 
> So im trying to write a core handler program and use lldb to attach and dump 
> important information about it. This works if a use my tool to attach to an 
> existing one but I found that lldb-server will hang in a waitpid call if the 
> kernel has invoked the tool after another process has cored.
>  
> Example:
> · /proc/sys/kernel/core_pattern is set to |/opt/core_tool
> · Run a.out and it segfaults
> · Kernel invokes core_tool that uses lldb AttachToProcess and a.out 
> is in state “S+”
> · lldb-server hangs in 
> source\Plugins\Process\Linux\NativeProcessLinux.cpp:867
> · if I remove the waitpid it doesn’t hang but fails to attach
>  
> Looks like gdb had a similar problem as well: 
> http://www.sourceware.org/ml/gdb-patches/2008-04/msg00224.html
> Any ideas on how to fix this?
>  
> Mark Chandler
> Battle.Net Engineering Systems | Blizzard Entertainment
> (P) 949-955-1380 x15353
>  
> ___
> lldb-dev mailing list
> lldb-dev@lists.llvm.org
> http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev

___
lldb-dev mailing list
lldb-dev@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev

Re: [lldb-dev] Accessing DWARF information from C++

[Frank Ch. Eigler via lldb-dev]

Stefan Kratochwil wrote:

> [...]
> I am developing a dynamic software updating tool for dynamically
> linked C libraries (in short: I want to patch dynamic libraries
> in-memory).

See also dyninst.

> [...]
> I am currently unaware of what types of information I _really_ need,
> so for now, getting as much information as possible is my way to go. I
> guess that the most important types of information will be [...]
> At the moment I am concentrating on the Linux / UNIX domain, so DWARF
> is my preferred format.

See also elfutils + https://github.com/pmachata/reflection

- FChE
___
lldb-dev mailing list
lldb-dev@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev

Re: [lldb-dev] Attaching to a stopped (cored) process hangs lldb-server

[Greg Clayton via lldb-dev]

So you would save the file, create a target:

const char *core_path = ...; // Save STDIN to a  file and put path in 
"core_path"
const bool source_init_files = false;
debugger = lldb::SBDebugger::Create(source_init_files);
target = debugger.CreateTarget(None);
process = target.LoadCore(core_path);
if (process.IsValid())
{
// Do any symbolication you need to on your process core file
// as it will behave just like a real process, you just can't run it
}


> On Nov 3, 2015, at 4:34 PM, Greg Clayton via lldb-dev 
>  wrote:
> 
> One different approach is to have your tool write all STDIN to a file (the 
> core file comes into the tool as STDIN bytes) and then hand LLDB the core 
> file and do any needed backtracing and data gathering from the core file 
> instead of actually attaching to the process for real. All executable and 
> shared library object files (ELF files) from the core file are still on disk 
> so you can get symbols and use the debug info, so LLDB should be able to load 
> all frames up and symbolicate up the crash location. It should be just as 
> good as having the process around without any bad side affects. Core files 
> are less useful if they must be archived and symbolicated later because the 
> executable files might not be around anymore since things like test suites 
> might produce binaries for testing and remove them after the test is run or 
> crashed.
> 
> What do you think about this approach?
> 
> Greg Clayton
> 
> 
>> On Nov 2, 2015, at 5:54 PM, Mark Chandler via lldb-dev 
>>  wrote:
>> 
>> So im trying to write a core handler program and use lldb to attach and dump 
>> important information about it. This works if a use my tool to attach to an 
>> existing one but I found that lldb-server will hang in a waitpid call if the 
>> kernel has invoked the tool after another process has cored.
>> 
>> Example:
>> · /proc/sys/kernel/core_pattern is set to |/opt/core_tool
>> · Run a.out and it segfaults
>> · Kernel invokes core_tool that uses lldb AttachToProcess and a.out 
>> is in state “S+”
>> · lldb-server hangs in 
>> source\Plugins\Process\Linux\NativeProcessLinux.cpp:867
>> · if I remove the waitpid it doesn’t hang but fails to attach
>> 
>> Looks like gdb had a similar problem as well: 
>> http://www.sourceware.org/ml/gdb-patches/2008-04/msg00224.html
>> Any ideas on how to fix this?
>> 
>> Mark Chandler
>> Battle.Net Engineering Systems | Blizzard Entertainment
>> (P) 949-955-1380 x15353
>> 
>> ___
>> lldb-dev mailing list
>> lldb-dev@lists.llvm.org
>> http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev
> 
> ___
> lldb-dev mailing list
> lldb-dev@lists.llvm.org
> http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev

___
lldb-dev mailing list
lldb-dev@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev

Re: [lldb-dev] Attaching to a stopped (cored) process hangs lldb-server

[Greg Clayton via lldb-dev]

Makes sense about not writing the core file to disk. 

Is there a way you can detect this "core" mode where we don't have to waitpid? 
Seems like that www.sourceware.org message had ideas on how to detect this case?

Greg

> On Nov 3, 2015, at 4:36 PM, Mark Chandler  wrote:
> 
> Not able to do that as the servers have no hard drives (use ram disk and net 
> boot) and the tool is trying to avoid a core storm that takes down the 
> network file share. I found out what is causing it to hang, there is a call 
> to waitpid in NativeLinuxProcess.cpp that waits forever. As the process is 
> already stopped, I disabled that and it looks to be working
> 
> Mark Chandler
> Battle.Net Engineering Systems | Blizzard Entertainment
> (P) 949-955-1380 x15353
> 
> -Original Message-
> From: Greg Clayton [mailto:gclay...@apple.com] 
> Sent: Tuesday, November 03, 2015 4:34 PM
> To: Mark Chandler 
> Cc: lldb-dev@lists.llvm.org
> Subject: Re: [lldb-dev] Attaching to a stopped (cored) process hangs 
> lldb-server
> 
> One different approach is to have your tool write all STDIN to a file (the 
> core file comes into the tool as STDIN bytes) and then hand LLDB the core 
> file and do any needed backtracing and data gathering from the core file 
> instead of actually attaching to the process for real. All executable and 
> shared library object files (ELF files) from the core file are still on disk 
> so you can get symbols and use the debug info, so LLDB should be able to load 
> all frames up and symbolicate up the crash location. It should be just as 
> good as having the process around without any bad side affects. Core files 
> are less useful if they must be archived and symbolicated later because the 
> executable files might not be around anymore since things like test suites 
> might produce binaries for testing and remove them after the test is run or 
> crashed.
> 
> What do you think about this approach?
> 
> Greg Clayton
> 
> 
>> On Nov 2, 2015, at 5:54 PM, Mark Chandler via lldb-dev 
>>  wrote:
>> 
>> So im trying to write a core handler program and use lldb to attach and dump 
>> important information about it. This works if a use my tool to attach to an 
>> existing one but I found that lldb-server will hang in a waitpid call if the 
>> kernel has invoked the tool after another process has cored.
>> 
>> Example:
>> · /proc/sys/kernel/core_pattern is set to |/opt/core_tool
>> · Run a.out and it segfaults
>> · Kernel invokes core_tool that uses lldb AttachToProcess and a.out 
>> is in state “S+”
>> · lldb-server hangs in 
>> source\Plugins\Process\Linux\NativeProcessLinux.cpp:867
>> · if I remove the waitpid it doesn’t hang but fails to attach
>> 
>> Looks like gdb had a similar problem as well: 
>> http://www.sourceware.org/ml/gdb-patches/2008-04/msg00224.html
>> Any ideas on how to fix this?
>> 
>> Mark Chandler
>> Battle.Net Engineering Systems | Blizzard Entertainment
>> (P) 949-955-1380 x15353
>> 
>> ___
>> lldb-dev mailing list
>> lldb-dev@lists.llvm.org
>> http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev
> 

___
lldb-dev mailing list
lldb-dev@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev

Re: [lldb-dev] Attaching to a stopped (cored) process hangs lldb-server

[Mark Chandler via lldb-dev]

The biggest tell is that the process state is already 'S' or stopped. I don’t 
know lldb at all to make a change to fix this though.


Mark Chandler
Battle.Net Engineering Systems | Blizzard Entertainment
(P) 949-955-1380 x15353

-Original Message-
From: Greg Clayton [mailto:gclay...@apple.com] 
Sent: Tuesday, November 03, 2015 4:39 PM
To: Mark Chandler 
Cc: lldb-dev@lists.llvm.org
Subject: Re: [lldb-dev] Attaching to a stopped (cored) process hangs lldb-server

Makes sense about not writing the core file to disk. 

Is there a way you can detect this "core" mode where we don't have to waitpid? 
Seems like that www.sourceware.org message had ideas on how to detect this case?

Greg

> On Nov 3, 2015, at 4:36 PM, Mark Chandler  wrote:
> 
> Not able to do that as the servers have no hard drives (use ram disk and net 
> boot) and the tool is trying to avoid a core storm that takes down the 
> network file share. I found out what is causing it to hang, there is a call 
> to waitpid in NativeLinuxProcess.cpp that waits forever. As the process is 
> already stopped, I disabled that and it looks to be working
> 
> Mark Chandler
> Battle.Net Engineering Systems | Blizzard Entertainment
> (P) 949-955-1380 x15353
> 
> -Original Message-
> From: Greg Clayton [mailto:gclay...@apple.com] 
> Sent: Tuesday, November 03, 2015 4:34 PM
> To: Mark Chandler 
> Cc: lldb-dev@lists.llvm.org
> Subject: Re: [lldb-dev] Attaching to a stopped (cored) process hangs 
> lldb-server
> 
> One different approach is to have your tool write all STDIN to a file (the 
> core file comes into the tool as STDIN bytes) and then hand LLDB the core 
> file and do any needed backtracing and data gathering from the core file 
> instead of actually attaching to the process for real. All executable and 
> shared library object files (ELF files) from the core file are still on disk 
> so you can get symbols and use the debug info, so LLDB should be able to load 
> all frames up and symbolicate up the crash location. It should be just as 
> good as having the process around without any bad side affects. Core files 
> are less useful if they must be archived and symbolicated later because the 
> executable files might not be around anymore since things like test suites 
> might produce binaries for testing and remove them after the test is run or 
> crashed.
> 
> What do you think about this approach?
> 
> Greg Clayton
> 
> 
>> On Nov 2, 2015, at 5:54 PM, Mark Chandler via lldb-dev 
>>  wrote:
>> 
>> So im trying to write a core handler program and use lldb to attach and dump 
>> important information about it. This works if a use my tool to attach to an 
>> existing one but I found that lldb-server will hang in a waitpid call if the 
>> kernel has invoked the tool after another process has cored.
>> 
>> Example:
>> · /proc/sys/kernel/core_pattern is set to |/opt/core_tool
>> · Run a.out and it segfaults
>> · Kernel invokes core_tool that uses lldb AttachToProcess and a.out 
>> is in state “S+”
>> · lldb-server hangs in 
>> source\Plugins\Process\Linux\NativeProcessLinux.cpp:867
>> · if I remove the waitpid it doesn’t hang but fails to attach
>> 
>> Looks like gdb had a similar problem as well: 
>> http://www.sourceware.org/ml/gdb-patches/2008-04/msg00224.html
>> Any ideas on how to fix this?
>> 
>> Mark Chandler
>> Battle.Net Engineering Systems | Blizzard Entertainment
>> (P) 949-955-1380 x15353
>> 
>> ___
>> lldb-dev mailing list
>> lldb-dev@lists.llvm.org
>> http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev
> 

___
lldb-dev mailing list
lldb-dev@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev

Re: [lldb-dev] Attaching to a stopped (cored) process hangs lldb-server

[Greg Clayton via lldb-dev]

Can someone with linux experience chime in here? It shouldn't be too hard to 
figure out which flag 'S' is in. On MacOS we can get a process info structure 
from a pid and that will have bits set that indicate 'S'...

If you want to checkin this tool into the LLDB source tree at 
trunk/tools/core_tool then we can get more people to work on it and improve it. 
It would be nice to have this available for all linux users. I would love to 
see an JSON output mode that is parseable by automated tools instead of people 
saving text formats that must be text scraped.

If you can get this into a tool, others can help get this working. Any interest 
in this?

Greg

> On Nov 3, 2015, at 4:41 PM, Mark Chandler  wrote:
> 
> The biggest tell is that the process state is already 'S' or stopped. I don’t 
> know lldb at all to make a change to fix this though.
> 
> 
> Mark Chandler
> Battle.Net Engineering Systems | Blizzard Entertainment
> (P) 949-955-1380 x15353
> 
> -Original Message-
> From: Greg Clayton [mailto:gclay...@apple.com] 
> Sent: Tuesday, November 03, 2015 4:39 PM
> To: Mark Chandler 
> Cc: lldb-dev@lists.llvm.org
> Subject: Re: [lldb-dev] Attaching to a stopped (cored) process hangs 
> lldb-server
> 
> Makes sense about not writing the core file to disk. 
> 
> Is there a way you can detect this "core" mode where we don't have to 
> waitpid? Seems like that www.sourceware.org message had ideas on how to 
> detect this case?
> 
> Greg
> 
>> On Nov 3, 2015, at 4:36 PM, Mark Chandler  wrote:
>> 
>> Not able to do that as the servers have no hard drives (use ram disk and net 
>> boot) and the tool is trying to avoid a core storm that takes down the 
>> network file share. I found out what is causing it to hang, there is a call 
>> to waitpid in NativeLinuxProcess.cpp that waits forever. As the process is 
>> already stopped, I disabled that and it looks to be working
>> 
>> Mark Chandler
>> Battle.Net Engineering Systems | Blizzard Entertainment
>> (P) 949-955-1380 x15353
>> 
>> -Original Message-
>> From: Greg Clayton [mailto:gclay...@apple.com] 
>> Sent: Tuesday, November 03, 2015 4:34 PM
>> To: Mark Chandler 
>> Cc: lldb-dev@lists.llvm.org
>> Subject: Re: [lldb-dev] Attaching to a stopped (cored) process hangs 
>> lldb-server
>> 
>> One different approach is to have your tool write all STDIN to a file (the 
>> core file comes into the tool as STDIN bytes) and then hand LLDB the core 
>> file and do any needed backtracing and data gathering from the core file 
>> instead of actually attaching to the process for real. All executable and 
>> shared library object files (ELF files) from the core file are still on disk 
>> so you can get symbols and use the debug info, so LLDB should be able to 
>> load all frames up and symbolicate up the crash location. It should be just 
>> as good as having the process around without any bad side affects. Core 
>> files are less useful if they must be archived and symbolicated later 
>> because the executable files might not be around anymore since things like 
>> test suites might produce binaries for testing and remove them after the 
>> test is run or crashed.
>> 
>> What do you think about this approach?
>> 
>> Greg Clayton
>> 
>> 
>>> On Nov 2, 2015, at 5:54 PM, Mark Chandler via lldb-dev 
>>>  wrote:
>>> 
>>> So im trying to write a core handler program and use lldb to attach and 
>>> dump important information about it. This works if a use my tool to attach 
>>> to an existing one but I found that lldb-server will hang in a waitpid call 
>>> if the kernel has invoked the tool after another process has cored.
>>> 
>>> Example:
>>> · /proc/sys/kernel/core_pattern is set to |/opt/core_tool
>>> · Run a.out and it segfaults
>>> · Kernel invokes core_tool that uses lldb AttachToProcess and a.out 
>>> is in state “S+”
>>> · lldb-server hangs in 
>>> source\Plugins\Process\Linux\NativeProcessLinux.cpp:867
>>> · if I remove the waitpid it doesn’t hang but fails to attach
>>> 
>>> Looks like gdb had a similar problem as well: 
>>> http://www.sourceware.org/ml/gdb-patches/2008-04/msg00224.html
>>> Any ideas on how to fix this?
>>> 
>>> Mark Chandler
>>> Battle.Net Engineering Systems | Blizzard Entertainment
>>> (P) 949-955-1380 x15353
>>> 
>>> ___
>>> lldb-dev mailing list
>>> lldb-dev@lists.llvm.org
>>> http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev
>> 
> 

___
lldb-dev mailing list
lldb-dev@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev

Re: [lldb-dev] Attaching to a stopped (cored) process hangs lldb-server

[Mark Chandler via lldb-dev]

Ill have to talk to the powers to be but for the most part I could check it in. 
There is an internal json format that the tool saves it into and a bit of it is 
geared towards Blizzard Entertainment server setup though but for the most part 
its pretty simple (under 1000 lines).







Mark Chandler

Battle.Net Engineering Systems | Blizzard Entertainment

(P) 949-955-1380 x15353



-Original Message-
From: Greg Clayton [mailto:gclay...@apple.com]
Sent: Tuesday, November 03, 2015 4:49 PM
To: Mark Chandler 
Cc: lldb-dev@lists.llvm.org
Subject: Re: [lldb-dev] Attaching to a stopped (cored) process hangs lldb-server



Can someone with linux experience chime in here? It shouldn't be too hard to 
figure out which flag 'S' is in. On MacOS we can get a process info structure 
from a pid and that will have bits set that indicate 'S'...



If you want to checkin this tool into the LLDB source tree at 
trunk/tools/core_tool then we can get more people to work on it and improve it. 
It would be nice to have this available for all linux users. I would love to 
see an JSON output mode that is parseable by automated tools instead of people 
saving text formats that must be text scraped.



If you can get this into a tool, others can help get this working. Any interest 
in this?



Greg



> On Nov 3, 2015, at 4:41 PM, Mark Chandler 
> mailto:mchand...@blizzard.com>> wrote:

>

> The biggest tell is that the process state is already 'S' or stopped. I don’t 
> know lldb at all to make a change to fix this though.

>

>

> Mark Chandler

> Battle.Net Engineering Systems | Blizzard Entertainment

> (P) 949-955-1380 x15353

>

> -Original Message-

> From: Greg Clayton [mailto:gclay...@apple.com]

> Sent: Tuesday, November 03, 2015 4:39 PM

> To: Mark Chandler mailto:mchand...@blizzard.com>>

> Cc: lldb-dev@lists.llvm.org

> Subject: Re: [lldb-dev] Attaching to a stopped (cored) process hangs 
> lldb-server

>

> Makes sense about not writing the core file to disk.

>

> Is there a way you can detect this "core" mode where we don't have to 
> waitpid? Seems like that www.sourceware.org 
> message had ideas on how to detect this case?

>

> Greg

>

>> On Nov 3, 2015, at 4:36 PM, Mark Chandler 
>> mailto:mchand...@blizzard.com>> wrote:

>>

>> Not able to do that as the servers have no hard drives (use ram disk and net 
>> boot) and the tool is trying to avoid a core storm that takes down the 
>> network file share. I found out what is causing it to hang, there is a call 
>> to waitpid in NativeLinuxProcess.cpp that waits forever. As the process is 
>> already stopped, I disabled that and it looks to be working

>>

>> Mark Chandler

>> Battle.Net Engineering Systems | Blizzard Entertainment

>> (P) 949-955-1380 x15353

>>

>> -Original Message-

>> From: Greg Clayton [mailto:gclay...@apple.com]

>> Sent: Tuesday, November 03, 2015 4:34 PM

>> To: Mark Chandler mailto:mchand...@blizzard.com>>

>> Cc: lldb-dev@lists.llvm.org

>> Subject: Re: [lldb-dev] Attaching to a stopped (cored) process hangs 
>> lldb-server

>>

>> One different approach is to have your tool write all STDIN to a file (the 
>> core file comes into the tool as STDIN bytes) and then hand LLDB the core 
>> file and do any needed backtracing and data gathering from the core file 
>> instead of actually attaching to the process for real. All executable and 
>> shared library object files (ELF files) from the core file are still on disk 
>> so you can get symbols and use the debug info, so LLDB should be able to 
>> load all frames up and symbolicate up the crash location. It should be just 
>> as good as having the process around without any bad side affects. Core 
>> files are less useful if they must be archived and symbolicated later 
>> because the executable files might not be around anymore since things like 
>> test suites might produce binaries for testing and remove them after the 
>> test is run or crashed.

>>

>> What do you think about this approach?

>>

>> Greg Clayton

>>

>>

>>> On Nov 2, 2015, at 5:54 PM, Mark Chandler via lldb-dev 
>>> mailto:lldb-dev@lists.llvm.org>> wrote:

>>>

>>> So im trying to write a core handler program and use lldb to attach and 
>>> dump important information about it. This works if a use my tool to attach 
>>> to an existing one but I found that lldb-server will hang in a waitpid call 
>>> if the kernel has invoked the tool after another process has cored.

>>>

>>> Example:

>>> · /proc/sys/kernel/core_pattern is set to |/opt/core_tool

>>> · Run a.out and it segfaults

>>> · Kernel invokes core_tool that uses lldb AttachToProcess and a.out 
>>> is in state “S+”

>>> · lldb-server hangs in 
>>> source\Plugins\Process\Linux\NativeProcessLinux.cpp:867

>>> · if I remove the waitpid it doesn’t hang but fails to attach

>>>

>>> Looks like gdb had a similar prob

Re: [lldb-dev] Attaching to a stopped (cored) process hangs lldb-server

[Pavel Labath via lldb-dev]

I'm following this discussion, but I don't yet understand what is
going on here completely. What I am sure is that the problem here is
not the S+ state, as that just means "interruptible sleep, foreground
process", and a lot of processes have that state and we attach to them
just fine. I would need to investigate what are the exact properties
or this cored state. I'll try to take a look when I get some spare
cycles, but that might not happen very soon.

Mark, have you investigated what is the next thing to fail after you
remove the waitpid call?

pl

On 3 November 2015 at 16:48, Greg Clayton via lldb-dev
 wrote:
> Can someone with linux experience chime in here? It shouldn't be too hard to 
> figure out which flag 'S' is in. On MacOS we can get a process info structure 
> from a pid and that will have bits set that indicate 'S'...
>
> If you want to checkin this tool into the LLDB source tree at 
> trunk/tools/core_tool then we can get more people to work on it and improve 
> it. It would be nice to have this available for all linux users. I would love 
> to see an JSON output mode that is parseable by automated tools instead of 
> people saving text formats that must be text scraped.
>
> If you can get this into a tool, others can help get this working. Any 
> interest in this?
>
> Greg
>
>> On Nov 3, 2015, at 4:41 PM, Mark Chandler  wrote:
>>
>> The biggest tell is that the process state is already 'S' or stopped. I 
>> don’t know lldb at all to make a change to fix this though.
>>
>>
>> Mark Chandler
>> Battle.Net Engineering Systems | Blizzard Entertainment
>> (P) 949-955-1380 x15353
>>
>> -Original Message-
>> From: Greg Clayton [mailto:gclay...@apple.com]
>> Sent: Tuesday, November 03, 2015 4:39 PM
>> To: Mark Chandler 
>> Cc: lldb-dev@lists.llvm.org
>> Subject: Re: [lldb-dev] Attaching to a stopped (cored) process hangs 
>> lldb-server
>>
>> Makes sense about not writing the core file to disk.
>>
>> Is there a way you can detect this "core" mode where we don't have to 
>> waitpid? Seems like that www.sourceware.org message had ideas on how to 
>> detect this case?
>>
>> Greg
>>
>>> On Nov 3, 2015, at 4:36 PM, Mark Chandler  wrote:
>>>
>>> Not able to do that as the servers have no hard drives (use ram disk and 
>>> net boot) and the tool is trying to avoid a core storm that takes down the 
>>> network file share. I found out what is causing it to hang, there is a call 
>>> to waitpid in NativeLinuxProcess.cpp that waits forever. As the process is 
>>> already stopped, I disabled that and it looks to be working
>>>
>>> Mark Chandler
>>> Battle.Net Engineering Systems | Blizzard Entertainment
>>> (P) 949-955-1380 x15353
>>>
>>> -Original Message-
>>> From: Greg Clayton [mailto:gclay...@apple.com]
>>> Sent: Tuesday, November 03, 2015 4:34 PM
>>> To: Mark Chandler 
>>> Cc: lldb-dev@lists.llvm.org
>>> Subject: Re: [lldb-dev] Attaching to a stopped (cored) process hangs 
>>> lldb-server
>>>
>>> One different approach is to have your tool write all STDIN to a file (the 
>>> core file comes into the tool as STDIN bytes) and then hand LLDB the core 
>>> file and do any needed backtracing and data gathering from the core file 
>>> instead of actually attaching to the process for real. All executable and 
>>> shared library object files (ELF files) from the core file are still on 
>>> disk so you can get symbols and use the debug info, so LLDB should be able 
>>> to load all frames up and symbolicate up the crash location. It should be 
>>> just as good as having the process around without any bad side affects. 
>>> Core files are less useful if they must be archived and symbolicated later 
>>> because the executable files might not be around anymore since things like 
>>> test suites might produce binaries for testing and remove them after the 
>>> test is run or crashed.
>>>
>>> What do you think about this approach?
>>>
>>> Greg Clayton
>>>
>>>
 On Nov 2, 2015, at 5:54 PM, Mark Chandler via lldb-dev 
  wrote:

 So im trying to write a core handler program and use lldb to attach and 
 dump important information about it. This works if a use my tool to attach 
 to an existing one but I found that lldb-server will hang in a waitpid 
 call if the kernel has invoked the tool after another process has cored.

 Example:
 · /proc/sys/kernel/core_pattern is set to |/opt/core_tool
 · Run a.out and it segfaults
 · Kernel invokes core_tool that uses lldb AttachToProcess and 
 a.out is in state “S+”
 · lldb-server hangs in 
 source\Plugins\Process\Linux\NativeProcessLinux.cpp:867
 · if I remove the waitpid it doesn’t hang but fails to attach

 Looks like gdb had a similar problem as well: 
 http://www.sourceware.org/ml/gdb-patches/2008-04/msg00224.html
 Any ideas on how to fix this?

 Mark Chandler
 Battle.Net Engineering Systems | Blizzard Entertainment
 (P) 949-955-138

Re: [lldb-dev] Attaching to a stopped (cored) process hangs lldb-server

[Mark Chandler via lldb-dev]

The ptrace options per thread id also fail so I removed that as well. Atm 
lldb-server is seg-faulting in ThreadAttach that im trying to work out why.



Mark Chandler
Battle.Net Engineering Systems | Blizzard Entertainment
(P) 949-955-1380 x15353

-Original Message-
From: Pavel Labath [mailto:lab...@google.com] 
Sent: Tuesday, November 03, 2015 5:41 PM
To: Greg Clayton 
Cc: Mark Chandler ; lldb-dev@lists.llvm.org
Subject: Re: [lldb-dev] Attaching to a stopped (cored) process hangs lldb-server

I'm following this discussion, but I don't yet understand what is going on here 
completely. What I am sure is that the problem here is not the S+ state, as 
that just means "interruptible sleep, foreground process", and a lot of 
processes have that state and we attach to them just fine. I would need to 
investigate what are the exact properties or this cored state. I'll try to take 
a look when I get some spare cycles, but that might not happen very soon.

Mark, have you investigated what is the next thing to fail after you remove the 
waitpid call?

pl

On 3 November 2015 at 16:48, Greg Clayton via lldb-dev 
 wrote:
> Can someone with linux experience chime in here? It shouldn't be too hard to 
> figure out which flag 'S' is in. On MacOS we can get a process info structure 
> from a pid and that will have bits set that indicate 'S'...
>
> If you want to checkin this tool into the LLDB source tree at 
> trunk/tools/core_tool then we can get more people to work on it and improve 
> it. It would be nice to have this available for all linux users. I would love 
> to see an JSON output mode that is parseable by automated tools instead of 
> people saving text formats that must be text scraped.
>
> If you can get this into a tool, others can help get this working. Any 
> interest in this?
>
> Greg
>
>> On Nov 3, 2015, at 4:41 PM, Mark Chandler  wrote:
>>
>> The biggest tell is that the process state is already 'S' or stopped. I 
>> don’t know lldb at all to make a change to fix this though.
>>
>>
>> Mark Chandler
>> Battle.Net Engineering Systems | Blizzard Entertainment
>> (P) 949-955-1380 x15353
>>
>> -Original Message-
>> From: Greg Clayton [mailto:gclay...@apple.com]
>> Sent: Tuesday, November 03, 2015 4:39 PM
>> To: Mark Chandler 
>> Cc: lldb-dev@lists.llvm.org
>> Subject: Re: [lldb-dev] Attaching to a stopped (cored) process hangs 
>> lldb-server
>>
>> Makes sense about not writing the core file to disk.
>>
>> Is there a way you can detect this "core" mode where we don't have to 
>> waitpid? Seems like that www.sourceware.org message had ideas on how to 
>> detect this case?
>>
>> Greg
>>
>>> On Nov 3, 2015, at 4:36 PM, Mark Chandler  wrote:
>>>
>>> Not able to do that as the servers have no hard drives (use ram disk 
>>> and net boot) and the tool is trying to avoid a core storm that 
>>> takes down the network file share. I found out what is causing it to 
>>> hang, there is a call to waitpid in NativeLinuxProcess.cpp that 
>>> waits forever. As the process is already stopped, I disabled that 
>>> and it looks to be working
>>>
>>> Mark Chandler
>>> Battle.Net Engineering Systems | Blizzard Entertainment
>>> (P) 949-955-1380 x15353
>>>
>>> -Original Message-
>>> From: Greg Clayton [mailto:gclay...@apple.com]
>>> Sent: Tuesday, November 03, 2015 4:34 PM
>>> To: Mark Chandler 
>>> Cc: lldb-dev@lists.llvm.org
>>> Subject: Re: [lldb-dev] Attaching to a stopped (cored) process hangs 
>>> lldb-server
>>>
>>> One different approach is to have your tool write all STDIN to a file (the 
>>> core file comes into the tool as STDIN bytes) and then hand LLDB the core 
>>> file and do any needed backtracing and data gathering from the core file 
>>> instead of actually attaching to the process for real. All executable and 
>>> shared library object files (ELF files) from the core file are still on 
>>> disk so you can get symbols and use the debug info, so LLDB should be able 
>>> to load all frames up and symbolicate up the crash location. It should be 
>>> just as good as having the process around without any bad side affects. 
>>> Core files are less useful if they must be archived and symbolicated later 
>>> because the executable files might not be around anymore since things like 
>>> test suites might produce binaries for testing and remove them after the 
>>> test is run or crashed.
>>>
>>> What do you think about this approach?
>>>
>>> Greg Clayton
>>>
>>>
 On Nov 2, 2015, at 5:54 PM, Mark Chandler via lldb-dev 
  wrote:

 So im trying to write a core handler program and use lldb to attach and 
 dump important information about it. This works if a use my tool to attach 
 to an existing one but I found that lldb-server will hang in a waitpid 
 call if the kernel has invoked the tool after another process has cored.

 Example:
 · /proc/sys/kernel/core_pattern is set to |/opt/core_tool
 · Run a.out and it segfaults
 · Kernel