Re: [lldb-dev] [cfe-dev] [RFC] Improving import failure checking strategy inside ASTImporter
An other idea, couldn't we just pass the cast operation as a template template parameter? ___ lldb-dev mailing list lldb-dev@lists.llvm.org http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev
Re: [lldb-dev] [cfe-dev] [RFC] Improving import failure checking strategy inside ASTImporter
Hi Aleksei, I think it is a very good idea to exploit [[nodiscard]] . On the other hand I find a bit misleading some of the names of these functions. `importCastedNonNull` does a simple `cast<>` inside, thus perhaps a name `importAndCast` would better explain what happens. Similarly, instead of `importCasted`, `importAndCastOrNull` (import_and_cast_or_null ?) sounds better for me, what do you think? Gabor ___ lldb-dev mailing list lldb-dev@lists.llvm.org http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev
Re: [lldb-dev] [cfe-dev] [RFC] Improving import failure checking strategy inside ASTImporter
Of course, I think there is no need to have different functions which
differ only in the cast operation. We can use a template template parameter
for the cast operation:
template class CastOp, typename DeclTy>
LLVM_NODISCARD bool import(DeclTy *&ToD, DeclTy *FromD) {
if (auto Res = Importer.Import(FromD)) {
ToD = CastOp(*Res);
return false;
}
return true;
}
And then the usage:
if (import(ToEPI.ExceptionSpec.SourceDecl,
FromEPI.ExceptionSpec.SourceDecl))
return {};
Gabor
On Fri, May 18, 2018 at 8:52 PM Aleksei Sidorin
wrote:
>
> Sorry, I didn't got the idea. Could please you explain a bit more?
>
> 18.05.2018 13:09, Gábor Márton via cfe-dev пишет:
> > An other idea, couldn't we just pass the cast operation as a template
> > template parameter?
> > ___
> > cfe-dev mailing list
> > cfe-...@lists.llvm.org
> > http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev
>
>
> --
> Best regards,
> Aleksei Sidorin,
> SRR, Samsung Electronics
>
___
lldb-dev mailing list
lldb-dev@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev
Re: [lldb-dev] [cfe-dev] [RFC] Improving import failure checking strategy inside ASTImporter
> But there are two issues. First, cast<> has two template arguments, not
one. Second, template instantiation cannot guess what overload of cast
should be used in each case. I'm sorry, but after a few attempts of making
this code to compile, I have to ask you to elaborate a bit more once again.
You are right, we can't just pass a template function as a template
parameter. However, we could use a lambda:
template
Optional import(CastOpTy CastOp, DeclTy *&ToD, DeclTy *FromD)
{
if (auto Res = Importer.Import(FromD)) {
return CastOp(Res);
}
return None;
}
template
Optional importNullable(DeclTy *&ToD, DeclTy *FromD) {
return import([](Decl* R){ return cast_or_null(R); }, ToD,
FromD);
}
This approach still has disadvantages: we have to use a concrete type in
the lambda parameter (`Decl`) because polymorphic lambdas are not supported
in C++11.
So I am not sure if this is really worth it to avoid just some very few
lines of repetition.
> Regarding naming: what do you think about importNonNull[Into]() and
importNullable[Into]()?
Sounds good to me.
> There is also another thing I am curious about: do we need to use
dyn_casts on the return type? I'm not sure we can return a node with type
so different from the imported node's type. Currently, there are some
dyn_cast[_or_null]s in our code, but, possibly, they can be replaced with
just casts, because we usually return nullptr or a pointer of same type as
the imported one.
I agree. Immediately after an import I think it is perfectly legitimate to
assume that the imported node has the very same type.
Gabor
___
lldb-dev mailing list
lldb-dev@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev
[lldb-dev] Linux: Failing lldb unit tests and TestLinuxCore.py
Hi By using "ninja check-lldb-unit" some of the unit tests fail with the tip of the master. (I am using Ubuntu Linux.) The error is always related to the CommandLine options: FAIL: lldb-Unit :: ScriptInterpreter/Python/./ScriptInterpreterPythonTests/PythonDataObjectsTest.TestBorrowedReferences (120 of 363) TEST 'lldb-Unit :: ScriptInterpreter/Python/./ScriptInterpreterPythonTests/PythonDataObjectsTest.TestBorrowedReferences' FAILED Note: Google Test filter = PythonDataObjectsTest.TestBorrowedReferences [==] Running 1 test from 1 test case. [--] Global test environment set-up. [--] 1 test from PythonDataObjectsTest [ RUN ] PythonDataObjectsTest.TestBorrowedReferences : CommandLine Error: Option 'help-list' registered more than once! LLVM ERROR: inconsistency in registered CommandLine options The other lit tests pass, except TestLinuxCore.py. I already set up LD_LIBRARY_PATH to point to $build/bin. Am I missing something else? Any help would be appreciated. Thanks, Gábor ___ lldb-dev mailing list lldb-dev@lists.llvm.org http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev
Re: [lldb-dev] Linux: Failing lldb unit tests and TestLinuxCore.py
Hi Pavel, I do an in-tree build, so lldb is next to clang. And I use ccache, could that be a problem? If yes then why just with the unit tests? Here is how I configured cmake: cmake ../../git/llvm -G Ninja -DCMAKE_EXPORT_COMPILE_COMMANDS=1 -DCMAKE_BUILD_TYPE=Release -DCMAKE_CXX_COMPILER=clang++ -DCMAKE_C_COMPILER=clang -DBUILD_SHARED_LIBS=1 -DLLVM_TARGETS_TO_BUILD=X86 -DLLVM_ENABLE_ASSERTIONS=1 Thanks, Gabor On Tue, Jun 26, 2018 at 5:05 PM Pavel Labath wrote: > Hello Gabor, > > The error message (Option 'help-list' registered more than once) leads > me to believe that this is caused by an uncommon (for lldb, at least) > build configuration. Can you share your cmake configuration? > > I suspect TestLinuxCore is also failing due to unexpected build > configuration, specifically missing targets in llvm (mips?). I'll try > to fix that this week. > On Tue, 26 Jun 2018 at 14:44, Gábor Márton via lldb-dev > wrote: > > > > Hi > > > > By using "ninja check-lldb-unit" some of the unit tests fail with the > tip of the master. (I am using Ubuntu Linux.) The error is always related > to the CommandLine options: > > > > FAIL: lldb-Unit :: > ScriptInterpreter/Python/./ScriptInterpreterPythonTests/PythonDataObjectsTest.TestBorrowedReferences > (120 of 363) > > TEST 'lldb-Unit :: > ScriptInterpreter/Python/./ScriptInterpreterPythonTests/PythonDataObjectsTest.TestBorrowedReferences' > FAILED > > Note: Google Test filter = PythonDataObjectsTest.TestBorrowedReferences > > [==] Running 1 test from 1 test case. > > [--] Global test environment set-up. > > [--] 1 test from PythonDataObjectsTest > > [ RUN ] PythonDataObjectsTest.TestBorrowedReferences > > : CommandLine Error: Option 'help-list' registered more than once! > > LLVM ERROR: inconsistency in registered CommandLine options > > > > The other lit tests pass, except TestLinuxCore.py. > > > > I already set up LD_LIBRARY_PATH to point to $build/bin. > > Am I missing something else? > > > > Any help would be appreciated. > > > > Thanks, > > Gábor > > ___ > > lldb-dev mailing list > > lldb-dev@lists.llvm.org > > http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev > ___ lldb-dev mailing list lldb-dev@lists.llvm.org http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev
Re: [lldb-dev] Linux: Failing lldb unit tests and TestLinuxCore.py
m of it if you're interested, but it's > going to take some interactive debugging (for start I'd need to see > the link command for the test binary in question). > > cheers, > pl > On Tue, 26 Jun 2018 at 16:33, Pavel Labath wrote: > > > > The BUILD_SHARED_LIBS part is likely to be the issue. LLDB libraries > > have cyclic dependencies, which don't play well with shared libraries. > > I think somebody "fixed" it some time ago (i.e., just made the flag be > > ignored for lldb), but I'm not sure if he also checked the unit tests. > > I'll see if I can reproduce that on my end. > > On Tue, 26 Jun 2018 at 16:28, Gábor Márton > wrote: > > > > > > Hi Pavel, > > > > > > I do an in-tree build, so lldb is next to clang. > > > And I use ccache, could that be a problem? If yes then why just with > the unit tests? > > > Here is how I configured cmake: > > > cmake ../../git/llvm -G Ninja -DCMAKE_EXPORT_COMPILE_COMMANDS=1 > -DCMAKE_BUILD_TYPE=Release -DCMAKE_CXX_COMPILER=clang++ > -DCMAKE_C_COMPILER=clang -DBUILD_SHARED_LIBS=1 -DLLVM_TARGETS_TO_BUILD=X86 > -DLLVM_ENABLE_ASSERTIONS=1 > > > > > > Thanks, > > > Gabor > > > > > > On Tue, Jun 26, 2018 at 5:05 PM Pavel Labath > wrote: > > >> > > >> Hello Gabor, > > >> > > >> The error message (Option 'help-list' registered more than once) leads > > >> me to believe that this is caused by an uncommon (for lldb, at least) > > >> build configuration. Can you share your cmake configuration? > > >> > > >> I suspect TestLinuxCore is also failing due to unexpected build > > >> configuration, specifically missing targets in llvm (mips?). I'll try > > >> to fix that this week. > > >> On Tue, 26 Jun 2018 at 14:44, Gábor Márton via lldb-dev > > >> wrote: > > >> > > > >> > Hi > > >> > > > >> > By using "ninja check-lldb-unit" some of the unit tests fail with > the tip of the master. (I am using Ubuntu Linux.) The error is always > related to the CommandLine options: > > >> > > > >> > FAIL: lldb-Unit :: > ScriptInterpreter/Python/./ScriptInterpreterPythonTests/PythonDataObjectsTest.TestBorrowedReferences > (120 of 363) > > >> > TEST 'lldb-Unit :: > ScriptInterpreter/Python/./ScriptInterpreterPythonTests/PythonDataObjectsTest.TestBorrowedReferences' > FAILED > > >> > Note: Google Test filter = > PythonDataObjectsTest.TestBorrowedReferences > > >> > [==] Running 1 test from 1 test case. > > >> > [--] Global test environment set-up. > > >> > [--] 1 test from PythonDataObjectsTest > > >> > [ RUN ] PythonDataObjectsTest.TestBorrowedReferences > > >> > : CommandLine Error: Option 'help-list' registered more than once! > > >> > LLVM ERROR: inconsistency in registered CommandLine options > > >> > > > >> > The other lit tests pass, except TestLinuxCore.py. > > >> > > > >> > I already set up LD_LIBRARY_PATH to point to $build/bin. > > >> > Am I missing something else? > > >> > > > >> > Any help would be appreciated. > > >> > > > >> > Thanks, > > >> > Gábor > > >> > ___ > > >> > lldb-dev mailing list > > >> > lldb-dev@lists.llvm.org > > >> > http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev > ___ lldb-dev mailing list lldb-dev@lists.llvm.org http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev
Re: [lldb-dev] Linux: Failing lldb unit tests and TestLinuxCore.py
r/lib/python2.7/dist-packages/lldb/../lib/libLLVM-4.0.so.1",
O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
open("/etc/ld.so.cache", O_RDONLY|O_CLOEXEC) = 5
open("/usr/lib/x86_64-linux-gnu/libLLVM-4.0.so.1", O_RDONLY|O_CLOEXEC) = 5
open("/usr/lib/x86_64-linux-gnu/../lib/tls/x86_64/libffi.so.6",
O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
open("/usr/lib/x86_64-linux-gnu/../lib/tls/libffi.so.6",
O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
open("/usr/lib/x86_64-linux-gnu/../lib/x86_64/libffi.so.6",
O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
open("/usr/lib/x86_64-linux-gnu/../lib/libffi.so.6", O_RDONLY|O_CLOEXEC) =
-1 ENOENT (No such file or directory)
open("/usr/lib/x86_64-linux-gnu/libffi.so.6", O_RDONLY|O_CLOEXEC) = 5
: CommandLine Error: Option 'help-list' registered more than once!
LLVM ERROR: inconsistency in registered CommandLine options
+++ exited with 1 +++
On Fri, Jun 29, 2018 at 12:41 PM Pavel Labath wrote:
> Hi Gábor,
>
> thanks for sending me that link line. Unfortunately, I don't see
> anything immediately obvious there. (I was expecting there would be
> something pulling in LLVMSupport twice, but I don't see anything like
> that there).
>
> To fix this, we need to figure out where is the second definition of
> this option is coming from (the first one should be in
> libLLVMSupport.so). After that, it should be relatively
> straightforward to fix the cmake project to avoid that.
>
> The variable declaring this command option is called HLOp (in
> CommandLine.cpp), so one of the ways to do that is to make find that
> out is to search for the _ZL4HLOp symbol in the symtab of the shared
> libraries (you should not even need debug info for that). Can you
> check which of the libraries in your build folder contain this symbol?
>
> On Fri, 29 Jun 2018 at 09:13, Gábor Márton
> wrote:
> >
> > Hi Pavel,
> >
> > Thank you for the fix and for taking care of this!
> > So, the particular test binary is: ScriptInterpreterPythonTests .
> > And here is the link command:
> > ) ninja ScriptInterpreterPythonTests -v
> > [1/1] : && /usr/lib/ccache/clang++ -fPIC -fvisibility-inlines-hidden
> -Werror=date-time -std=c++11 -Wall -Wextra -Wno-unused-parameter
> -Wwrite-strings -Wcast-qual -Wmissing-field-initializers -pedantic
> -Wno-long-long -Wcovered-switch-default -Wnon-virtual-dtor
> -Wdelete-non-virtual-dtor -Wstring-conversion -fdiagnostics-color
> -ffunction-sections -fdata-sections -Wno-deprecated-declarations
> -Wno-unknown-pragmas -Wno-strict-aliasing -Wno-deprecated-register
> -Wno-vla-extension -O3 -Wl,-allow-shlib-undefined -Wl,-O3
> -Wl,--gc-sections
> tools/lldb/unittests/ScriptInterpreter/Python/CMakeFiles/ScriptInterpreterPythonTests.dir/PythonDataObjectsTests.cpp.o
> tools/lldb/unittests/ScriptInterpreter/Python/CMakeFiles/ScriptInterpreterPythonTests.dir/PythonExceptionStateTests.cpp.o
> tools/lldb/unittests/ScriptInterpreter/Python/CMakeFiles/ScriptInterpreterPythonTests.dir/PythonTestSuite.cpp.o
> -o
> tools/lldb/unittests/ScriptInterpreter/Python/ScriptInterpreterPythonTests
> -Wl,-rpath,/home/egbomrt/WORK/llvm3/build/release_assert/lib -lpthread
> lib/libgtest_main.so.7svn lib/libgtest.so.7svn -lpthread lib/liblldbHost.a
> lib/liblldbPluginScriptInterpreterPython.a /usr/lib/x86_64-linux-gnu/
> libpython2.7.so lib/liblldbHost.a lib/liblldbCore.a lib/liblldbSymbol.a
> lib/liblldbTarget.a lib/liblldbBreakpoint.a lib/liblldbDataFormatters.a
> lib/liblldbInterpreter.a lib/liblldbExpression.a
> lib/liblldbPluginProcessUtility.a lib/liblldbPluginCPlusPlusLanguage.a
> lib/liblldbPluginObjCLanguage.a lib/liblldbPluginExpressionParserClang.a
> lib/liblldbPluginExpressionParserGo.a lib/liblldbPluginSymbolFileDWARF.a
> lib/liblldbPluginSymbolFilePDB.a lib/liblldbCommands.a
> lib/liblldbPluginObjectFileJIT.a lib/liblldbPluginAppleObjCRuntime.a
> lib/liblldbHost.a lib/liblldbCore.a lib/liblldbSymbol.a lib/liblldbTarget.a
> lib/liblldbBreakpoint.a lib/liblldbDataFormatters.a
> lib/liblldbInterpreter.a lib/liblldbExpression.a
> lib/liblldbPluginProcessUtility.a lib/liblldbPluginCPlusPlusLanguage.a
> lib/liblldbPluginObjCLanguage.a lib/liblldbPluginExpressionParserClang.a
> lib/liblldbPluginExpressionParserGo.a lib/liblldbPluginSymbolFileDWARF.a
> lib/liblldbPluginSymbolFilePDB.a lib/liblldbCommands.a
> lib/liblldbPluginObjectFileJIT.a lib/liblldbPluginAppleObjCRuntime.a
> lib/liblldbHost.a lib/liblldbCore.a lib/liblldbSymbol.a lib/liblldbTarget.a
> lib/liblldbBreakpoint.a lib/liblldbDataFormatters.a
> lib/liblldbInterpreter.a lib/liblldbExpression.a
> lib/liblldbPluginProcessUtility.a lib/liblldbPluginCPlusPlusLanguage.a
Re: [lldb-dev] Linux: Failing lldb unit tests and TestLinuxCore.py
So, on Ubuntu "sudo apt-get remove python-lldb-4.0" solved this issue.
Thanks again for the guidance.
Cheers,
Gabor
On Fri, Jun 29, 2018 at 4:10 PM Gábor Márton wrote:
> I did a search for the "HLOp" symbol and it turned out it is only used by
> the "LLVMSupport.so" lib.
> So, this did not help, thus I dug deeper and examined the test binary
> further.
>
> `_exit` is called during the initialization of the static HLOp object
> (dlopen() calls call_init() which executes the static initialization of the
> object).
> But for some strange reason it turned out the init function
> (llvm::cl::OptionCategory::registerCategory()) is called
> from /usr/lib/x86_64-linux-gnu/libLLVM-4.0.so.1 .
> So then I used strace to find out that first "libLLVMSupport.so.7" is
> loaded by the dynamic loader and then later the "
> /usr/lib/x86_64-linux-gnu/libLLVM-4.0.so.1".
> Both have the `registerCategory()` function called during the load of the
> lib and the second call causes the error.
> The sequence of the `open` syscalls shows that my local python
> installation brings in my local lldb lib which in turn loads the local
> libLLVM lib. And this is the problem. Below attached the detailed log of
> the open calls.
> One solution is to uninstall my local lldb or use a different python, or
> use a virtual env for python - which does not contain any lldb modules -
> when running these tests.
> Perhaps, somehow we could discover with cmake that the local python
> install contains lldb and issue a diagnostic in that case, but this seems
> quite complicated.
>
> Thanks,
> Gabor
>
> open("/home/egbomrt/WORK/llvm3/build/release_assert/lib/tls/x86_64/libgtest_main.so.7",
> O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
> open("/home/egbomrt/WORK/llvm3/build/release_assert/lib/tls/libgtest_main.so.7",
> O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
> open("/home/egbomrt/WORK/llvm3/build/release_assert/lib/x86_64/libgtest_main.so.7",
> O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
> open("/home/egbomrt/WORK/llvm3/build/release_assert/lib/libgtest_main.so.7",
> O_RDONLY|O_CLOEXEC) = 3
> open("/home/egbomrt/WORK/llvm3/build/release_assert/lib/libgtest.so.7",
> O_RDONLY|O_CLOEXEC) = 3
> open("/home/egbomrt/WORK/llvm3/build/release_assert/lib/libpython2.7.so.1.0",
> O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
> open("/etc/ld.so.cache", O_RDONLY|O_CLOEXEC) = 3
> open("/usr/lib/x86_64-linux-gnu/libpython2.7.so.1.0", O_RDONLY|O_CLOEXEC)
> = 3
> open("/home/egbomrt/WORK/llvm3/build/release_assert/lib/libLLVMDemangle.so.7",
> O_RDONLY|O_CLOEXEC) = 3
> open("/home/egbomrt/WORK/llvm3/build/release_assert/lib/libclangFrontend.so.7",
> O_RDONLY|O_CLOEXEC) = 3
> open("/home/egbomrt/WORK/llvm3/build/release_assert/lib/libLLVMExecutionEngine.so.7",
> O_RDONLY|O_CLOEXEC) = 3
> open("/home/egbomrt/WORK/llvm3/build/release_assert/lib/libLLVMCore.so.7",
> O_RDONLY|O_CLOEXEC) = 3
> open("/home/egbomrt/WORK/llvm3/build/release_assert/lib/libLLVMRuntimeDyld.so.7",
> O_RDONLY|O_CLOEXEC) = 3
> open("/home/egbomrt/WORK/llvm3/build/release_assert/lib/libclangCodeGen.so.7",
> O_RDONLY|O_CLOEXEC) = 3
> open("/home/egbomrt/WORK/llvm3/build/release_assert/lib/libclangDriver.so.7",
> O_RDONLY|O_CLOEXEC) = 3
> open("/home/egbomrt/WORK/llvm3/build/release_assert/lib/libclangEdit.so.7",
> O_RDONLY|O_CLOEXEC) = 3
> open("/home/egbomrt/WORK/llvm3/build/release_assert/lib/libclangParse.so.7",
> O_RDONLY|O_CLOEXEC) = 3
> open("/home/egbomrt/WORK/llvm3/build/release_assert/lib/libclangRewrite.so.7",
> O_RDONLY|O_CLOEXEC) = 3
> open("/home/egbomrt/WORK/llvm3/build/release_assert/lib/libclangSema.so.7",
> O_RDONLY|O_CLOEXEC) = 3
> open("/home/egbomrt/WORK/llvm3/build/release_assert/lib/libLLVMMCJIT.so.7",
> O_RDONLY|O_CLOEXEC) = 3
> open("/home/egbomrt/WORK/llvm3/build/release_assert/lib/libclangBasic.so.7",
> O_RDONLY|O_CLOEXEC) = 3
> open("/home/egbomrt/WORK/llvm3/build/release_assert/lib/libLLVMDebugInfoDWARF.so.7",
> O_RDONLY|O_CLOEXEC) = 3
> open("/home/egbomrt/WORK/llvm3/build/release_assert/lib/libclangLex.so.7",
> O_RDONLY|O_CLOEXEC) = 3
> open("/home/egbomrt/WORK/llvm3/build/release_assert/lib/libLLVMDebugInfoPDB.so.7",
> O_RDONLY|O_CLOEXEC) = 3
> open("/home/egbomrt/WORK/llvm3/build/release_assert/lib/libLLVMSupport.so.7",
> O_RDONLY|O_CLOEXEC) = 3
> open("/home/egbomrt/WORK/llvm3/build/release_assert/lib/libpthread.so.0",
> O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
> open("/lib/x86_64-linux-gnu/libpthread.so.0", O_RDONLY|O_CLOEXEC) = 3
> open("/home/egbomrt/WORK/llvm3/build/release_assert/lib/libdl.so.2",
> O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
> open("/lib/x86_64-linux-gnu/libdl.so.2", O_RDONLY|O_CLOEXEC) = 3
> open("/home/egbomrt/WORK/llvm3/build/release_assert/lib/libedit.so.2",
> O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
> open("/usr/lib/x86_64-linux-gnu/libedit.so.2", O_RDONLY|O_CLOEXEC) = 3
> open("/home/egbomrt/WORK/llvm3/build/release_assert/
[lldb-dev] run tests with ASAN (dotest.py)
Hi,
I'd like to run the tests, when LLVM, Clang and LLDB are all built with ASAN.
I am using release_60 version of LLDB.
The unittests just run fine, but with `dotest.py`, there is an error.
Maybe the LLDB build does not respect the global -DLLVM_USE_SANITIZER
flag of cmake ?
Any help would be appreciated,
Thanks,
Gabor
```
/var/jenkins_home/workspace/ctu_pipeline/llvm/tools/lldb/test/dotest.py
--executable /var/jenkins_home/workspace/ctu_pipeline/build/bin/lldb
/var/jenkins_home/workspace/ctu_pipeline/llvm/tools/lldb/packages/Python/lldbsuite/test/
-v --excluded ./lldb_test_exclude
lldb version 6.0.0 (https://github.com/llvm-mirror/lldb.git revision
b6df24ff1b258b18041161b8f32ac316a3b5d8d9)
clang revision 64eed461cdd3705e7bc1ccc95df9858f7fe216a8
llvm revision 089d4c0c490687db6c75f1d074e99c4d42936a50
['/var/jenkins_home/workspace/ctu_pipeline/llvm/tools/lldb/test/dotest.py',
'--executable',
'/var/jenkins_home/workspace/ctu_pipeline/build/bin/lldb',
'/var/jenkins_home/workspace/ctu_pipeline/llvm/tools/lldb/packages/Python/lldbsuite/test/',
'-v', '--excluded', './lldb_test_exclude']
LLDB library dir: /var/jenkins_home/workspace/ctu_pipeline/build/bin
LLDB import library dir: /var/jenkins_home/workspace/ctu_pipeline/build/bin
The 'lldb-mi' executable cannot be located. The lldb-mi tests can not
be run as a result.
Traceback (most recent call last):
File
"/var/jenkins_home/workspace/ctu_pipeline/llvm/tools/lldb/test/dotest.py",
line 7, in
lldbsuite.test.run_suite()
File
"/var/jenkins_home/workspace/ctu_pipeline/llvm/tools/lldb/packages/Python/lldbsuite/test/dotest.py",
line 1129, in run_suite
import lldb
File
"/var/jenkins_home/workspace/ctu_pipeline/build/lib/python2.7/site-packages/lldb/__init__.py",
line 53, in
_lldb = swig_import_helper()
File
"/var/jenkins_home/workspace/ctu_pipeline/build/lib/python2.7/site-packages/lldb/__init__.py",
line 49, in swig_import_helper
_mod = imp.load_module('_lldb', fp, pathname, description)
ImportError:
/var/jenkins_home/workspace/ctu_pipeline/build/lib/libLLVMDemangle.so.6:
undefined symbol: __asan_option_detect_stack_use_after_return
```
___
lldb-dev mailing list
lldb-dev@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev
Re: [lldb-dev] run tests with ASAN (dotest.py)
Hi Vedant and Pavel, Thanks for your reply. I agree that asan may report false positives in case of leak detection: E.g. library A (instrumented) allocates memory, while library B (not instrumented by asan) is responsible for deallocation. However, my goal is to discover memory corruption errors around the ASTImporter. For that it is enough if all llvm and clang libs are sanitized, plus liblldb.so too; I don't think that we need to sanitize python as well. So what I have discovered so far is that the lldb binary itself statically links the asan library: ``` nm -C bin/lldb | ag __asan_op 0098cdf0 B __asan_option_detect_stack_use_after_return ``` In the meanwhile the liblldb.so is not: ``` nm -C ./lib/liblldb.so | ag __asan_op U __asan_option_detect_stack_use_after_return ``` liblldb.so is then loaded by SWIG from dotest.py and that causes the error. With LD_PRELOAD yes I can preload one libasan.so but it is not trivial to find the compatible ASAN runtime. When I could find the compatible ASAN runtime then I couldn't get rid of an annoying error: ``` python dotest.py --executable ~/WORK/llvm2/build/debug_san/bin/lldb ../packages/Python/lldbsuite/test/ -v --excluded ~/lldb_test_exclude --skip-category dataformatters --skip-category flakey --skip-category lldb-mi ['dotest.py', '--executable', '/home/egbomrt/WORK/llvm2/build/debug_san/bin/lldb', '../packages/Python/lldbsuite/test/', '-v', '--excluded', '/home/egbomrt/lldb_test_exclude', '--skip-category', 'dataformatters', '--skip-category', 'flakey', '--skip-category', 'lldb-mi'] LLDB library dir: /home/egbomrt/WORK/llvm2/build/debug_san/bin LLDB import library dir: /home/egbomrt/WORK/llvm2/build/debug_san/bin ==25952==The following global variable is not properly aligned. ==25952==This may happen if another global with the same name ==25952==resides in another non-instrumented module. ==25952==Or the global comes from a C file built w/o -fno-common. ==25952==In either case this is likely an ODR violation bug, ==25952==but AddressSanitizer can not provide more details. = ==25952==ERROR: AddressSanitizer: odr-violation (0x7f28e415f713): [1] size=1 'llvm::ErrorInfoBase::ID' ../../git/llvm/lib/Support/Error.cpp:52:21 [2] size=1 'llvm::ErrorInfoBase::ID' ../../git/llvm/lib/Support/Error.cpp:52:21 These globals were registered at these points: [1]: #0 0x4345f0 (/home/egbomrt/WORK/llvm2/build/debug_san/bin/lldb+0x4345f0) #1 0x7f28e0a523bb (/home/egbomrt/WORK/llvm2/build/debug_san/bin/../lib/libLLVMSupport.so.6+0x4fe3bb) [2]: #0 0x4345f0 (/home/egbomrt/WORK/llvm2/build/debug_san/bin/lldb+0x4345f0) #1 0x7f28e0a523bb (/home/egbomrt/WORK/llvm2/build/debug_san/bin/../lib/libLLVMSupport.so.6+0x4fe3bb) ==25952==HINT: if you don't care about these errors you may set ASAN_OPTIONS=detect_odr_violation=0 SUMMARY: AddressSanitizer: odr-violation: global 'llvm::ErrorInfoBase::ID' at ../../git/llvm/lib/Support/Error.cpp:52:21 ``` Even if I exported ASAN_OPTIONS=detect_odr_violation=0 the error remained. So, this is where I gave up. Maybe one other workaround could be to link statically the whole LLVM project... ,but that would take too much time in our CI. Probably we will run only the lldb unittests with asan turned on. GaborOn Fri, Aug 3, 2018 at 11:31 AM Pavel Labath wrote: > > When I looked into this in the past (two years ago), the problem was > that the sanitizer runtimes were expected to be linked into the main > executable. For the dotest tests, the main executable is "python", so > unless you have built an asanified python, you will not have them. > > You might be able to get them loaded via some LD_PRELOAD tricks, but I > am not sure if the overall result will be worth the trouble. In > general, the sanitizers expect that your whole process is sanitized, > and they tend to report a lot of false positives if that is not the > case. > On Fri, 3 Aug 2018 at 00:31, Vedant Kumar via lldb-dev > wrote: > > > > Hi Gábor, > > > > That symbol appears to be defined in compiler-rt/lib/asan/asan_rtl.cc, so > > it should be provided by the asan runtime. Are you sure that the runtime > > expected by the host compiler is being dynamically loaded here? You can > > check this using 'ldd' (IIRC) on Linux or 'otool -l' on Darwin. Also, did > > you take the extra step needed to preload the runtime > > (LD_PRELOAD/DYLD_INSERT_LIBRARIES)? > > > > best, > > vedant > > > > > > On Aug 2, 2018, at 12:24 PM, Gábor Márton via lldb-dev > > wrote: > > > > Hi, > > > > I'd like to run the tests, when L
Re: [lldb-dev] stable layout bug for imported record decls.
Hi Lang, Alexey,
I dug deeper into this and it seems like the issue happens only when a
**minimal** import is used. LLDB uses the minimal import. CrossTU
static analyzer uses the normal mode.
In normal import mode, in `ImportDeclContext` we do import all the
methods in the correct order. However, in minimal mode we early return
before importing the methods.
So by merging D44100 this particular problem will not be solved.
Still, that patch sounds very reasonable and I support that we should
reorder all imported Decls, not just fields.
One trivial solution would be to change `ImporDeclContext` to behave
the same way in both modes. This is somewhat similar to the "brute
force" method you mentioned.
I am not an LLDB expert, so I am not sure if this is acceptable, and
really don't know how many LLDB tests would it break, but this seems
the simplest solution (and preferred) to me.
The other solution if you'd like to keep the minimal behavior is the
index based solution (as you mentioned).
You should compare the order of all the imported methods (and fields)
to the order in the original class in the "From" context. And I
believe you have to do that at the end of VisitFunctionDecl. It would
not work if you did that check when the type become complete, since in
minimal mode we never set the class to be incomplete.
I have created a unit test case, which fails in minimal mode and
succeeds in normal mode. You can change the mode in
`ASTImporterTestBase::TU::lazyInitImporter`.
If you provide a patch then could you please also add this test (or
similar) for both normal and minimal mode?
```
TEST_P(ASTImporterTestBase, OrderOfVirtualMethods) {
auto Code =
R"(
class Base {
public:
virtual void bar() {}
virtual void foo() {}
};
class Derived : public Base {
public:
void foo() override {}
};
)";
Decl *FromTU = getTuDecl(Code, Lang_CXX11, "input0.cc");
auto *DerivedFoo = FirstDeclMatcher().match(
FromTU, functionDecl(hasName("foo"),
hasParent(cxxRecordDecl(hasName("Derived");
auto *BaseBar = FirstDeclMatcher().match(
FromTU, functionDecl(hasName("bar"),
hasParent(cxxRecordDecl(hasName("Base");
Import(DerivedFoo, Lang_CXX);
// Importing Base::bar explicitly is needed only in minimal mode,
// in normal mode we already imported all methods of Base.
Import(BaseBar, Lang_CXX);
Decl *ToTU = ToAST->getASTContext().getTranslationUnitDecl();
auto *ImportedBase = FirstDeclMatcher().match(
ToTU, cxxRecordDecl(hasName("Base")));
MatchVerifier Verifier;
EXPECT_TRUE(Verifier.match(ImportedBase,
cxxRecordDecl(hasMethodOrder({"bar", "foo"};
}
```
Thanks,
Gabor
On Fri, Aug 10, 2018 at 3:47 PM Alexey Sidorin wrote:
>
> (+ Gabor and Gabor)
>
> Hi Lang,
>
> We faced a very similar issue with record fields where import order can
> change the order of imported FieldDecls resulting in broken ASTRecordLayout.
> The patch for this issue is on review: https://reviews.llvm.org/D44100. It
> just reorders the fields after structure import is finished. CSA guys also
> reported the same problem with FriendDecls in the same review.The order of
> methods was not a problem for us but your report adds a new item to support.
> It looks like _all_ decls inside RecordDecl have to be reordered. I'll try to
> resurrect the patch this weekend (it is a bit outdated due to my workload,
> sorry) and add you as a reviewer so you can check if it solves the problem or
> not.
>
> 09.08.2018 20:46, Lang Hames via lldb-dev пишет:
>
> Hi clang-dev, lldb-dev,
>
> It looks like my clang commit r305850, which modified ASTImporter to import
> method override tables from an external context, introduced a new bug which
> manifests as incorrect vtable layouts for LLDB expression code.
>
> The bug itself is fairly straightforward. In r305850 I introduced the
> following method, which is called from ASTNodeImporter::VisitFunctionDecl:
>
> void ASTNodeImporter::ImportOverrides(CXXMethodDecl *ToMethod,
> CXXMethodDecl *FromMethod) {
> for (auto *FromOverriddenMethod : FromMethod->overridden_methods())
> ToMethod->addOverriddenMethod(
> cast(Importer.Import(const_cast(
> FromOverriddenMethod;
> }
>
> This will produce the correct override table, but can also cause methods in
> the Base class to be visited in the wrong order. Consider:
>
> class Base {
> public:
> virtual void bar() {}
> virtual void foo() {}
> };
>
> class Derived : public Base {
> public:
> void foo() override {}
> };
>
> If Derived is imported into a new context before Base, then the importer will
> visit Derived::foo, and (via ImportOverrides) immediately import Base::foo,
> but this happens before Base::bar is imported. As a consequence, the decl
> order on the imported Base class wil
Re: [lldb-dev] stable layout bug for imported record decls.
I have forgot to include the matcher I used for the test:
```
AST_MATCHER_P(CXXRecordDecl, hasMethodOrder, std::vector, Order) {
size_t Index = 0;
for (CXXMethodDecl *Method : Node.methods()) {
if (!Method->isImplicit()) {
if (Method->getName() != Order[Index])
return false;
++Index;
}
}
return Index == Order.size();
}
```
Gabor
On Fri, Aug 10, 2018 at 6:42 PM Gábor Márton wrote:
>
> Hi Lang, Alexey,
>
> I dug deeper into this and it seems like the issue happens only when a
> **minimal** import is used. LLDB uses the minimal import. CrossTU
> static analyzer uses the normal mode.
> In normal import mode, in `ImportDeclContext` we do import all the
> methods in the correct order. However, in minimal mode we early return
> before importing the methods.
> So by merging D44100 this particular problem will not be solved.
> Still, that patch sounds very reasonable and I support that we should
> reorder all imported Decls, not just fields.
>
> One trivial solution would be to change `ImporDeclContext` to behave
> the same way in both modes. This is somewhat similar to the "brute
> force" method you mentioned.
> I am not an LLDB expert, so I am not sure if this is acceptable, and
> really don't know how many LLDB tests would it break, but this seems
> the simplest solution (and preferred) to me.
>
> The other solution if you'd like to keep the minimal behavior is the
> index based solution (as you mentioned).
> You should compare the order of all the imported methods (and fields)
> to the order in the original class in the "From" context. And I
> believe you have to do that at the end of VisitFunctionDecl. It would
> not work if you did that check when the type become complete, since in
> minimal mode we never set the class to be incomplete.
>
> I have created a unit test case, which fails in minimal mode and
> succeeds in normal mode. You can change the mode in
> `ASTImporterTestBase::TU::lazyInitImporter`.
> If you provide a patch then could you please also add this test (or
> similar) for both normal and minimal mode?
> ```
> TEST_P(ASTImporterTestBase, OrderOfVirtualMethods) {
> auto Code =
> R"(
> class Base {
> public:
> virtual void bar() {}
> virtual void foo() {}
> };
>
> class Derived : public Base {
> public:
> void foo() override {}
> };
> )";
> Decl *FromTU = getTuDecl(Code, Lang_CXX11, "input0.cc");
> auto *DerivedFoo = FirstDeclMatcher().match(
> FromTU, functionDecl(hasName("foo"),
>hasParent(cxxRecordDecl(hasName("Derived");
> auto *BaseBar = FirstDeclMatcher().match(
> FromTU, functionDecl(hasName("bar"),
>hasParent(cxxRecordDecl(hasName("Base");
>
> Import(DerivedFoo, Lang_CXX);
> // Importing Base::bar explicitly is needed only in minimal mode,
> // in normal mode we already imported all methods of Base.
> Import(BaseBar, Lang_CXX);
>
> Decl *ToTU = ToAST->getASTContext().getTranslationUnitDecl();
> auto *ImportedBase = FirstDeclMatcher().match(
> ToTU, cxxRecordDecl(hasName("Base")));
> MatchVerifier Verifier;
> EXPECT_TRUE(Verifier.match(ImportedBase,
> cxxRecordDecl(hasMethodOrder({"bar", "foo"};
> }
> ```
>
> Thanks,
> Gabor
> On Fri, Aug 10, 2018 at 3:47 PM Alexey Sidorin wrote:
> >
> > (+ Gabor and Gabor)
> >
> > Hi Lang,
> >
> > We faced a very similar issue with record fields where import order can
> > change the order of imported FieldDecls resulting in broken
> > ASTRecordLayout. The patch for this issue is on review:
> > https://reviews.llvm.org/D44100. It just reorders the fields after
> > structure import is finished. CSA guys also reported the same problem with
> > FriendDecls in the same review.The order of methods was not a problem for
> > us but your report adds a new item to support. It looks like _all_ decls
> > inside RecordDecl have to be reordered. I'll try to resurrect the patch
> > this weekend (it is a bit outdated due to my workload, sorry) and add you
> > as a reviewer so you can check if it solves the problem or not.
> >
> > 09.08.2018 20:46, Lang Hames via lldb-dev пишет:
> >
> > Hi clang-dev, lldb-dev,
> >
> > It looks like my clang commit r305850, which modified ASTImporter to import
> > method override tables from an external context, introduced a new bug which
> > manifests as incorrect vtable layouts for LLDB expression code.
> >
> > The bug itself is fairly straightforward. In r305850 I introduced the
> > following method, which is called from ASTNodeImporter::VisitFunctionDecl:
> >
> > void ASTNodeImporter::ImportOverrides(CXXMethodDecl *ToMethod,
> > CXXMethodDecl *FromMethod) {
> > for (auto *FromOverriddenMethod : FromMethod->overridden_methods())
> > ToMethod->addOverriddenMethod(
> > cast(Importer.Import(const_cast(
> >
Re: [lldb-dev] [cfe-dev] stable layout bug for imported record decls.
> It might be interesting to consider how this is addressed by the > ExternalASTSource interface. There, we have separate "import the lexical > contents of this AST node (including populating the lexical declarations list > with all members, in the right order)", "import the lookup results for this > name in this context (and cache them but don't add them to the list of > lexical members)" and "import this specific declaration member (but don't add > it to the list of lexical members)" queries. One could imagine doing > something similar for the AST importer: when you're performing a minimal > import but you want to import a method declaration, don't insert it into the > list of lexical members of the enclosing record. Instead, defer doing that > until a complete type is required (at that point, you'd need to import all > the relevant members anyway, including the base classes and virtual functions > in the right order). Hi Richard, maybe my understanding is not correct, but I believe the situation is somewhat twisted and reversed. Seems like LLDB already exercises the ExternalASTSource interface. The purpose of using it is exactly to load the external lexical contents of an AST node (as you suggested). However, the load is implemented by the means of the ASTImporter (in minimal mode). Consider the attached function call trace: When the user instructs LLDB to evaluate an expression then LLDB exercises the parser `clang::ParseAST`, which in turn does a lookup `clang::DeclContext::lookup`. During the lookup, `lldb_private::ClangASTSource::FindExternalVisibleDeclsByName` is called (this function overloads `ExternalASTSource::FindExternalVisibleDeclsByName`). And finally, through `lldb_private::ClangASTImporter::CopyType` we end up in `clang::ASTImporter::Import`. Gabor ``` #4 0x741a72ed in clang::ASTNodeImporter::ImportDeclParts (this=this@entry=0x7fff8be0, D=D@entry=0x632038, DC=@0x7fff8ac8: 0x0, LexicalDC=@0x7fff8ad0: 0x0, Name=..., ToD=@0x7fff8ad8: 0x0, Loc=...) at /home/gerazo/repos/codechecker_dev_env/CTU/llvm/tools/clang/lib/AST/ASTImporter.cpp:1062 #5 0x741a8d55 in clang::ASTNodeImporter::VisitRecordDecl (this=0x7fff8be0, D=0x632038) at /home/gerazo/repos/codechecker_dev_env/CTU/llvm/tools/clang/lib/AST/ASTImporter.cpp:2126 #6 0x7419377b in clang::ASTImporter::Import (this=0x70cc90, FromD=0x632038) at /home/gerazo/repos/codechecker_dev_env/CTU/llvm/tools/clang/lib/AST/ASTImporter.cpp:7054 #7 0x741939f7 in clang::ASTNodeImporter::VisitRecordType (this=0x7fff8c40, T=) at /home/gerazo/repos/codechecker_dev_env/CTU/llvm/tools/clang/lib/AST/ASTImporter.cpp:851 #8 0x74196f65 in clang::TypeVisitor::Visit (this=this@entry=0x7fff8c40, T=) at /home/gerazo/repos/codechecker_dev_env/CTU/llvm/tools/clang/include/clang/AST/TypeNodes.def:92 #9 0x74197187 in clang::ASTImporter::Import (this=0x70cc90, FromT=FromT@entry=...) at /home/gerazo/repos/codechecker_dev_env/CTU/llvm/tools/clang/lib/AST/ASTImporter.cpp:6983 #10 0x77351c92 in lldb_private::ClangASTImporter::CopyType (this=, dst_ast=, src_ast=, type=type@entry=...) at /home/gerazo/repos/codechecker_dev_env/CTU/llvm/tools/lldb/source/Symbol/ClangASTImporter.cpp:61 #11 0x774bdb95 in lldb_private::ClangASTSource::GuardedCopyType (this=this@entry=0x611e20, src_type=...) at /home/gerazo/repos/codechecker_dev_env/CTU/llvm/tools/lldb/source/Plugins/ExpressionParser/Clang/ClangASTSource.cpp:2040 #12 0x774e9ffe in lldb_private::ClangExpressionDeclMap::GetVariableValue (this=this@entry=0x611e20, var=std::shared_ptr (count 6, weak 1) 0x7fffe80020f0, var_location=..., user_type=user_type@entry=0x7fff8fc0, parser_type=parser_type@entry=0x7fff8fe0) at /home/gerazo/repos/codechecker_dev_env/CTU/llvm/tools/lldb/source/Plugins/ExpressionParser/Clang/ClangExpressionDeclMap.cpp:1627 #13 0x774ea3a7 in lldb_private::ClangExpressionDeclMap::AddOneVariable (this=this@entry=0x611e20, context=..., var=std::shared_ptr (count 6, weak 1) 0x7fffe80020f0, valobj=..., current_id=current_id@entry=2) at /home/gerazo/repos/codechecker_dev_env/CTU/llvm/tools/lldb/source/Plugins/ExpressionParser/Clang/ClangExpressionDeclMap.cpp:1679 #14 0x774ec50b in lldb_private::ClangExpressionDeclMap::FindExternalVisibleDecls (this=this@entry=0x611e20, context=..., module_sp= std::shared_ptr (empty) 0x0, namespace_decl=..., current_id=current_id@entry=2) at /home/gerazo/repos/codechecker_dev_env/CTU/llvm/tools/lldb/source/Plugins/ExpressionParser/Clang/ClangExpressionDeclMap.cpp:1240 #15 0x774ef4ce in lldb_private::ClangExpressionDeclMap::FindExternalVisibleDecls (this=0x611e20, context=...) at /home/gerazo/repos/codechecker_dev_env/CTU/llvm/tools/lldb/source/Plugins/ExpressionParser/Clang/ClangExpressionDeclMap.cpp:843 #16 0x774c71cc in lldb_private::ClangASTSource::FindExternalV
Re: [lldb-dev] [cfe-dev] stable layout bug for imported record decls.
Lang, I've done some more research on this and here is what I've found so far. LLDB uses the ExternalASTSource interface and it provides its own implementation for it (lldb_private::ClangASTSource). ClangASTSource is being used in at least 3 modes (all of these uses the ASTImporter): (1) to do a minimal import for a Type/Decl, this is used for lookup (2) to do a minimal import for a Type/Decl, this is used for computing the record layout (3) to complete the Type which has been imported in a minimal mode previously. The completion is done via ASTImporter::ImportDefinition, which does a full import of a TagDecl even in minimal mode. Besides, there is a fourth method which exercises the ASTImporter, but it does not use the ExternalASTSource interface (4). [[ Actually, (4) happens before than (3) in program runtime. ]] So, it is kind of weird to me, that we use a simple minimal import in (2) for computing the record layout, perhaps we could make a change in LLDB to call `lldb_private::ClangASTSource::CompleteType` to achieve a full import in this case? Instead of changing the logic in clang::ASTImporter? By doing so, I think the logic would be aligned with Richard's idea. Gabor (1) === "To" ASTContext A, "From" ASTContext X, MINIMAL import through ExternalASTSource inteface clang::ASTImporter::Import (this=0x13b3c50, FromT=...) at ../../git/llvm/tools/clang/lib/AST/ASTImporter.cpp:6983 lldb_private::ClangASTImporter::CopyType (this=0x12a3480, dst_ast=0x1363420, src_ast=0x11f7980, type=...) at ../../git/llvm/tools/lldb/source/Symbol/ClangASTImporter.cpp:61 lldb_private::ClangASTSource::GuardedCopyType (this=0x12394b0, src_type=...) at ../../git/llvm/tools/lldb/source/Plugins/ExpressionParser/Clang/ClangASTSource.cpp:2038 lldb_private::ClangExpressionDeclMap::GetVariableValue (this=0x12394b0, var=warning: RTTI symbol not found for class 'std::_Sp_counted_ptr' lldb_private::ClangExpressionDeclMap::AddOneVariable (this=0x12394b0, context=..., var=warning: RTTI symbol not found for class 'std::_Sp_counted_ptr' lldb_private::ClangExpressionDeclMap::FindExternalVisibleDecls (this=0x12394b0, context=..., module_sp=std::shared_ptr (empty) 0x0, namespace_decl=..., current_id=3) at ../../git/llvm/tools/lldb/source/Plugins/ExpressionParser/Clang/ClangExpressionDeclMap.cpp:1235 lldb_private::ClangExpressionDeclMap::FindExternalVisibleDecls (this=0x12394b0, context=...) at ../../git/llvm/tools/lldb/source/Plugins/ExpressionParser/Clang/ClangExpressionDeclMap.cpp:842 lldb_private::ClangASTSource::FindExternalVisibleDeclsByName (this=0x12394b0, decl_ctx=0x1371298, clang_decl_name=...) at ../../git/llvm/tools/lldb/source/Plugins/ExpressionParser/Clang/ClangASTSource.cpp:261 lldb_private::ClangASTSource::ClangASTSourceProxy::FindExternalVisibleDeclsByName (this=0x105e230, DC=0x1371298, Name=...) at ../../git/llvm/tools/lldb/source/Plugins/ExpressionParser/Clang/ClangASTSource.h:244 clang::DeclContext::lookup (this=0x1371298, Name=...) at ../../git/llvm/tools/clang/lib/AST/DeclBase.cpp:1599 LookupDirect (S=..., R=..., DC=0x1371298) at ../../git/llvm/tools/clang/lib/Sema/SemaLookup.cpp:84 CppNamespaceLookup (S=..., R=..., Context=..., NS=0x1d69dd8, UDirs=...) at ../../git/llvm/tools/clang/lib/Sema/SemaLookup.cpp:942 clang::Sema::CppLookupName (this=0x1d74cd0, R=..., S=0x1d80510) at ../../git/llvm/tools/clang/lib/Sema/SemaLookup.cpp:1322 ... clang::Sema::ActOnUsingDeclaration (this=0x1d74cd0, S=0x1d93820, AS=clang::AS_none, UsingLoc=..., TypenameLoc=..., SS=..., Name=..., EllipsisLoc=..., AttrList=...) at ../../git/llvm/tools/clang/lib/Sema/SemaDeclCXX.cpp:9418 clang::Parser::ParseUsingDeclaration (this=0x1d7bd90, Context=clang::DeclaratorContext::BlockContext, TemplateInfo=..., UsingLoc=..., DeclEnd=..., AS=clang::AS_none) at ../../git/llvm/tools/clang/lib/Parse/ParseDeclCXX.cpp:710 ... clang::Parser::ParseTopLevelDecl (this=0x1d7bd90, Result=...) at ../../git/llvm/tools/clang/lib/Parse/Parser.cpp:610 clang::ParseAST (S=..., PrintStats=false, SkipFunctionBodies=false) at ../../git/llvm/tools/clang/lib/Parse/ParseAST.cpp:158 clang::ParseAST (PP=..., Consumer=0x1a2dfc0, Ctx=..., PrintStats=false, TUKind=clang::TU_Complete, CompletionConsumer=0x0, SkipFunctionBodies=false) at ../../git/llvm/tools/clang/lib/Parse/ParseAST.cpp:111 lldb_private::ClangExpressionParser::Parse (this=0x7fff14cdfe90, diagnostic_manager=...) at ../../git/llvm/tools/lldb/source/Plugins/ExpressionParser/Clang/ClangExpressionParser.cpp:608 lldb_private::ClangUserExpression::Parse (this=0x1d1d530, diagnostic_manager=..., exe_ctx=..., execution_policy=lldb_private::eExecutionPolicyOnlyWhenNeeded, keep_result_in_memory=true, generate_debug_info=false) at ../../git/llvm/tools/lldb/source/Plugins/ExpressionParser/Clang/ClangUserExpression.cpp:495 lldb_private::UserExpression::Evaluate (exe_ctx=..., options=..., expr="tpi", prefix="", result_valobj_sp=..., error=..., line_offset=0, fixed_expression=0x1a95738, jit_module_sp_ptr=0x0) at ../../git/llvm/tools/lld
Re: [lldb-dev] [cfe-dev] stable layout bug for imported record decls.
Just sending again the trace, because gmail broke the lines pretty badly. (1) === "To" ASTContext A, "From" ASTContext X, MINIMAL import through ExternalASTSource inteface clang::ASTImporter::Import (this=0x13b3c50, FromT=...) at ../../git/llvm/tools/clang/lib/AST/ASTImporter.cpp:6983 lldb_private::ClangASTImporter::CopyType (this=0x12a3480, dst_ast=0x1363420, src_ast=0x11f7980, type=...) at ../../git/llvm/tools/lldb/source/Symbol/ClangASTImporter.cpp:61 lldb_private::ClangASTSource::GuardedCopyType (this=0x12394b0, src_type=...) at ../../git/llvm/tools/lldb/source/Plugins/ExpressionParser/Clang/ClangASTSource.cpp:2038 lldb_private::ClangExpressionDeclMap::GetVariableValue (this=0x12394b0, var=warning: RTTI symbol not found for class 'std::_Sp_counted_ptr' lldb_private::ClangExpressionDeclMap::AddOneVariable (this=0x12394b0, context=..., var=warning: RTTI symbol not found for class 'std::_Sp_counted_ptr' lldb_private::ClangExpressionDeclMap::FindExternalVisibleDecls (this=0x12394b0, context=..., module_sp=std::shared_ptr (empty) 0x0, namespace_decl=..., current_id=3) at ../../git/llvm/tools/lldb/source/Plugins/ExpressionParser/Clang/ClangExpressionDeclMap.cpp:1235 lldb_private::ClangExpressionDeclMap::FindExternalVisibleDecls (this=0x12394b0, context=...) at ../../git/llvm/tools/lldb/source/Plugins/ExpressionParser/Clang/ClangExpressionDeclMap.cpp:842 lldb_private::ClangASTSource::FindExternalVisibleDeclsByName (this=0x12394b0, decl_ctx=0x1371298, clang_decl_name=...) at ../../git/llvm/tools/lldb/source/Plugins/ExpressionParser/Clang/ClangASTSource.cpp:261 lldb_private::ClangASTSource::ClangASTSourceProxy::FindExternalVisibleDeclsByName (this=0x105e230, DC=0x1371298, Name=...) at ../../git/llvm/tools/lldb/source/Plugins/ExpressionParser/Clang/ClangASTSource.h:244 clang::DeclContext::lookup (this=0x1371298, Name=...) at ../../git/llvm/tools/clang/lib/AST/DeclBase.cpp:1599 LookupDirect (S=..., R=..., DC=0x1371298) at ../../git/llvm/tools/clang/lib/Sema/SemaLookup.cpp:84 CppNamespaceLookup (S=..., R=..., Context=..., NS=0x1d69dd8, UDirs=...) at ../../git/llvm/tools/clang/lib/Sema/SemaLookup.cpp:942 clang::Sema::CppLookupName (this=0x1d74cd0, R=..., S=0x1d80510) at ../../git/llvm/tools/clang/lib/Sema/SemaLookup.cpp:1322 ... clang::Sema::ActOnUsingDeclaration (this=0x1d74cd0, S=0x1d93820, AS=clang::AS_none, UsingLoc=..., TypenameLoc=..., SS=..., Name=..., EllipsisLoc=..., AttrList=...) at ../../git/llvm/tools/clang/lib/Sema/SemaDeclCXX.cpp:9418 clang::Parser::ParseUsingDeclaration (this=0x1d7bd90, Context=clang::DeclaratorContext::BlockContext, TemplateInfo=..., UsingLoc=..., DeclEnd=..., AS=clang::AS_none) at ../../git/llvm/tools/clang/lib/Parse/ParseDeclCXX.cpp:710 ... clang::Parser::ParseTopLevelDecl (this=0x1d7bd90, Result=...) at ../../git/llvm/tools/clang/lib/Parse/Parser.cpp:610 clang::ParseAST (S=..., PrintStats=false, SkipFunctionBodies=false) at ../../git/llvm/tools/clang/lib/Parse/ParseAST.cpp:158 clang::ParseAST (PP=..., Consumer=0x1a2dfc0, Ctx=..., PrintStats=false, TUKind=clang::TU_Complete, CompletionConsumer=0x0, SkipFunctionBodies=false) at ../../git/llvm/tools/clang/lib/Parse/ParseAST.cpp:111 lldb_private::ClangExpressionParser::Parse (this=0x7fff14cdfe90, diagnostic_manager=...) at ../../git/llvm/tools/lldb/source/Plugins/ExpressionParser/Clang/ClangExpressionParser.cpp:608 lldb_private::ClangUserExpression::Parse (this=0x1d1d530, diagnostic_manager=..., exe_ctx=..., execution_policy=lldb_private::eExecutionPolicyOnlyWhenNeeded, keep_result_in_memory=true, generate_debug_info=false) at ../../git/llvm/tools/lldb/source/Plugins/ExpressionParser/Clang/ClangUserExpression.cpp:495 lldb_private::UserExpression::Evaluate (exe_ctx=..., options=..., expr="tpi", prefix="", result_valobj_sp=..., error=..., line_offset=0, fixed_expression=0x1a95738, jit_module_sp_ptr=0x0) at ../../git/llvm/tools/lldb/source/Expression/UserExpression.cpp:237 lldb_private::Target::EvaluateExpression (this=0x1b32570, expr="tpi", exe_scope=0x7fe73ca42d90, result_valobj_sp=..., options=..., fixed_expression=0x1a95738) at ../../git/llvm/tools/lldb/source/Target/Target.cpp:2308 lldb_private::CommandObjectExpression::EvaluateExpression (this=0x1a953b0, expr=0x7fff14ce0d58 "tpi", output_stream=0x7fff14ce16e8, error_stream=0x7fff14ce1740, result=0x7fff14ce16e8) at ../../git/llvm/tools/lldb/source/Commands/CommandObjectExpression.cpp:377 (2) === ASTContext A, "From" ASTContext X, MINIMAL import through ExternalASTSource inteface clang::ASTImporter::Import (this=0x13b3c50, FromD=0x13bba18) at ../../git/llvm/tools/clang/lib/AST/ASTImporter.cpp:7040 lldb_private::ClangASTImporter::CopyDecl (this=0x12a3480, dst_ast=0x1363420, src_ast=0x11f7980, decl=0x13bba18) at ../../git/llvm/tools/lldb/source/Symbol/ClangASTImporter.cpp:102 lldb_private::ClangASTSource::CopyDecl (this=0x12394b0, src_decl=0x13bba18) at ../../git/llvm/tools/lldb/source/Plugins/ExpressionParser/Clang/ClangASTSource.cpp:1991 lldb_private::ClangASTSource::FindExterna
Re: [lldb-dev] [cfe-dev] [7.0.0 Release] The final tag is in
Hi Hans, > The final version of 7.0.0 has been tagged from the branch at r342370. I'd like to fork from 7.0.0 final but I got confused: The tip of release_70 branch is still r341805, which is identical to rc3. This should be r342370 instead, shouldn't it? Or the final (r342370) does not go into the release branch? Or it just takes some more time? Thanks, Gabor On Mon, Sep 17, 2018 at 1:42 PM Hans Wennborg via cfe-dev wrote: > > Dear testers, > > The final version of 7.0.0 has been tagged from the branch at r342370. > It is identical to rc3 modulo release notes and docs changes. > > Please build the final binaries and upload to the sftp. > > For those following along: this means 7.0.0 is done, but it will take > a few days to get all the tarballs ready and published on the web > page. I will send the announcement once everything is ready. > > Thanks again everyone for your work! > > Hans > ___ > cfe-dev mailing list > cfe-...@lists.llvm.org > http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev ___ lldb-dev mailing list lldb-dev@lists.llvm.org http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev
[lldb-dev] skip some tests with "check-lldb"
Hi, I'd like to skip some tests when I run "ninja check-lldb", because they fail. I am on release_70 branch. I know I could use dotest.py directly, but that would exercise only one thread. Is there a way to execute the tests parallel on all cores and in the same time skip some of the tests? Thanks, Gabor ___ lldb-dev mailing list lldb-dev@lists.llvm.org http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev
Re: [lldb-dev] skip some tests with "check-lldb"
I just realized that `dotest.py` has a --thread option. Is that the one which is used during the lit test (`ninja check-lldb`) ? On Wed, Sep 19, 2018 at 6:00 PM Gábor Márton wrote: > > Hi, > > I'd like to skip some tests when I run "ninja check-lldb", because they fail. > I am on release_70 branch. > I know I could use dotest.py directly, but that would exercise only one > thread. > Is there a way to execute the tests parallel on all cores and in the > same time skip some of the tests? > > Thanks, > Gabor ___ lldb-dev mailing list lldb-dev@lists.llvm.org http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev
Re: [lldb-dev] skip some tests with "check-lldb"
That's okay, but is it possible to skip a few tests, when using lit? I was thinking about moving the test files I want to skip, but that has obvious drawbacks. Also --filter does not seem so useful in this case. On Wed, 19 Sep 2018, 18:46 , wrote: > Unless you pass --no-multiprocess to dotest, it should detect how many > cores your system has and use them. > > -- > Ted Woodward > Qualcomm Innovation Center, Inc. > Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum, a Linux > Foundation Collaborative Project > > -Original Message----- > From: lldb-dev On Behalf Of Gábor > Márton via lldb-dev > Sent: Wednesday, September 19, 2018 11:04 AM > To: lldb-dev@lists.llvm.org > Subject: Re: [lldb-dev] skip some tests with "check-lldb" > > I just realized that `dotest.py` has a --thread option. Is that the one > which is used during the lit test (`ninja check-lldb`) ? > > On Wed, Sep 19, 2018 at 6:00 PM Gábor Márton > wrote: > > > > Hi, > > > > I'd like to skip some tests when I run "ninja check-lldb", because they > fail. > > I am on release_70 branch. > > I know I could use dotest.py directly, but that would exercise only one > thread. > > Is there a way to execute the tests parallel on all cores and in the > > same time skip some of the tests? > > > > Thanks, > > Gabor > ___ > lldb-dev mailing list > lldb-dev@lists.llvm.org > http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev > > ___ lldb-dev mailing list lldb-dev@lists.llvm.org http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev
Re: [lldb-dev] skip some tests with "check-lldb"
Hi Jonas, Thanks for the clarification. Finally I could skip some of the tests by using a negative lookahead regex passed to --filter. I wouldn't say that this is so convenient and easy to overview, but works. For example, to skip the TestCalculatorMode.py and TestPrintArray.py tests I used the following argument (appended to the normal lit command line reported by "ninja check-lldb -v" ) : --filter '^((?!(TestCalculatorMode\.py|TestPrintArray\.py)).)*$' Would be beneficial though to have a --exclude besides --filter to avoid using a complex negative lookahead regex. Cheers, Gabor On Thu, Sep 20, 2018 at 12:53 PM Jonas Devlieghere wrote: > > When using lit as the driver (which is the case for check-lldb), the test > granularity is at file-level: lit invokes dotest.py for every test file. You > should be able to specify files to skip with lit's --filter. > > With regards to threading, lit will schedule one instance of dotest.py on > every > thread, which processes one test file. Unless you passed specific options for > dotest.py, the latter will run the different tests within that one file on > different threads as well. (IIRC) > > On Sep 19, 2018, at 7:20 PM, Gábor Márton via lldb-dev > wrote: > > That's okay, but is it possible to skip a few tests, when using lit? I was > thinking about moving the test files I want to skip, but that has obvious > drawbacks. Also --filter does not seem so useful in this case. > > On Wed, 19 Sep 2018, 18:46 , wrote: >> >> Unless you pass --no-multiprocess to dotest, it should detect how many >> cores your system has and use them. >> >> -- >> Ted Woodward >> Qualcomm Innovation Center, Inc. >> Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum, a Linux >> Foundation Collaborative Project >> >> -Original Message- >> From: lldb-dev On Behalf Of Gábor Márton >> via lldb-dev >> Sent: Wednesday, September 19, 2018 11:04 AM >> To: lldb-dev@lists.llvm.org >> Subject: Re: [lldb-dev] skip some tests with "check-lldb" >> >> I just realized that `dotest.py` has a --thread option. Is that the one >> which is used during the lit test (`ninja check-lldb`) ? >> >> On Wed, Sep 19, 2018 at 6:00 PM Gábor Márton wrote: >> > >> > Hi, >> > >> > I'd like to skip some tests when I run "ninja check-lldb", because they >> > fail. >> > I am on release_70 branch. >> > I know I could use dotest.py directly, but that would exercise only one >> > thread. >> > Is there a way to execute the tests parallel on all cores and in the >> > same time skip some of the tests? >> > >> > Thanks, >> > Gabor >> ___ >> lldb-dev mailing list >> lldb-dev@lists.llvm.org >> http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev >> > ___ > lldb-dev mailing list > lldb-dev@lists.llvm.org > http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev > > ___ lldb-dev mailing list lldb-dev@lists.llvm.org http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev
[lldb-dev] ASTImporter patches and improvements, please help
Dear LLDB Developers, There is an ongoing activity to improve ASTImporter in Clang to make it support C++ (and C) completely and correctly. Our team works on cross translation unit (CTU) static analysis where we use the in-tree Clang static analyzer in combination with the ASTImporter (via the scan-build.py script or CodeChecker). In the past 18 months we have committed more than 70 patches to make the ASTImporter better. Our primary target is Linux and we do run the LLDB tests on Linux before we commit. Unfortunately, quite recently 3 of these patches broke the LLDB macOS buildbots (green.lab.llvm.org/green/view/LLDB) and this caused some turmoil. We are very sorry about this and we are making actions to avoid such inconveniences in the future: We ordered a Mac Mini, until it arrives we are using a borrowed Mac Book. We are going to create a CI job which will execute the macOS LLDB test suite for a specific patch. Besides this, for every patch we are going to monitor the macOS buildbots once they are committed. However, we are experiencing problems both with the buildbots and with the LLDB tests, therefore we are asking help from the LLDB community in the following: (1) Apparently, the green lab macOS buildbots are not displayed in the build bot console (http://lab.llvm.org:8011/console). More importantly they are not capable of sending emails to the authors of the commit in case of failure. Would it be possible to setup the these buildbots to send out the emails for the authors? (2) We are facing difficulties with the LLDB lit tests both on macOS and on Linux. E.g. on a fresh macOS mojave install, checking out master LLVM, Clang, LLDB and then executing ninja check-lldb fails immediately. On Linux we experienced that some tests fail because they try to read a non-existent register. Some tests fail non-deterministically because they can't kill a process or a timeout is not long enough. Some tests fail because of a linker error of libcxx. We would like to address all these issues. Would you like to discuss these issues on lldb-dev or should we create a bugzilla ticket for each? (3) ASTImporter unit tests and lit tests in Clang for the LLDB specific usage are very-very limited. LLDB uses the so called "minimal" import, but none of the unit tests exercises that (CTU uses the full import). We should have a unit test suite where the LLDB use cases are covered, e.g. after a minimal import an `importDefinition` is called. Also, a test double which implements the ExternalASTSource could be used. I believe it would be possible to cover with the unit tests all LLDB/macOS related scenarios and these tests could run on Linux too. We do something similar in case of windows: we execute all unit tests with "-fdelayed-template-parsing" and with "-fms-compatibility" too. I think, the more unit tests we have in Clang the sooner we catch the LLDB specific importer errors. I am asking the LLDB community's help here, because we lack the domain knowledge in LLDB so we don't know what expectations should we write in each unit test cases. (About lit tests, there is clang-import-test but its coverage seems pretty low and I don't know if that really exercises all LLDB use cases.) I think that from a better ASTImporter the LLDB debugger can benefit too: the `expr` command might give better experience by supporting more C++ constructs and by giving better diagnostics. Thank you, Gabor ___ lldb-dev mailing list lldb-dev@lists.llvm.org http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev
Re: [lldb-dev] [cfe-dev] ASTImporter patches and improvements, please help
Hi Davide,
Thank you for your email.
> In particular, what's the error you get when lldb fails immediately
running the tests?
> Also, have you checked libcxx and libcxx-abi in your build? We might
> consider making that a mandatory dependency for the Cmake build.
Finally I could run the test suite ("ninja check-lldb) on our macOS without
errors. Here are my findings:
1) "sudo /usr/sbin/DevToolsSecurity --enable" is a must.
2) I indeed did not checked out libcxx. (libcxx-abi was not needed to have
a successful run). Would be great to make that a mandatory dependency.
3) Besides that I recognized that "ninja check-lldb" target does not build
all it's dependencies. E.g. lldb-server tests failed until I executed
"ninja debugserver" before "ninja check-lldb". I am not sure, but I think
"ninja obj2yaml" and "ninja llvm-pdbutil" are also prerequisites (and these
targets seem not to be built by a simple "ninja" command).
Thanks,
Gabor
On Sat, Dec 1, 2018 at 9:40 PM Davide Italiano
wrote:
> On Sat, Dec 1, 2018 at 8:00 AM Gábor Márton via cfe-dev
> wrote:
> >
> > Dear LLDB Developers,
> >
> > There is an ongoing activity to improve ASTImporter in Clang to make it
> support
> > C++ (and C) completely and correctly. Our team works on cross
> translation unit
> > (CTU) static analysis where we use the in-tree Clang static analyzer in
> > combination with the ASTImporter (via the scan-build.py script or
> CodeChecker).
> > In the past 18 months we have committed more than 70 patches to make the
> > ASTImporter better. Our primary target is Linux and we do run the LLDB
> tests
> > on Linux before we commit. Unfortunately, quite recently 3 of these
> patches
> > broke the LLDB macOS buildbots (green.lab.llvm.org/green/view/LLDB) and
> this
> > caused some turmoil. We are very sorry about this and we are making
> actions to
> > avoid such inconveniences in the future: We ordered a Mac Mini, until it
> > arrives we are using a borrowed Mac Book. We are going to create a CI
> job which
> > will execute the macOS LLDB test suite for a specific patch. Besides
> this, for
> > every patch we are going to monitor the macOS buildbots once they are
> > committed.
> >
> > However, we are experiencing problems both with the buildbots and with
> the LLDB
> > tests, therefore we are asking help from the LLDB community in the
> following:
> >
> > (1) Apparently, the green lab macOS buildbots are not displayed in the
> build
> > bot console (http://lab.llvm.org:8011/console). More importantly they
> are not
> > capable of sending emails to the authors of the commit in case of
> failure.
> > Would it be possible to setup the these buildbots to send out the emails
> for
> > the authors?
> >
>
> Thanks for reporting this problem. I cc:ed Mike, as he owns the bots,
> and he should be able to help us with this.
> (I do have access to the Jenkins configuration but I don't feel
> confident enough to make the change myself).
>
> > (2) We are facing difficulties with the LLDB lit tests both on macOS and
> on
> > Linux. E.g. on a fresh macOS mojave install, checking out master LLVM,
> Clang,
> > LLDB and then executing ninja check-lldb fails immediately. On Linux we
> > experienced that some tests fail because they try to read a non-existent
> > register. Some tests fail non-deterministically because they can't kill a
> > process or a timeout is not long enough. Some tests fail because of a
> linker
> > error of libcxx. We would like to address all these issues. Would you
> like to
> > discuss these issues on lldb-dev or should we create a bugzilla ticket
> for
> > each?
> >
>
> I think either of them is fine. If you're willing to spend some time
> on this, I'm sure people on the LLDB side will consider helping you.
> In particular, what's the error you get when lldb fails immediately
> running the tests?
> Also, have you checked libcxx and libcxx-abi in your build? We might
> consider making that a mandatory dependency for the Cmake build.
>
> > (3) ASTImporter unit tests and lit tests in Clang for the LLDB specific
> usage
> > are very-very limited. LLDB uses the so called "minimal" import, but
> none of
> > the unit tests exercises that (CTU uses the full import). We should
> have a
> > unit test suite where the LLDB use cases are covered, e.g. after a
> minimal
> > import an `importDefinition` is called. Also, a test double which
> implements
> > the ExternalASTSource could be used. I believe it would be possible to
> cover
> > with the unit tests all LLDB/macOS related scenarios and these tests
> could run
> > on Linux too. We do something similar in case of windows: we execute
> all unit
> > tests with "-fdelayed-template-parsing" and with "-fms-compatibility"
> too. I
> > think, the more unit tests we have in Clang the sooner we catch the LLDB
> > specific importer errors. I am asking the LLDB community's help here,
> because
> > we lack the domain knowledge in LLDB so we don't know what expectations
> should
> > we write
Re: [lldb-dev] [cfe-dev] ASTImporter patches and improvements, please help
Sure, I have created the reports:
https://bugs.llvm.org/show_bug.cgi?id=39909
https://bugs.llvm.org/show_bug.cgi?id=39910
And there was one already which I extended:
https://bugs.llvm.org/show_bug.cgi?id=22988
Gabor
On Thu, Dec 6, 2018 at 6:39 PM Davide Italiano
wrote:
> On Thu, Dec 6, 2018 at 6:39 AM Gábor Márton
> wrote:
> >
> > Hi Davide,
> >
> > Thank you for your email.
> >
> > > In particular, what's the error you get when lldb fails immediately
> running the tests?
> > > Also, have you checked libcxx and libcxx-abi in your build? We might
> > > consider making that a mandatory dependency for the Cmake build.
> >
> > Finally I could run the test suite ("ninja check-lldb) on our macOS
> without errors. Here are my findings:
> > 1) "sudo /usr/sbin/DevToolsSecurity --enable" is a must.
> > 2) I indeed did not checked out libcxx. (libcxx-abi was not needed to
> have a successful run). Would be great to make that a mandatory dependency.
> > 3) Besides that I recognized that "ninja check-lldb" target does not
> build all it's dependencies. E.g. lldb-server tests failed until I executed
> "ninja debugserver" before "ninja check-lldb". I am not sure, but I think
> "ninja obj2yaml" and "ninja llvm-pdbutil" are also prerequisites (and these
> targets seem not to be built by a simple "ninja" command).
> >
>
> Any chance you can submit fixes for this? Or, if you can't just open a
> PR and I'll take a look.
>
> --
> Davide
>
___
lldb-dev mailing list
lldb-dev@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev
Re: [lldb-dev] [cfe-dev] ASTImporter patches and improvements, please help
> Thanks for reporting this problem. I cc:ed Mike, as he owns the bots, and he should be able to help us with this. > (I do have access to the Jenkins configuration but I don't feel confident enough to make the change myself). Hi Davide and Mike, I have received an email from llvm.greendra...@gmail.com about a broken job of lldb-xcode. This is definitely an improvement and I would like to thank you. (This time it seems it is not my commit which breaks the bot, it looks like this is a commit related to breakpad). Cheers, Gabor On Sat, Dec 1, 2018 at 9:40 PM Davide Italiano wrote: > On Sat, Dec 1, 2018 at 8:00 AM Gábor Márton via cfe-dev > wrote: > > > > Dear LLDB Developers, > > > > There is an ongoing activity to improve ASTImporter in Clang to make it > support > > C++ (and C) completely and correctly. Our team works on cross > translation unit > > (CTU) static analysis where we use the in-tree Clang static analyzer in > > combination with the ASTImporter (via the scan-build.py script or > CodeChecker). > > In the past 18 months we have committed more than 70 patches to make the > > ASTImporter better. Our primary target is Linux and we do run the LLDB > tests > > on Linux before we commit. Unfortunately, quite recently 3 of these > patches > > broke the LLDB macOS buildbots (green.lab.llvm.org/green/view/LLDB) and > this > > caused some turmoil. We are very sorry about this and we are making > actions to > > avoid such inconveniences in the future: We ordered a Mac Mini, until it > > arrives we are using a borrowed Mac Book. We are going to create a CI > job which > > will execute the macOS LLDB test suite for a specific patch. Besides > this, for > > every patch we are going to monitor the macOS buildbots once they are > > committed. > > > > However, we are experiencing problems both with the buildbots and with > the LLDB > > tests, therefore we are asking help from the LLDB community in the > following: > > > > (1) Apparently, the green lab macOS buildbots are not displayed in the > build > > bot console (http://lab.llvm.org:8011/console). More importantly they > are not > > capable of sending emails to the authors of the commit in case of > failure. > > Would it be possible to setup the these buildbots to send out the emails > for > > the authors? > > > > Thanks for reporting this problem. I cc:ed Mike, as he owns the bots, > and he should be able to help us with this. > (I do have access to the Jenkins configuration but I don't feel > confident enough to make the change myself). > > > (2) We are facing difficulties with the LLDB lit tests both on macOS and > on > > Linux. E.g. on a fresh macOS mojave install, checking out master LLVM, > Clang, > > LLDB and then executing ninja check-lldb fails immediately. On Linux we > > experienced that some tests fail because they try to read a non-existent > > register. Some tests fail non-deterministically because they can't kill a > > process or a timeout is not long enough. Some tests fail because of a > linker > > error of libcxx. We would like to address all these issues. Would you > like to > > discuss these issues on lldb-dev or should we create a bugzilla ticket > for > > each? > > > > I think either of them is fine. If you're willing to spend some time > on this, I'm sure people on the LLDB side will consider helping you. > In particular, what's the error you get when lldb fails immediately > running the tests? > Also, have you checked libcxx and libcxx-abi in your build? We might > consider making that a mandatory dependency for the Cmake build. > > > (3) ASTImporter unit tests and lit tests in Clang for the LLDB specific > usage > > are very-very limited. LLDB uses the so called "minimal" import, but > none of > > the unit tests exercises that (CTU uses the full import). We should > have a > > unit test suite where the LLDB use cases are covered, e.g. after a > minimal > > import an `importDefinition` is called. Also, a test double which > implements > > the ExternalASTSource could be used. I believe it would be possible to > cover > > with the unit tests all LLDB/macOS related scenarios and these tests > could run > > on Linux too. We do something similar in case of windows: we execute > all unit > > tests with "-fdelayed-template-parsing" and with "-fms-compatibility" > too. I > > think, the more unit tests we have in Clang the sooner we catch the LLDB > > specific importer errors. I am asking the LLDB community's help here, > because > > we lack the domain knowledge in LLDB so we don't know what expectations > should > > we write in each unit test cases. (About lit tests, there is > clang-import-test > > but its coverage seems pretty low and I don't know if that really > exercises all > > LLDB use cases.) > > > > Definitely. +1. > > -- > Davide > ___ lldb-dev mailing list lldb-dev@lists.llvm.org http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev
[lldb-dev] Checkpoint/Restart with LLDB ?
Hi, Is there an LLDB counterpart to GDB's Checkpoint/Restart feature ( https://sourceware.org/gdb/onlinedocs/gdb/Checkpoint_002fRestart.html)? I could not find anything here http://lldb.llvm.org/lldb-gdb.html. Is there a way to mimic that feature somehow on macOS (or do reverse debugging) ? Thanks, Gabor ___ lldb-dev mailing list lldb-dev@lists.llvm.org http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev
[lldb-dev] [ASTImporter][macOS] Cannot link debugserver, keychain error
Hi, Recently I have updated the LLDB source and now I cannot link the debugserver. I guess, the problem is started in one of the commits at last week, because before that I had not experienced it. Any help would be appreciated. I'd like to get a command line only solution because I am building via ssh, so I can't click on any windows. With respect to Clang/ASTImporter, would it be a solution to disable the build of the debugserver and use the system debugserver? Do you think I would loose some ASTImporter related test coverage? How to disable the build of the debugserver? Thanks, Gabor [4/172] Linking CXX executable bin/debugserver FAILED: bin/debugserver : && /usr/local/opt/ccache/libexec/c++ -fPIC -fvisibility-inlines-hidden -Werror=date-time -Werror=unguarded-availability-new -std=c++11 -fmodules -fmodules-cache-path=/Users/egbomrt/llvm/build/release_assert/module.cache -fcxx-modules -Wall -Wextra -Wno-unused-parameter -Wwrite-strings -Wcast-qual -Wmissing-field-initializers -pedantic -Wno-long-long -Wimplicit-fallthrough -Wcovered-switch-default -Wno-noexcept-type -Wnon-virtual-dtor -Wdelete-non-virtual-dtor -Wstring-conversion -fdiagnostics-color -Wno-deprecated-declarations -Wno-unknown-pragmas -Wno-strict-aliasing -Wno-deprecated-register -Wno-vla-extension -Wno-gnu-zero-variadic-macro-arguments -Wno-zero-length-array -Wno-extended-offsetof -O3 -isysroot /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.14.sdk -Wl,-search_paths_first -Wl,-headerpad_max_install_names -stdlib=libc++ -Wl,-sectcreate,__TEXT,__info_plist,/Users/egbomrt/llvm/git/llvm/tools/lldb/tools/debugserver/source/../resources/lldb-debugserver-Info.plist -Wl,-dead_strip tools/lldb/tools/debugserver/source/CMakeFiles/debugserver.dir/debugserver.cpp.o -o bin/debugserver -Wl,-rpath,@loader_path/../lib lib/liblldbDebugserverCommon.a -framework Cocoa -framework CoreFoundation -framework Foundation lib/liblldbDebugserverArchSupport.a lib/liblldbDebugserverDarwin_DarwinLog.a -lcompression && cd /Users/egbomrt/llvm/build/release_assert/tools/lldb/tools/debugserver/source && /usr/local/Cellar/cmake/3.12.4/bin/cmake -E env CODESIGN_ALLOCATE=/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/codesign_allocate codesign --force --sign lldb_codesign --entitlements /Users/egbomrt/llvm/git/llvm/tools/lldb/tools/debugserver/source/../../../resources/debugserver-macosx-entitlements.plist /Users/egbomrt/llvm/build/release_assert/bin/debugserver *error: The specified item could not be found in the keychain.* ___ lldb-dev mailing list lldb-dev@lists.llvm.org http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev
Re: [lldb-dev] [ASTImporter][macOS] Cannot link debugserver, keychain error
Thank you Pavel! On Mon, Dec 10, 2018 at 8:51 PM Pavel Labath wrote: > On 10/12/2018 20:28, Gábor Márton via lldb-dev wrote: > > Hi, > > > > Recently I have updated the LLDB source and now I cannot link the > > debugserver. > > I guess, the problem is started in one of the commits at last week, > > because before that I had not experienced it. > > Any help would be appreciated. I'd like to get a command line only > > solution because I am building via ssh, so I can't click on any windows. > > With respect to Clang/ASTImporter, > > would it be a solution to disable the build of the debugserver and use > > the system debugserver? Do you think I would loose some ASTImporter > > related test coverage? How to disable the build of the debugserver? > > > > Thanks, > > Gabor > > > > [4/172] Linking CXX executable bin/debugserver > > FAILED: bin/debugserver > > : && /usr/local/opt/ccache/libexec/c++ -fPIC > > -fvisibility-inlines-hidden -Werror=date-time > > -Werror=unguarded-availability-new -std=c++11 -fmodules > > > -fmodules-cache-path=/Users/egbomrt/llvm/build/release_assert/module.cache > > -fcxx-modules -Wall -Wextra -Wno-unused-parameter -Wwrite-strings > > -Wcast-qual -Wmissing-field-initializers -pedantic -Wno-long-long > > -Wimplicit-fallthrough -Wcovered-switch-default -Wno-noexcept-type > > -Wnon-virtual-dtor -Wdelete-non-virtual-dtor -Wstring-conversion > > -fdiagnostics-color -Wno-deprecated-declarations -Wno-unknown-pragmas > > -Wno-strict-aliasing -Wno-deprecated-register -Wno-vla-extension > > -Wno-gnu-zero-variadic-macro-arguments -Wno-zero-length-array > > -Wno-extended-offsetof -O3 -isysroot > > > /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.14.sdk > > > -Wl,-search_paths_first -Wl,-headerpad_max_install_names -stdlib=libc++ > > > -Wl,-sectcreate,__TEXT,__info_plist,/Users/egbomrt/llvm/git/llvm/tools/lldb/tools/debugserver/source/../resources/lldb-debugserver-Info.plist > > >-Wl,-dead_strip > > > tools/lldb/tools/debugserver/source/CMakeFiles/debugserver.dir/debugserver.cpp.o > > > -o bin/debugserver -Wl,-rpath,@loader_path/../lib > > lib/liblldbDebugserverCommon.a -framework Cocoa -framework > > CoreFoundation -framework Foundation lib/liblldbDebugserverArchSupport.a > > lib/liblldbDebugserverDarwin_DarwinLog.a -lcompression && cd > > > /Users/egbomrt/llvm/build/release_assert/tools/lldb/tools/debugserver/source > > > && /usr/local/Cellar/cmake/3.12.4/bin/cmake -E env > > > CODESIGN_ALLOCATE=/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/codesign_allocate > > > codesign --force --sign lldb_codesign --entitlements > > > /Users/egbomrt/llvm/git/llvm/tools/lldb/tools/debugserver/source/../../../resources/debugserver-macosx-entitlements.plist > > > /Users/egbomrt/llvm/build/release_assert/bin/debugserver > > *error: The specified item could not be found in the keychain.* > > > > There is a setting to disable it. These days, you should be able to > achieve it with LLDB_CODESIGN_IDENTITY="" or SKIP_DEBUGSERVER=ON, though > that may change in the future (/me points at Stefan). > > > I don't think the choice of debugserver will impact the AST importer > tests in any way. > > pl > ___ lldb-dev mailing list lldb-dev@lists.llvm.org http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev
[lldb-dev] Failing tests on Linux
Hi, There are 6 tests in the lldb test framework (ninja check-lldb) which fail on Linux: Failing Tests (6): LLDB :: ExecControl/StopHook/stop-hook-threads.test lldb-Suite :: functionalities/data-formatter/data-formatter-stl/libcxx/function/TestLibCxxFunction.py lldb-Suite :: functionalities/data-formatter/data-formatter-stl/libcxx/list/loop/TestDataFormatterLibcxxListLoop.py lldb-Suite :: functionalities/data-formatter/data-formatter-stl/libcxx/variant/TestDataFormatterLibcxxVariant.py lldb-Suite :: functionalities/register/register_command/TestRegisters.py lldb-Suite :: tools/lldb-server/TestGdbRemoteRegisterState.py Seems like libcxx test issues are related, so I have created 4 new bugs: https://bugs.llvm.org/show_bug.cgi?id=39954 https://bugs.llvm.org/show_bug.cgi?id=39955 https://bugs.llvm.org/show_bug.cgi?id=39958 https://bugs.llvm.org/show_bug.cgi?id=39959 Thanks & Regards Gabor ___ lldb-dev mailing list lldb-dev@lists.llvm.org http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev
[lldb-dev] New test failures on Linux
Hi, I have discovered new test failures on Linux, on trunk. They seem to fail when DWO is used (DWARF test cases pass). https://bugs.llvm.org/show_bug.cgi?id=40048 https://bugs.llvm.org/show_bug.cgi?id=40049 https://bugs.llvm.org/show_bug.cgi?id=40050 Gabor ___ lldb-dev mailing list lldb-dev@lists.llvm.org http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev
Re: [lldb-dev] [ASTImporter][macOS] Cannot link debugserver, keychain error
Hi Stefan, I could not reproduce this on master. However, I don't want to use any code signing, so I use -DLLDB_USE_SYSTEM_DEBUGSERVER=ON, which is working fine for me. Thanks, Gabor On Fri, Jan 11, 2019 at 11:56 AM Stefan Gränitz wrote: > > Hi Gabor > > Is this behavior reproducible with the current upstream LLDB? (after > https://reviews.llvm.org/D55013) > Did you follow the instructions in docs/code-signing.txt? > > There is a setting to disable it. These days, you should be able to achieve > it with LLDB_CODESIGN_IDENTITY="" or SKIP_DEBUGSERVER=ON, though that may > change in the future (/me points at Stefan). > > > My changes that landed last week kept the default behavior intact: build > debugserver from source and code sign with lldb_codesign. > In order to change this you can configure with: > > * -DLLDB_USE_SYSTEM_DEBUGSERVER=ON to use a copy of the system's debugserver > instead > * -DLLDB_NO_DEBUGSERVER=ON to have no debugserver at all > > Best > Stefan > > On 10. Dec 2018, at 20:28, Gábor Márton via lldb-dev > wrote: > > Hi, > > Recently I have updated the LLDB source and now I cannot link the debugserver. > I guess, the problem is started in one of the commits at last week, because > before that I had not experienced it. > Any help would be appreciated. I'd like to get a command line only solution > because I am building via ssh, so I can't click on any windows. > With respect to Clang/ASTImporter, > would it be a solution to disable the build of the debugserver and use the > system debugserver? Do you think I would loose some ASTImporter related test > coverage? How to disable the build of the debugserver? > > Thanks, > Gabor > > [4/172] Linking CXX executable bin/debugserver > FAILED: bin/debugserver > : && /usr/local/opt/ccache/libexec/c++ -fPIC -fvisibility-inlines-hidden > -Werror=date-time -Werror=unguarded-availability-new -std=c++11 -fmodules > -fmodules-cache-path=/Users/egbomrt/llvm/build/release_assert/module.cache > -fcxx-modules -Wall -Wextra -Wno-unused-parameter -Wwrite-strings -Wcast-qual > -Wmissing-field-initializers -pedantic -Wno-long-long -Wimplicit-fallthrough > -Wcovered-switch-default -Wno-noexcept-type -Wnon-virtual-dtor > -Wdelete-non-virtual-dtor -Wstring-conversion -fdiagnostics-color > -Wno-deprecated-declarations -Wno-unknown-pragmas -Wno-strict-aliasing > -Wno-deprecated-register -Wno-vla-extension > -Wno-gnu-zero-variadic-macro-arguments -Wno-zero-length-array > -Wno-extended-offsetof -O3 -isysroot > /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.14.sdk > -Wl,-search_paths_first -Wl,-headerpad_max_install_names -stdlib=libc++ > -Wl,-sectcreate,__TEXT,__info_plist,/Users/egbomrt/llvm/git/llvm/tools/lldb/tools/debugserver/source/../resources/lldb-debugserver-Info.plist > -Wl,-dead_strip > tools/lldb/tools/debugserver/source/CMakeFiles/debugserver.dir/debugserver.cpp.o > -o bin/debugserver -Wl,-rpath,@loader_path/../lib > lib/liblldbDebugserverCommon.a -framework Cocoa -framework CoreFoundation > -framework Foundation lib/liblldbDebugserverArchSupport.a > lib/liblldbDebugserverDarwin_DarwinLog.a -lcompression && cd > /Users/egbomrt/llvm/build/release_assert/tools/lldb/tools/debugserver/source > && /usr/local/Cellar/cmake/3.12.4/bin/cmake -E env > CODESIGN_ALLOCATE=/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/codesign_allocate > codesign --force --sign lldb_codesign --entitlements > /Users/egbomrt/llvm/git/llvm/tools/lldb/tools/debugserver/source/../../../resources/debugserver-macosx-entitlements.plist > /Users/egbomrt/llvm/build/release_assert/bin/debugserver > error: The specified item could not be found in the keychain. > ___ > lldb-dev mailing list > lldb-dev@lists.llvm.org > http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev > > ___ lldb-dev mailing list lldb-dev@lists.llvm.org http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev
[lldb-dev] Build failure on Linux
Hi, On trunk I receive the following build error on Linux: ../../git/llvm/tools/lldb/include/lldb/lldb-forward.h:181:7: note: forward declaration of 'lldb_private::ProcessInstanceInfoList' class ProcessInstanceInfoList; ^ ../../git/llvm/tools/lldb/source/Host/linux/Host.cpp:260:23: error: member access into incomplete type 'lldb_private::ProcessInstanceInfoList' return process_infos.GetSize(); When I put back the below line into linux/Host.cpp #include "lldb/Target/Process.h" the error is gone. Based on this info, this commit could be blamed: commit e9a520afd257bf71f7ead0d33e4734008abf690c Author: Zachary Turner AuthorDate: Mon Mar 4 21:51:03 2019 + Commit: Zachary Turner CommitDate: Mon Mar 4 21:51:03 2019 + Move ProcessInfo from Host to Utility. Could you please take a look? Thanks, Gabor ___ lldb-dev mailing list lldb-dev@lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev
Re: [lldb-dev] Build failure on Linux
Yes, it works. Thanks! On Tue, Mar 5, 2019 at 4:54 PM Davide Italiano wrote: > > I think this might have been fixed, try to pull. > > On Tue, Mar 5, 2019 at 3:10 AM Gábor Márton via lldb-dev > wrote: > > > > Hi, > > > > On trunk I receive the following build error on Linux: > > ../../git/llvm/tools/lldb/include/lldb/lldb-forward.h:181:7: note: > > forward declaration of 'lldb_private::ProcessInstanceInfoList' > > class ProcessInstanceInfoList; > > ^ > > ../../git/llvm/tools/lldb/source/Host/linux/Host.cpp:260:23: error: > > member access into incomplete type > > 'lldb_private::ProcessInstanceInfoList' > > return process_infos.GetSize(); > > > > When I put back the below line into linux/Host.cpp > > #include "lldb/Target/Process.h" > > the error is gone. > > > > Based on this info, this commit could be blamed: > > commit e9a520afd257bf71f7ead0d33e4734008abf690c > > Author: Zachary Turner > > AuthorDate: Mon Mar 4 21:51:03 2019 + > > Commit: Zachary Turner > > CommitDate: Mon Mar 4 21:51:03 2019 + > > > > Move ProcessInfo from Host to Utility. > > > > Could you please take a look? > > > > Thanks, > > Gabor > > ___ > > lldb-dev mailing list > > lldb-dev@lists.llvm.org > > https://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev ___ lldb-dev mailing list lldb-dev@lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev
[lldb-dev] Flaky test on GreenDragon?
Hi guys, I've seen that recently the test lldb-Suite.macosx/queues.TestQueues.py fails non-deterministically. E.g. http://green.lab.llvm.org/green/view/LLDB/job/lldb-cmake/21271/testReport/ http://green.lab.llvm.org/green/view/LLDB/job/lldb-cmake/21244/testReport/ http://green.lab.llvm.org/green/view/LLDB/job/lldb-cmake/21229/testReport/ This is causing some inconvenience for us, e.g. our last commit triggered a build and we thought we broke something but the next build was successful without any action on our part. Could we disable this flaky test? Or is there some other solution? Thanks, Gabor ___ lldb-dev mailing list lldb-dev@lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev
[lldb-dev] Cross Translation Unit (CTU) static analysis milestone and status
Hi, Cross Translation Unit (CTU) static analysis on master(trunk) reached a milestone. Now we can CTU analyse a not trivial C project - TMUX in this case - without any crashes. We are preparing a buildbot which does this analysis of TMUX with the help of CodeChecker (https://github.com/Ericsson/codechecker/). We would like to include this bot in the network of llvm buildbots during the following weeks. We are getting closer and closer to be able to CTU analyse complex C++11 projects with the master branch. But before that, there are more patches to create and commit. This work is hard and slow, because most of the patches target the ASTImporter which is used in LLDB too, but with different parameters and set up. And of course we don't want to create regression for the LLDB folks. So, sometimes this work does involve creating LLDB patches as well. We continuously develop the ASTImporter in our fork, and we prepare patches for LLVM master. However, our fork will be always ahead of the LLVM master. Currently, we plan to create 30 more patches, and 6 are already on Phabricator. I you are interested about what do we plan to upstream have a look at here: https://github.com/Ericsson/clang/projects/2. If you'd like to get the full power of CTU analysis you should really try our fork: https://github.com/Ericsson/clang . We recommend to use it together with CodeChecker, since the scan-build script is abandoned. Our fork can analyse complex C++11(14) projects like bitcoin and protobuf without crashes. Currently we work on the analysis of LLVM/Clang itself, atm we fail to analyse 39 TUs out of ~2000. We regularly test and ship CodeChecker with this fork to our internal customers; there are dozens of projects which use the CTU analysis. Note that, however, all our projects are on Linux, so we focus on Linux. Currently the fork is based on Clang7, but soon we'll start rebasing to Clang8. Cheers, Gabor ___ lldb-dev mailing list lldb-dev@lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev
Re: [lldb-dev] Evaluating the same expression at the same breakpoint gets slower after a certain number of steps
Guilherme,
Could you please check if you have any structural differences between the
ASTs you see when
1) you first evaluate your expression at breakpoint A
2) you evaluate your expression the second time at breakpoint A ?
The AST of the expression evaluator's context is dumped once a TagDecl is
completed, but you need to enable a specific logging: (log enable lldb ast).
I have a theory about the root cause of the slowness you experience, but it
requires proof from your test scenario.
There are known problems with the lookup we use in LLDB [1].
If the lookup mechanism cannot find a symbol then clang::ASTImporter will
create a new AST node.
Thus I am expecting a growing AST in your case at 2) with duplicated and
redundant AST nodes.
Why would the grown AST results in any slowness?
Because the lookup we use is in `clang::DeclContext::localUncachedLookup()`
and it has a fallback branch which I assume is executed in your case:
```
// Slow case: grovel through the declarations in our chain looking for
// matches.
// FIXME: If we have lazy external declarations, this will not find them!
// FIXME: Should we CollectAllContexts and walk them all here?
for (Decl *D = FirstDecl; D; D = D->getNextDeclInContext()) {
if (auto *ND = dyn_cast(D))
if (ND->getDeclName() == Name)
Results.push_back(ND);
}
```
This for loop does a linear search on the list of the decls of a decl
context, which explains the slowing factor as the AST grows.
I wounder if you could help proving this theorem by first checking if the
AST grows and if yes then checking if this linear search is executed or not.
[1] If a symbol is in an `extern "C"` block then the existing lookup fails
to find it. I try to fix it in https://reviews.llvm.org/D61333
Thanks,
Gabor
On Thu, Jul 11, 2019 at 8:24 PM Jim Ingham via lldb-dev <
lldb-dev@lists.llvm.org> wrote:
> lldb realizes types from DWARF lazily. So for instance, if an expression
> refers to a pointer to type Foo, we won't necessarily realize the full type
> of Foo from DWARF to parse that expression. Then if you write a second
> expression that accesses a member of an object of type Foo, we will realize
> the full type for Foo. Then if you run the first expression again, the
> pointer to Foo type in the lldb type system will now point to a realized
> type of Foo. That should not make any difference, since if we were right
> the first time that we didn't need to know anything about Foo, it shouldn't
> matter whether the full type is realized or not.
>
> Similarly, the "expression with no side effects" could have also caused
> lldb to realize any number of other types. We find names from type
> information in two ways: looking in the realized types, and looking in the
> name indexes we read from DWARF or (on systems without accelerator tables)
> from indexing the DWARF manually. So the expression with no side effects
> will change the types in the realized types pool. That also "should not
> matter" because if the expression X was looking up by name, the lookup
> through the name indexes when the type wasn't realized yet should be
> equivalent to the lookup in the realized types.
>
> We really have no choice but to realize types lazily in this way, the
> debugger would just run way too slowly on big projects if we didn't. So
> the fact that one expression can change the state of the type system for a
> subsequent expression is a fact of life for lldb. But if the lookup
> mechanism is working properly, then these changes shouldn't matter. OTOH
> shouldn't and don't are not quite the same...
>
> Jim
>
>
> > On Jul 11, 2019, at 9:53 AM, Guilherme Andrade
> wrote:
> >
> > Speaking more generally, can evaluating an expression with no side
> effects affect the outcome of subsequent evaluations? I.e, is it possible
> for the expression evaluation X to produce different results in the
> following scenarios?
> >
> > Scenario 1: effects>
> > Scenario 2:evaluation X>
> >
> > Thanks!
> >
> > On Tue, Jul 9, 2019 at 2:52 PM Guilherme Andrade
> wrote:
> > Thanks for looking into this, Jason. I ended up realizing that it isn't
> about the number of evaluations; it is the exact location of the
> breakpoints that matters.
> >
> > I'm being able to consistently reproduce this by having two breakpoints
> (A and B) inside the same function, but B being inside the scope of a for
> loop that defines a couple extra variables. The first evaluation of
> 'undefinedVariable' at breakpoint A generates the following
> FindExternalLexicalDecls calls:
> >
> > FindExternalLexicalDecls[5] on (ASTContext*)01566104F030 in 'Type0'
> (CXXRecordDecl*)015668964438
> > FindExternalLexicalDecls[6] on (ASTContext*)01566104F030 in 'Type6'
> (CXXRecordDecl*)015668961BA8
> > FindExternalLexicalDecls[7] on (ASTContext*)01566104F030 in 'Type7'
> (CXXRecordDecl*)015668961A68
> > FindExternalLexicalDecls[8] on (ASTContext*)01566104F030 in 'Type8'
> (CXXRecordDecl*)015674F5
[lldb-dev] Cannot use system debugserver for testing
Hi Stefan, Since the commit "[CMake] Always build debugserver on Darwin and allow tests to use the system's one" I cannot use the system debugserver for testing. I receive the following error message from lldb when I execute "ninja check-lldb": ``` runCmd: run runCmd failed! error: process exited with status -1 (Error 1) ``` I do set up "-DLLDB_USE_SYSTEM_DEBUGSERVER=ON" with cmake so I see ``` -- LLDB tests use out-of-tree debugserver: /Library/Developer/CommandLineTools/Library/PrivateFrameworks/LLDB.framework/Resources/debugserver ``` Also, I have inspected the following test output ``` Command invoked: /usr/bin/python /Users/egbomrt/llvm2/git/llvm/tools/lldb/test/dotest.py -q --arch=x86_64 -s /Users/egbomrt/llvm2/build/release_assert/lldb-test-traces --build-dir /Users/egbomrt/llvm2/build/release_assert/lldb-test-build.noindex -S nm -u CXXFLAGS -u CFLAGS --executable /Users/egbomrt/llvm2/build/release_assert/./bin/lldb --dsymutil /Users/egbomrt/llvm2/build/release_assert/./bin/dsymutil --filecheck /Users/egbomrt/llvm2/build/release_assert/./bin/FileCheck -C /Users/egbomrt/llvm2/build/release_assert/bin/clang --codesign-identity - --out-of-tree-debugserver --arch x86_64 -t --env TERM=vt100 -p TestCModules.py --results-port 49931 -S nm --inferior -p TestCModules.py /Users/egbomrt/llvm2/git/llvm/tools/lldb/packages/Python/lldbsuite/test/lang/c/modules --event-add-entries worker_index=0:int 1 out of 736 test suites processed - TestCModules.py ``` so it seems like the argument for --out-of-tree-debugserver is missing... Could you please advise? Thank you, Gabor ___ lldb-dev mailing list lldb-dev@lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev
Re: [lldb-dev] Cannot use system debugserver for testing
This might not be related to the debugserver, I just realized that I get "error: process exited with status -1 (Error 1)" even with the simplest main.c. This may be some kind of security issue on mac OS... Though I've checked and I have SIP disabled and I have executed "sudo /usr/sbin/DevToolsSecurity --enable". On Fri, Jul 19, 2019 at 4:46 PM Gábor Márton wrote: > Hi Stefan, > > Since the commit > "[CMake] Always build debugserver on Darwin and allow tests to use the > system's one" > I cannot use the system debugserver for testing. > I receive the following error message from lldb when I execute "ninja > check-lldb": > ``` > runCmd: run > runCmd failed! > error: process exited with status -1 (Error 1) > ``` > > I do set up "-DLLDB_USE_SYSTEM_DEBUGSERVER=ON" with cmake so I see > ``` > -- LLDB tests use out-of-tree debugserver: > /Library/Developer/CommandLineTools/Library/PrivateFrameworks/LLDB.framework/Resources/debugserver > ``` > > Also, I have inspected the following test output > ``` > Command invoked: /usr/bin/python > /Users/egbomrt/llvm2/git/llvm/tools/lldb/test/dotest.py -q --arch=x86_64 -s > /Users/egbomrt/llvm2/build/release_assert/lldb-test-traces --build-dir > /Users/egbomrt/llvm2/build/release_assert/lldb-test-build.noindex -S nm -u > CXXFLAGS -u CFLAGS --executable > /Users/egbomrt/llvm2/build/release_assert/./bin/lldb --dsymutil > /Users/egbomrt/llvm2/build/release_assert/./bin/dsymutil --filecheck > /Users/egbomrt/llvm2/build/release_assert/./bin/FileCheck -C > /Users/egbomrt/llvm2/build/release_assert/bin/clang --codesign-identity - > --out-of-tree-debugserver --arch x86_64 -t --env TERM=vt100 -p > TestCModules.py --results-port 49931 -S nm --inferior -p TestCModules.py > /Users/egbomrt/llvm2/git/llvm/tools/lldb/packages/Python/lldbsuite/test/lang/c/modules > --event-add-entries worker_index=0:int > 1 out of 736 test suites processed - TestCModules.py > ``` > so it seems like the argument for --out-of-tree-debugserver is missing... > > Could you please advise? > > Thank you, > Gabor > ___ lldb-dev mailing list lldb-dev@lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev
Re: [lldb-dev] Cannot use system debugserver for testing
Actually, it is embarrassing (perhaps for macOS and not for me) that after a reboot the problem is gone. Perhaps after "sudo /usr/sbin/DevToolsSecurity --enable" a reboot is required, but could not find anything official about that. On Fri, Jul 19, 2019 at 7:20 PM Gábor Márton wrote: > This might not be related to the debugserver, I just realized that I get > "error: process exited with status -1 (Error 1)" > even with the simplest main.c. > This may be some kind of security issue on mac OS... > Though I've checked and I have SIP disabled and I have executed "sudo > /usr/sbin/DevToolsSecurity --enable". > > On Fri, Jul 19, 2019 at 4:46 PM Gábor Márton > wrote: > >> Hi Stefan, >> >> Since the commit >> "[CMake] Always build debugserver on Darwin and allow tests to use the >> system's one" >> I cannot use the system debugserver for testing. >> I receive the following error message from lldb when I execute "ninja >> check-lldb": >> ``` >> runCmd: run >> runCmd failed! >> error: process exited with status -1 (Error 1) >> ``` >> >> I do set up "-DLLDB_USE_SYSTEM_DEBUGSERVER=ON" with cmake so I see >> ``` >> -- LLDB tests use out-of-tree debugserver: >> /Library/Developer/CommandLineTools/Library/PrivateFrameworks/LLDB.framework/Resources/debugserver >> ``` >> >> Also, I have inspected the following test output >> ``` >> Command invoked: /usr/bin/python >> /Users/egbomrt/llvm2/git/llvm/tools/lldb/test/dotest.py -q --arch=x86_64 -s >> /Users/egbomrt/llvm2/build/release_assert/lldb-test-traces --build-dir >> /Users/egbomrt/llvm2/build/release_assert/lldb-test-build.noindex -S nm -u >> CXXFLAGS -u CFLAGS --executable >> /Users/egbomrt/llvm2/build/release_assert/./bin/lldb --dsymutil >> /Users/egbomrt/llvm2/build/release_assert/./bin/dsymutil --filecheck >> /Users/egbomrt/llvm2/build/release_assert/./bin/FileCheck -C >> /Users/egbomrt/llvm2/build/release_assert/bin/clang --codesign-identity - >> --out-of-tree-debugserver --arch x86_64 -t --env TERM=vt100 -p >> TestCModules.py --results-port 49931 -S nm --inferior -p TestCModules.py >> /Users/egbomrt/llvm2/git/llvm/tools/lldb/packages/Python/lldbsuite/test/lang/c/modules >> --event-add-entries worker_index=0:int >> 1 out of 736 test suites processed - TestCModules.py >> ``` >> so it seems like the argument for --out-of-tree-debugserver is missing... >> >> Could you please advise? >> >> Thank you, >> Gabor >> > ___ lldb-dev mailing list lldb-dev@lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev
Re: [lldb-dev] Cannot use system debugserver for testing
I am still struggling with this issue. Now I decided to work with the codesigned version of the debugserver, becasue I had an error when I tried to use the system debugserver. So I've run scripts/macos-setup-codesign.sh After a reboot and fresh build (I have removed the CMakeCache.txt and the whole build dir) I have the debugserver signed: ``` $ codesign -d ~/llvm2/build/release_assert/bin/debugserver Executable=/Users/egbomrt/llvm2/build/release_assert/bin/debugserver Identifier=com.apple.debugserver Format=Mach-O thin (x86_64) CodeDirectory v=20100 size=38534 flags=0x0(none) hashes=1197+5 location=embedded VersionPlatform=1 VersionMin=658944 VersionSDK=658944 Hash type=sha256 size=32 CandidateCDHash sha256=7b475cfa7127c84281ceb206093d13dd464dad74 Hash choices=sha256 Page size=4096 CDHash=7b475cfa7127c84281ceb206093d13dd464dad74 Signature size=1611 Authority=lldb_codesign Signed Time=2019. Jul 22. 15:26:29 Info.plist entries=6 TeamIdentifier=not set Sealed Resources=none Internal requirements count=1 size=100 $ ``` So far so good. But then when I try to use lldb I have permission problems: ``` egbomrt@msmarple ~/llvm2/build/release_assert $ ./bin/lldb /bin/ls (lldb) target create "/bin/ls" Current executable set to '/bin/ls' (x86_64). (lldb) r *error: process exited with status -1 (Error 1)* (lldb) ^D egbomrt@msmarple ~/llvm2/build/release_assert $ ``` However, as root I can use lldb: ``` egbomrt@msmarple ~/llvm2/build/release_assert $ sudo ./bin/lldb /bin/ls (lldb) target create "/bin/ls" Current executable set to '/bin/ls' (x86_64). (lldb) r Process 28052 launched: '/bin/ls' (x86_64) .ninja_deps compile_commands.json .ninja_log docs CMakeCache.txt examples CMakeDoxyfile.ininclude ... Process 28052 exited with status = 0 (0x) (lldb) ^D egbomrt@msmarple ~/llvm2/build/release_assert $ ``` Is it possible to codesign in a way that a regular user can run the built debugserver? Or what else could be the reason behind this permission problem? Thanks, Gabor On Fri, Jul 19, 2019 at 11:47 PM Stefan Gränitz wrote: > Hi Gábor, I am sorry this caused an issue for you. Good that apparently > it's resolved now. > > Did you reconfigure an existing build-tree? Your observations would make > sense in this context, because the change affects CMake cached variables. > This is unfortunate, but can not always be avoided. If this happens again > (or to anyone else), a clean build seems to be a good first step. > > Best, > Stefan > > On 19/07/2019 19:36, Gábor Márton wrote: > > Actually, it is embarrassing (perhaps for macOS and not for me) that after > a reboot the problem is gone. > Perhaps after "sudo /usr/sbin/DevToolsSecurity --enable" a reboot is > required, but could not find anything official about that. > > On Fri, Jul 19, 2019 at 7:20 PM Gábor Márton > wrote: > >> This might not be related to the debugserver, I just realized that I get >> "error: process exited with status -1 (Error 1)" >> even with the simplest main.c. >> This may be some kind of security issue on mac OS... >> Though I've checked and I have SIP disabled and I have executed "sudo >> /usr/sbin/DevToolsSecurity --enable". >> >> On Fri, Jul 19, 2019 at 4:46 PM Gábor Márton >> wrote: >> >>> Hi Stefan, >>> >>> Since the commit >>> "[CMake] Always build debugserver on Darwin and allow tests to use the >>> system's one" >>> I cannot use the system debugserver for testing. >>> I receive the following error message from lldb when I execute "ninja >>> check-lldb": >>> ``` >>> runCmd: run >>> runCmd failed! >>> error: process exited with status -1 (Error 1) >>> ``` >>> >>> I do set up "-DLLDB_USE_SYSTEM_DEBUGSERVER=ON" with cmake so I see >>> ``` >>> -- LLDB tests use out-of-tree debugserver: >>> /Library/Developer/CommandLineTools/Library/PrivateFrameworks/LLDB.framework/Resources/debugserver >>> ``` >>> >>> Also, I have inspected the following test output >>> ``` >>> Command invoked: /usr/bin/python >>> /Users/egbomrt/llvm2/git/llvm/tools/lldb/test/dotest.py -q --arch=x86_64 -s >>> /Users/egbomrt/llvm2/build/release_assert/lldb-test-traces --build-dir >>> /Users/egbomrt/llvm2/build/release_assert/lldb-test-build.noindex -S nm -u >>> CXXFLAGS -u CFLAGS --executable >>> /Users/egbomrt/llvm2/build/release_assert/./bin/lldb --dsymutil >>> /Users/egbomrt/llvm2/build/release_assert/./bin/dsymutil --filecheck >>> /Users/egbomrt/llvm2/build/release_assert/./bin/FileCheck -C >>> /Users/egbomrt/llvm2/build/release_assert/bin/clang --codesign-identity - >>> --out-of-tree-debugserver --arch x86_64 -t --env TERM=vt100 -p >>> TestCModules.py --results-port 49931 -S nm --inferior -p TestCModules.py >>> /Users/egbomrt/llvm2/git/llvm/tools/lldb/packages/Python/lldbsuite/test/lang/c/modules >>> --event-add-entries worker_index=0:int >>> 1 out of 736 test suites processed - TestCModules.py >>> ``` >>> so it seems like the argument for --out-of-tree-debugserver is missing... >>> >>> C
Re: [lldb-dev] Cannot use system debugserver for testing
Well, SIP is turned off and I experience the same with a binary I just built: ``` egbomrt@msmarple ~/llvm2/build/release_assert $ csrutil status System Integrity Protection status: disabled. egbomrt@msmarple ~/llvm2/build/release_assert $ ./bin/lldb ~/a.out (lldb) target create "/Users/egbomrt/a.out" Current executable set to '/Users/egbomrt/a.out' (x86_64). (lldb) r error: process exited with status -1 (Error 1) (lldb) ^D egbomrt@msmarple ~/llvm2/build/release_assert $ ls -la ~/a.out -rwxr-xr-x 1 egbomrt admin 8736 Júl 22 16:16 /Users/egbomrt/a.out egbomrt@msmarple ~/llvm2/build/release_assert $ ``` On Mon, Jul 22, 2019 at 4:29 PM Stefan Gränitz wrote: > egbomrt@msmarple ~/llvm2/build/release_assert $ ./bin/lldb /bin/ls > (lldb) target create "/bin/ls" > Current executable set to '/bin/ls' (x86_64). > (lldb) r > *error: process exited with status -1 (Error 1)* > > I don't think this is related to debugserver codesigning. If you really > need to debug system binaries, you may need to disable SIP. > > On 22/07/2019 16:14, Gábor Márton wrote: > > I am still struggling with this issue. Now I decided to work with the > codesigned version of the debugserver, becasue I had an error when I tried > to use the system debugserver. > So I've run scripts/macos-setup-codesign.sh > After a reboot and fresh build (I have removed the CMakeCache.txt and the > whole build dir) I have the debugserver signed: > ``` > $ codesign -d ~/llvm2/build/release_assert/bin/debugserver > Executable=/Users/egbomrt/llvm2/build/release_assert/bin/debugserver > Identifier=com.apple.debugserver > Format=Mach-O thin (x86_64) > CodeDirectory v=20100 size=38534 flags=0x0(none) hashes=1197+5 > location=embedded > VersionPlatform=1 > VersionMin=658944 > VersionSDK=658944 > Hash type=sha256 size=32 > CandidateCDHash sha256=7b475cfa7127c84281ceb206093d13dd464dad74 > Hash choices=sha256 > Page size=4096 > CDHash=7b475cfa7127c84281ceb206093d13dd464dad74 > Signature size=1611 > Authority=lldb_codesign > Signed Time=2019. Jul 22. 15:26:29 > Info.plist entries=6 > TeamIdentifier=not set > Sealed Resources=none > Internal requirements count=1 size=100 > $ > ``` > > So far so good. > But then when I try to use lldb I have permission problems: > ``` > egbomrt@msmarple ~/llvm2/build/release_assert $ ./bin/lldb /bin/ls > (lldb) target create "/bin/ls" > Current executable set to '/bin/ls' (x86_64). > (lldb) r > *error: process exited with status -1 (Error 1)* > (lldb) ^D > egbomrt@msmarple ~/llvm2/build/release_assert $ > ``` > > However, as root I can use lldb: > ``` > egbomrt@msmarple ~/llvm2/build/release_assert $ sudo ./bin/lldb /bin/ls > (lldb) target create "/bin/ls" > Current executable set to '/bin/ls' (x86_64). > (lldb) r > Process 28052 launched: '/bin/ls' (x86_64) > .ninja_deps compile_commands.json > .ninja_log docs > CMakeCache.txt examples > CMakeDoxyfile.ininclude > ... > Process 28052 exited with status = 0 (0x) > (lldb) ^D > egbomrt@msmarple ~/llvm2/build/release_assert $ > ``` > > Is it possible to codesign in a way that a regular user can run the built > debugserver? Or what else could be the reason behind this permission > problem? > > Thanks, > Gabor > > On Fri, Jul 19, 2019 at 11:47 PM Stefan Gränitz > wrote: > >> Hi Gábor, I am sorry this caused an issue for you. Good that apparently >> it's resolved now. >> >> Did you reconfigure an existing build-tree? Your observations would make >> sense in this context, because the change affects CMake cached variables. >> This is unfortunate, but can not always be avoided. If this happens again >> (or to anyone else), a clean build seems to be a good first step. >> >> Best, >> Stefan >> >> On 19/07/2019 19:36, Gábor Márton wrote: >> >> Actually, it is embarrassing (perhaps for macOS and not for me) that >> after a reboot the problem is gone. >> Perhaps after "sudo /usr/sbin/DevToolsSecurity --enable" a reboot is >> required, but could not find anything official about that. >> >> On Fri, Jul 19, 2019 at 7:20 PM Gábor Márton >> wrote: >> >>> This might not be related to the debugserver, I just realized that I >>> get >>> "error: process exited with status -1 (Error 1)" >>> even with the simplest main.c. >>> This may be some kind of security issue on mac OS... >>> Though I've checked and I have SIP disabled and I have executed "sudo >>> /usr/sbin/DevToolsSecurity --enable". >>> >>> On Fri, Jul 19, 2019 at 4:46 PM Gábor Márton >>> wrote: >>> Hi Stefan, Since the commit "[CMake] Always build debugserver on Darwin and allow tests to use the system's one" I cannot use the system debugserver for testing. I receive the following error message from lldb when I execute "ninja check-lldb": ``` runCmd: run runCmd failed! error: process exited with status -1 (Error 1) ``` I do set up "-DLLDB_USE_SYSTEM_DEBUGSERVER=ON" with cmake so I see
Re: [lldb-dev] Cannot use system debugserver for testing
Yes, here it is. egbomrt@msmarple ~/llvm2/build/release_assert $ ./bin/lldb ~/a.out (lldb) target create "/Users/egbomrt/a.out" Current executable set to '/Users/egbomrt/a.out' (x86_64). (lldb) log enable lldb default (lldb) r Processing command: r HandleCommand, cmd_obj : 'process launch' HandleCommand, (revised) command_string: 'process launch -X true --' HandleCommand, wants_raw_input:'False' HandleCommand, command line after removing command name(s): '-X true --' Target::Launch() called for /Users/egbomrt/a.out Target::Launch the process instance doesn't currently exist. have platform=true, platform_sp->IsHost()=true, default_to_use_pty=true at least one of stdin/stdout/stderr was not set, evaluating default handling target stdin='(empty)', target stdout='(empty)', stderr='(empty)' Generating a pty to use for stdin/out/err Target::Launch asking the platform to debug the process Host::StartMonitoringChildProcess (callback, pid=94887, monitor_signals=0) source = 0x7f9bb923ec10 ::waitpid (pid = 94887, &status, 0) => pid = 94887, status = 0x (EXITED), signal = 0, exit_status = 0 Host::StartMonitoringChildProcess (callback, pid=94888, monitor_signals=0) source = 0x7f9bb9218180 Went to stop the private state thread, but it was already invalid. Process::SetPublicState (state = attaching, restarted = 0) Host::StartMonitoringChildProcess (callback, pid=94889, monitor_signals=0) source = 0x7f9bb9243bd0 thread created Process::AttachCompletionHandler::AttachCompletionHandler process=0x7f9bb8803c18, exec_count=0 Process::ControlPrivateStateThread (signal = 4) Sending control event of type: 4. thread created Process::RunPrivateStateThread (arg = 0x7f9bb8803c18, pid = 94888) thread starting... timeout = , event_sp)... Process::RunPrivateStateThread (arg = 0x7f9bb8803c18, pid = 94888) got a control event: 4 timeout = , event_sp)... timeout = timeout = , event_sp)... thread created Process::SetExitStatus (status=-1 (0x), description="Error 1") Process::SetPrivateState (exited) Process::SetPrivateState (exited) stop_id = 1 Process::AttachCompletionHandler::PerformAction called with state exited (10) Ran next event action, result was 2. Process::ShouldBroadcastEvent (0x7f9bb92423b0) => new state: exited, last broadcast state: exited - YES Process::HandlePrivateEvent (pid = 94888) broadcasting new state exited (old state attaching) to hijacked Process::RunPrivateStateThread (arg = 0x7f9bb8803c18, pid = 94888) about to exit with internal state exited... Process::RunPrivateStateThread (arg = 0x7f9bb8803c18, pid = 94888) thread exiting... Process::SetPublicState (state = exited, restarted = 0) timeout = , event_sp) => exited HandleCommand, command did not succeed error: process exited with status -1 (Error 1) (lldb) ::waitpid (pid = 94889, &status, 0) => pid = 94889, status = 0x (EXITED), signal = 0, exit_status = 0 (lldb) On Mon, Jul 22, 2019 at 4:44 PM Stefan Gränitz wrote: > Interesting. Is there any extra info dumped to the log (e.g. `log enable > lldb default`) > > On 22/07/2019 16:34, Gábor Márton wrote: > > Well, SIP is turned off and I experience the same with a binary I just > built: > ``` > egbomrt@msmarple ~/llvm2/build/release_assert $ csrutil status > System Integrity Protection status: disabled. > egbomrt@msmarple ~/llvm2/build/release_assert $ ./bin/lldb ~/a.out > (lldb) target create "/Users/egbomrt/a.out" > Current executable set to '/Users/egbomrt/a.out' (x86_64). > (lldb) r > error: process exited with status -1 (Error 1) > (lldb) ^D > egbomrt@msmarple ~/llvm2/build/release_assert $ ls -la ~/a.out > -rwxr-xr-x 1 egbomrt admin 8736 Júl 22 16:16 /Users/egbomrt/a.out > egbomrt@msmarple ~/llvm2/build/release_assert $ > ``` > > On Mon, Jul 22, 2019 at 4:29 PM Stefan Gränitz > wrote: > >> egbomrt@msmarple ~/llvm2/build/release_assert $ ./bin/lldb /bin/ls >> (lldb) target create "/bin/ls" >> Current executable set to '/bin/ls' (x86_64). >> (lldb) r >> *error: process exited with status -1 (Error 1)* >> >> I don't think this is related to debugserver codesigning. If you really >> need to debug system binaries, you may need to disable SIP. >> >> On 22/07/2019 16:14, Gábor Márton wrote: >> >> I am still struggling with this issue. Now I decided to work with the >> codesigned version of the debugserver, becasue I had an error when I tried >> to use the system debugserver. >> So I've run scripts/macos-setup-codesign.sh >> After a reboot and fresh build (I have removed the CMakeCache.txt and the >> whole build dir) I have the debugserver signed: >> ``` >> $ codesign -d ~/llvm2/build/release_assert/bin/debugserver >> Executable=/Users/egbomrt/llvm2/build/release_assert/bin/debugserver >> Identifier=com.apple.debugserver >> Format=Mach-O thin (x86_64) >> CodeDirectory v=20100 size=38534 flags=0x0(none) hashes=1197+5 >> location=embedded >> VersionPlatform=1 >> VersionMin=658944 >> VersionSDK=658944 >> Hash type=sha256 size=32 >> Candidate
Re: [lldb-dev] Cannot use system debugserver for testing
Some more info: I've stopped lldb (with the system lldb) when
" Process::SetExitStatus (status=-1 (0x), description="Error 1")"
is logged and gathered some more information.
Is it possible that some packet types are not implemented?
* frame #0: 0x7fff599452c6 libsystem_kernel.dylib`__pthread_kill + 10
frame #1: 0x7fff59a00bf1 libsystem_pthread.dylib`pthread_kill + 284
frame #2: 0x7fff598af6a6 libsystem_c.dylib`abort + 127
frame #3: 0x7fff5987820d libsystem_c.dylib`__assert_rtn + 324
frame #4: 0x00010b393539
liblldb.10.0.99svn.dylib`lldb_private::Process::SetExitStatus(this=0x7fcfaa470218,
status=-1, cstr="Error 1") at Process.cpp:1149:3
frame #5: 0x00010bbd1792
liblldb.10.0.99svn.dylib`lldb_private::process_gdb_remote::ProcessGDBRemote::AsyncThread(arg=0x7fcfaa470218)
at ProcessGDBRemote.cpp:3877:28
frame #6: 0x00010b08f87b
liblldb.10.0.99svn.dylib`lldb_private::HostNativeThreadBase::ThreadCreateTrampoline(arg=0x7fcfab218740)
at HostNativeThreadBase.cpp:69:10
frame #7: 0x000112be72fd
liblldb.10.0.99svn.dylib`lldb_private::HostThreadMacOSX::ThreadCreateTrampoline(arg=0x7fcfab218740)
at HostThreadMacOSX.mm:68:10
frame #8: 0x7fff599fe2eb libsystem_pthread.dylib`_pthread_body + 126
frame #9: 0x7fff59a01249 libsystem_pthread.dylib`_pthread_start + 66
frame #10: 0x7fff599fd40d libsystem_pthread.dylib`thread_start + 13
(lldb) frame select 5
frame #5: 0x00010bbd1792
liblldb.10.0.99svn.dylib`lldb_private::process_gdb_remote::ProcessGDBRemote::AsyncThread(arg=0x7fcfaa470218)
at ProcessGDBRemote.cpp:3877:28
3874 "System Integrity
Protection");
3875 } else if (::strstr(continue_cstr, "vAttach") !=
nullptr &&
3876response.GetStatus().Fail()) {
-> 3877 process->SetExitStatus(-1,
response.GetStatus().AsCString());
3878 } else {
3879 process->SetExitStatus(-1, "lost connection");
3880 }
(lldb) p response.GetError()
(uint8_t) $0 = '\x01'
(lldb) p response.GetServerPacketType()
(StringExtractorGDBRemote::ServerPacketType) $1 =
*eServerPacketType_unimplemented*
(lldb) p response.GetResponseType()
(StringExtractorGDBRemote::ResponseType) $2 = eError
(lldb) p response.IsUnsupportedResponse()
(bool) $3 = false
(lldb) p response.GetStatus()
(lldb_private::Status) $4 = (m_code = 1, m_type = eErrorTypeGeneric,
m_string = "Error 1")
Thanks,
Gabor
On Mon, Jul 22, 2019 at 4:59 PM Gábor Márton wrote:
> Yes, here it is.
>
> egbomrt@msmarple ~/llvm2/build/release_assert $ ./bin/lldb ~/a.out
> (lldb) target create "/Users/egbomrt/a.out"
> Current executable set to '/Users/egbomrt/a.out' (x86_64).
> (lldb) log enable lldb default
> (lldb) r
> Processing command: r
> HandleCommand, cmd_obj : 'process launch'
> HandleCommand, (revised) command_string: 'process launch -X true --'
> HandleCommand, wants_raw_input:'False'
> HandleCommand, command line after removing command name(s): '-X true --'
> Target::Launch() called for /Users/egbomrt/a.out
> Target::Launch the process instance doesn't currently exist.
> have platform=true, platform_sp->IsHost()=true, default_to_use_pty=true
> at least one of stdin/stdout/stderr was not set, evaluating default
> handling
> target stdin='(empty)', target stdout='(empty)', stderr='(empty)'
> Generating a pty to use for stdin/out/err
> Target::Launch asking the platform to debug the process
> Host::StartMonitoringChildProcess (callback, pid=94887,
> monitor_signals=0) source = 0x7f9bb923ec10
>
> ::waitpid (pid = 94887, &status, 0) => pid = 94887, status = 0x
> (EXITED), signal = 0, exit_status = 0
> Host::StartMonitoringChildProcess (callback, pid=94888,
> monitor_signals=0) source = 0x7f9bb9218180
>
> Went to stop the private state thread, but it was already invalid.
> Process::SetPublicState (state = attaching, restarted = 0)
> Host::StartMonitoringChildProcess (callback, pid=94889,
> monitor_signals=0) source = 0x7f9bb9243bd0
>
> thread created
> Process::AttachCompletionHandler::AttachCompletionHandler
> process=0x7f9bb8803c18, exec_count=0
> Process::ControlPrivateStateThread (signal = 4)
> Sending control event of type: 4.
> thread created
> Process::RunPrivateStateThread (arg = 0x7f9bb8803c18, pid = 94888) thread
> starting...
> timeout = , event_sp)...
> Process::RunPrivateStateThread (arg = 0x7f9bb8803c18, pid = 94888) got a
> control event: 4
> timeout = , event_sp)...
> timeout =
> timeout = , event_sp)...
> thread created
> Process::SetExitStatus (status=-1 (0x), description="Error 1")
> Process::SetPrivateState (exited)
> Process::SetPrivateState (exited) stop_id = 1
> Process::AttachCompletionHandler::PerformAction called with state exited
> (10)
> Ran next event action, result was 2.
> Process::ShouldBroadcastEvent (0x7f9bb92423b0) => new state: ex
[lldb-dev] ASTImporter Tutorial/TechTalk and RoundTable
Hi, I am planning to submit a talk about ASTImporter to the upcoming Dev Meeting at October 22-23, 2019. I'd like to talk about - the API, - how it is used in CTU analysis and in LLDB, - internal subtleties and difficulties (Error handling, ExternalASTSource, ...) The goal would be to attract more developers to use and improve ASTImporter related code, so perhaps this will be a tutorial. Independently from the talk, I'd like to have a round table discussion if there is enough interest. Some topics could cover future development ideas and existing problems we have. Please get back to me if you are interested and think about the topics you have in mind, also don't forget to buy your ticket to the DevMeeting. Thanks, Gabor ___ lldb-dev mailing list lldb-dev@lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev
Re: [lldb-dev] Cannot use system debugserver for testing
Ok, I've done that, here are the logs which happen before the first
debugserver error:
error 18:33:20.236506 +0200 taskgated cannot open file at line 42270 of
[95fbac39ba]
error 18:33:20.236526 +0200 taskgated os_unix.c:42270: (2)
open(/var/db/DetachedSignatures) - No such file or directory
default 18:33:20.236586 +0200 taskgated MacOS error: -67062
error 18:33:20.246771 +0200 taskgated cannot open file at line 42270 of
[95fbac39ba]
error 18:33:20.246787 +0200 taskgated os_unix.c:42270: (2)
open(/var/db/DetachedSignatures) - No such file or directory
default 18:33:20.246841 +0200 taskgated MacOS error: -67062
default 18:33:20.260319 +0200 debugserver debugserver will use os_log for
internal logging.
default 18:33:20.260491 +0200 debugserver debugserver-@(#)PROGRAM:LLDB
PROJECT:lldb-360.99.0
for x86_64.
default 18:33:20.260615 +0200 debugserver Got a connection, waiting for
process information for launching or attaching.
default 18:33:20.264541 +0200 trustd cert[0]: AnchorTrusted =(leaf)[force]>
0
default 18:33:20.272256 +0200 trustd cert[2]: AnchorTrusted =(leaf)[force]>
0
default 18:33:20.276567 +0200 trustd cert[2]: AnchorTrusted =(leaf)[force]>
0
default 18:33:20.278680 +0200 authd UNIX error exception: 3
error 18:33:20.279462 +0200 authd process: PID 27648 failed to create code
ref 13
error 18:33:20.280017 +0200 authd Fatal: interaction not allowed (session
has no ui access) (engine 3727)
default 18:33:20.280031 +0200 authd Failed to authorize right
'system.privilege.taskport' by client '/usr/libexec/taskgated' [254] for
authorization created by '/usr/libexec/taskgated' [27648] (3,1) (-60007)
(engine 3727)
error 18:33:20.280092 +0200 authd copy_rights: authorization failed
error 18:33:20.280442 +0200 debugserver error:
MachTask::TaskPortForProcessID task_for_pid failed: ::task_for_pid (
target_tport = 0x0103, pid = 27646, &task ) => err = 0x0005 ((os/kern)
failure)
On Mon, Jul 22, 2019 at 6:10 PM Frédéric Riss wrote:
> I think the system logs (in Console.app) would tell us more. Search for
> debugserver and you should find attach failures. Then remove the filter and
> look at what happens right before that. There should be a log from
> taskgated or authd that is a little more explicit about what’s failing.
>
> Fred
>
> On Jul 22, 2019, at 8:55 AM, Gábor Márton via lldb-dev <
> lldb-dev@lists.llvm.org> wrote:
>
> Some more info: I've stopped lldb (with the system lldb) when
> " Process::SetExitStatus (status=-1 (0x), description="Error 1")"
> is logged and gathered some more information.
> Is it possible that some packet types are not implemented?
>
> * frame #0: 0x7fff599452c6 libsystem_kernel.dylib`__pthread_kill + 10
> frame #1: 0x7fff59a00bf1 libsystem_pthread.dylib`pthread_kill + 284
> frame #2: 0x7fff598af6a6 libsystem_c.dylib`abort + 127
> frame #3: 0x7fff5987820d libsystem_c.dylib`__assert_rtn + 324
> frame #4: 0x00010b393539
> liblldb.10.0.99svn.dylib`lldb_private::Process::SetExitStatus(this=0x7fcfaa470218,
> status=-1, cstr="Error 1") at Process.cpp:1149:3
> frame #5: 0x00010bbd1792
> liblldb.10.0.99svn.dylib`lldb_private::process_gdb_remote::ProcessGDBRemote::AsyncThread(arg=0x7fcfaa470218)
> at ProcessGDBRemote.cpp:3877:28
> frame #6: 0x00010b08f87b
> liblldb.10.0.99svn.dylib`lldb_private::HostNativeThreadBase::ThreadCreateTrampoline(arg=0x7fcfab218740)
> at HostNativeThreadBase.cpp:69:10
> frame #7: 0x000112be72fd
> liblldb.10.0.99svn.dylib`lldb_private::HostThreadMacOSX::ThreadCreateTrampoline(arg=0x7fcfab218740)
> at HostThreadMacOSX.mm:68:10
> frame #8: 0x7fff599fe2eb libsystem_pthread.dylib`_pthread_body +
> 126
> frame #9: 0x7fff59a01249 libsystem_pthread.dylib`_pthread_start +
> 66
> frame #10: 0x7fff599fd40d libsystem_pthread.dylib`thread_start + 13
> (lldb) frame select 5
> frame #5: 0x00010bbd1792
> liblldb.10.0.99svn.dylib`lldb_private::process_gdb_remote::ProcessGDBRemote::AsyncThread(arg=0x7fcfaa470218)
> at ProcessGDBRemote.cpp:3877:28
>3874 "System Integrity
> Protection");
>3875 } else if (::strstr(continue_cstr, "vAttach") !=
> nullptr &&
>3876response.GetStatus().Fail()) {
> -> 3877 process->SetExitStatus(-1,
> response.GetStatus().AsCString());
>3878 } else {
>3879 process->SetExitStatus(-1, "lost connection");
>3880 }
> (lldb) p response.GetError()
> (uint8_t) $0 = '\x01'
> (lldb) p response.GetServerPacketType()
> (StringExtractorGDBRemote::Server
Re: [lldb-dev] Cannot use system debugserver for testing
*error 18:33:20.280017 +0200 authd Fatal: interaction not allowed (session has no ui access) (engine 3727)* This gave me a hint, so I used VPN to have a gui and got a gui window popped up to authenticate lldb. And then I could run lldb as a normal user. Hurray! BUT through ssh I still cannot run lldb that as a normal user. I've seen you have ``` @@@ Setup @@@ Unlocking keychain /Users/buildslave/Library/Keychains/lldb.keychain-db ... [1;32mOK [0m + echo @@ @@ ``` at your build bot at greenlab. So I tried "security -v unlock-keychain /Library/Keychains/System.keychain" but that did not work, I believe because scripts/macos-setup-codesign.sh did not ask for a password for the keychain (it asked for pw because of sudo). Is this the way to work if I don't have GUI (I must work via SSH, and this ought to be part of a CI) ? Should I recreate the keychain with a pw somehow? Thanks On Mon, Jul 22, 2019 at 6:47 PM Gábor Márton wrote: > Ok, I've done that, here are the logs which happen before the first > debugserver error: > > error 18:33:20.236506 +0200 taskgated cannot open file at line 42270 of > [95fbac39ba] > error 18:33:20.236526 +0200 taskgated os_unix.c:42270: (2) > open(/var/db/DetachedSignatures) - No such file or directory > default 18:33:20.236586 +0200 taskgated MacOS error: -67062 > error 18:33:20.246771 +0200 taskgated cannot open file at line 42270 of > [95fbac39ba] > error 18:33:20.246787 +0200 taskgated os_unix.c:42270: (2) > open(/var/db/DetachedSignatures) - No such file or directory > default 18:33:20.246841 +0200 taskgated MacOS error: -67062 > default 18:33:20.260319 +0200 debugserver debugserver will use os_log for > internal logging. > default 18:33:20.260491 +0200 debugserver debugserver-@(#)PROGRAM:LLDB > PROJECT:lldb-360.99.0 > for x86_64. > default 18:33:20.260615 +0200 debugserver Got a connection, waiting for > process information for launching or attaching. > default 18:33:20.264541 +0200 trustd cert[0]: AnchorTrusted > =(leaf)[force]> 0 > default 18:33:20.272256 +0200 trustd cert[2]: AnchorTrusted > =(leaf)[force]> 0 > default 18:33:20.276567 +0200 trustd cert[2]: AnchorTrusted > =(leaf)[force]> 0 > default 18:33:20.278680 +0200 authd UNIX error exception: 3 > error 18:33:20.279462 +0200 authd process: PID 27648 failed to create code > ref 13 > error 18:33:20.280017 +0200 authd Fatal: interaction not allowed (session > has no ui access) (engine 3727) > default 18:33:20.280031 +0200 authd Failed to authorize right > 'system.privilege.taskport' by client '/usr/libexec/taskgated' [254] for > authorization created by '/usr/libexec/taskgated' [27648] (3,1) (-60007) > (engine 3727) > error 18:33:20.280092 +0200 authd copy_rights: authorization failed > error 18:33:20.280442 +0200 debugserver error: > MachTask::TaskPortForProcessID task_for_pid failed: ::task_for_pid ( > target_tport = 0x0103, pid = 27646, &task ) => err = 0x0005 ((os/kern) > failure) > > > On Mon, Jul 22, 2019 at 6:10 PM Frédéric Riss wrote: > >> I think the system logs (in Console.app) would tell us more. Search for >> debugserver and you should find attach failures. Then remove the filter and >> look at what happens right before that. There should be a log from >> taskgated or authd that is a little more explicit about what’s failing. >> >> Fred >> >> On Jul 22, 2019, at 8:55 AM, Gábor Márton via lldb-dev < >> lldb-dev@lists.llvm.org> wrote: >> >> Some more info: I've stopped lldb (with the system lldb) when >> " Process::SetExitStatus (status=-1 (0x), description="Error 1")" >> is logged and gathered some more information. >> Is it possible that some packet types are not implemented? >> >> * frame #0: 0x7fff599452c6 libsystem_kernel.dylib`__pthread_kill + >> 10 >> frame #1: 0x7fff59a00bf1 libsystem_pthread.dylib`pthread_kill + >> 284 >> frame #2: 0x7fff598af6a6 libsystem_c.dylib`abort + 127 >> frame #3: 0x7fff5987820d libsystem_c.dylib`__assert_rtn + 324 >> frame #4: 0x00010b393539 >> liblldb.10.0.99svn.dylib`lldb_private::Process::SetExitStatus(this=0x7fcfaa470218, >> status=-1, cstr="Error 1") at Process.cpp:1149:3 >> frame #5: 0x00010bbd1792 >> liblldb.10.0.99svn.dylib`lldb_private::process_gdb_remote::ProcessGDBRemote::AsyncThread(arg=0x7fcfaa470218) >> at ProcessGDBRemote.cpp:3877:28 >> frame #6: 0x00010b08f87b >> liblldb.10.0.99svn.dylib`lldb_private::HostNativeThreadBase::ThreadCreateTrampoline(arg=0x7fcfab218740) >> at HostNativeThreadBase.cpp:69:10 >> frame #7: 0x000112
Re: [lldb-dev] Cannot use system debugserver for testing
lt 18:33:20.260491 +0200 debugserver debugserver-@(#)PROGRAM:LLDB
>> PROJECT:lldb-360.99.0
>> for x86_64.
>> default 18:33:20.260615 +0200 debugserver Got a connection, waiting for
>> process information for launching or attaching.
>> default 18:33:20.264541 +0200 trustd cert[0]: AnchorTrusted
>> =(leaf)[force]> 0
>> default 18:33:20.272256 +0200 trustd cert[2]: AnchorTrusted
>> =(leaf)[force]> 0
>> default 18:33:20.276567 +0200 trustd cert[2]: AnchorTrusted
>> =(leaf)[force]> 0
>> default 18:33:20.278680 +0200 authd UNIX error exception: 3
>> error 18:33:20.279462 +0200 authd process: PID 27648 failed to create
>> code ref 13
>> error 18:33:20.280017 +0200 authd Fatal: interaction not allowed (session
>> has no ui access) (engine 3727)
>> default 18:33:20.280031 +0200 authd Failed to authorize right
>> 'system.privilege.taskport' by client '/usr/libexec/taskgated' [254] for
>> authorization created by '/usr/libexec/taskgated' [27648] (3,1) (-60007)
>> (engine 3727)
>> error 18:33:20.280092 +0200 authd copy_rights: authorization failed
>> error 18:33:20.280442 +0200 debugserver error:
>> MachTask::TaskPortForProcessID task_for_pid failed: ::task_for_pid (
>> target_tport = 0x0103, pid = 27646, &task ) => err = 0x0005 ((os/kern)
>> failure)
>>
>>
>> On Mon, Jul 22, 2019 at 6:10 PM Frédéric Riss wrote:
>>
>>> I think the system logs (in Console.app) would tell us more. Search for
>>> debugserver and you should find attach failures. Then remove the filter and
>>> look at what happens right before that. There should be a log from
>>> taskgated or authd that is a little more explicit about what’s failing.
>>>
>>> Fred
>>>
>>> On Jul 22, 2019, at 8:55 AM, Gábor Márton via lldb-dev <
>>> lldb-dev@lists.llvm.org> wrote:
>>>
>>> Some more info: I've stopped lldb (with the system lldb) when
>>> " Process::SetExitStatus (status=-1 (0x), description="Error 1")"
>>> is logged and gathered some more information.
>>> Is it possible that some packet types are not implemented?
>>>
>>> * frame #0: 0x7fff599452c6 libsystem_kernel.dylib`__pthread_kill +
>>> 10
>>> frame #1: 0x7fff59a00bf1 libsystem_pthread.dylib`pthread_kill +
>>> 284
>>> frame #2: 0x7fff598af6a6 libsystem_c.dylib`abort + 127
>>> frame #3: 0x7fff5987820d libsystem_c.dylib`__assert_rtn + 324
>>> frame #4: 0x00010b393539
>>> liblldb.10.0.99svn.dylib`lldb_private::Process::SetExitStatus(this=0x7fcfaa470218,
>>> status=-1, cstr="Error 1") at Process.cpp:1149:3
>>> frame #5: 0x00010bbd1792
>>> liblldb.10.0.99svn.dylib`lldb_private::process_gdb_remote::ProcessGDBRemote::AsyncThread(arg=0x7fcfaa470218)
>>> at ProcessGDBRemote.cpp:3877:28
>>> frame #6: 0x00010b08f87b
>>> liblldb.10.0.99svn.dylib`lldb_private::HostNativeThreadBase::ThreadCreateTrampoline(arg=0x7fcfab218740)
>>> at HostNativeThreadBase.cpp:69:10
>>> frame #7: 0x000112be72fd
>>> liblldb.10.0.99svn.dylib`lldb_private::HostThreadMacOSX::ThreadCreateTrampoline(arg=0x7fcfab218740)
>>> at HostThreadMacOSX.mm:68:10
>>> frame #8: 0x7fff599fe2eb libsystem_pthread.dylib`_pthread_body +
>>> 126
>>> frame #9: 0x7fff59a01249 libsystem_pthread.dylib`_pthread_start
>>> + 66
>>> frame #10: 0x7fff599fd40d libsystem_pthread.dylib`thread_start +
>>> 13
>>> (lldb) frame select 5
>>> frame #5: 0x00010bbd1792
>>> liblldb.10.0.99svn.dylib`lldb_private::process_gdb_remote::ProcessGDBRemote::AsyncThread(arg=0x7fcfaa470218)
>>> at ProcessGDBRemote.cpp:3877:28
>>>3874 "System Integrity
>>> Protection");
>>>3875 } else if (::strstr(continue_cstr, "vAttach") !=
>>> nullptr &&
>>>3876response.GetStatus().Fail()) {
>>> -> 3877 process->SetExitStatus(-1,
>>> response.GetStatus().AsCString());
>>>3878 } else {
>>>3879 process->SetExitStatus(-1, "lost connection");
>>>3880 }
>>> (lldb) p response.GetError()
>>> (uint8_t) $0 = '\x01'
>>> (lldb) p response.GetServerPacketType()
>>> (StringExtractorGDBRemote::ServerPacketTyp
Re: [lldb-dev] ASTImporter Tutorial/TechTalk and RoundTable
Hi, At the upcoming LLVM Dev Conf, we will have a round table discussion for ASTImporter, right after the ASTImporter Tutorial. The time slot for the round table is Wednesday, Oct 23 2:55-4:00. I have gathered things about possible future work and improvements, bring your own topic to discuss! Thanks and see you at the conference, Gabor Big stuff - Error handling: rollback mechanism - Replace `StructuralEquivalency` with ODRHash - Step 0: StructuralEquivalency should not diagnose when called from the Importer, diags should be coming from the importer. - ODRViolation handling - Class(Var)TemplateSpecializationDecl: Problem! We can't have more than 1 spec (https://reviews.llvm.org/D66999) - VarTemplateSpecializationDecl: ODR violation is not even detected - Renaming strategy - Strategies for AccumulateChildErrors Clients of the ASTImporter should be able to choose an appropriate error handling strategy for their needs. For instance, they may not want to mark an entire namespace as erroneous merely because there is an ODR error with two typedefs. As another example, the client may allow EnumConstantDecls with same names but with different values in two distinct translation units. Smaller issues/tasks/FIXMEs (techn debt) - VisitFunctionDecl: - Member function templates are not handled similarly to simple fun specializations - Merge function definitions of members - Merge exception specifications - Handling of inheritable attributes (https://reviews.llvm.org/D68634) - Use PrevDecl in `GetImportedOrCreateDecl` ? - ObjC/ObjC++ support and stabilization - No test cases (not interesting for E///, we don't have objc/c++ code) - ClassTemplateSpecializationDecl: merge instantiated default arguments, exceptions specifications - Structural Eq: - Polluted cache of nonequivalent declarations - Some diagnostics are completely missing, this is misleading - Several minor issues/fixmes with VarTemplateDecl - Check visibility/linkage for ClassTemplateDecl, VarTemplateDecl - Fix import of equivalent but repeated FriendDecls - Handle redecl chain of TypeDefNameDecl - Add Decls to their context in a unified way, and only if the "From"DC contains it (`AddDeclToContexts`) - VisitVarDecl: - Check for ODR error if the two definitions have different initializers? - Diagnose ODR error if the two initializers are different - Remove obsolate FIXMEs and TODOs - Import default arguments of templates On Mon, Jul 22, 2019 at 6:28 PM Gábor Márton wrote: > Hi, > > I am planning to submit a talk about ASTImporter to the upcoming Dev > Meeting at October 22-23, 2019. > I'd like to talk about > - the API, > - how it is used in CTU analysis and in LLDB, > - internal subtleties and difficulties (Error handling, ExternalASTSource, > ...) > The goal would be to attract more developers to use and improve > ASTImporter related code, so perhaps this will be a tutorial. > > Independently from the talk, I'd like to have a round table discussion if > there is enough interest. > Some topics could cover future development ideas and existing problems we > have. > Please get back to me if you are interested and think about the topics you > have in mind, also don't forget to buy your ticket to the DevMeeting. > > Thanks, > Gabor > ___ lldb-dev mailing list lldb-dev@lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev
Re: [lldb-dev] ASTImporter Tutorial/TechTalk and RoundTable
Hi, Thank you guys for coming to the round table. I am trying to summarize what we were talking about. - Error handling: rollback mechanism We agreed that this is something that would be incredibly problematic to implement mostly because the AST is not designed to be trimmed. In normal cases, the AST just grows, but nodes and interconnections between them are never deleted. - Replace `StructuralEquivalency` with ODRHash - Step 0: StructuralEquivalency should not diagnose when called from the Importer, diags should be coming from the importer. We agreed that the class StructuralEquivalenceContext is used for multiple things in the ASTImporter. These are 1) ODR violation discovery 2) identifying the existing part of a redecl chain for a Decl. Actually, ODR checking is based on tokens if we compare definitions, so structural eq is not the best thing to use for that. We have concerns whether we could use the ODRHash, but we agreed that it would be worth to prototype it. - ODRViolation handling - Class(Var)TemplateSpecializationDecl: Problem! We can't have more than 1 spec (https://reviews.llvm.org/D66999) We agreed that this is fine to be conservative in case of specializations, in these cases we will always use the existing definition and will report back an error. (In case of simple classes and other non-specialization decls ,however, the liberal strategy can be used.) - Renaming strategy in case of ODR violation Raphael is worried about name mangling issues that could cause. - Strategies for AccumulateChildErrors Clients of the ASTImporter should be able to choose an appropriate error handling strategy for their needs. For instance, they may not want to mark an entire namespace as erroneous merely because there is an ODR error with two typedefs. As another example, the client may allow EnumConstantDecls with same names but with different values in two distinct translation units. We agreed that this is fine to use this approach, probably there is no need to make this behavior configurable. - VarTemplateSpecializationDecl: ODR violation is not even detected We didn't have time to discuss this. Smaller issues/tasks/FIXMEs (techn debt) - VisitFunctionDecl: - Member function templates are not handled similarly to simple fun specializations We agreed that this is a big technical debt - Merge function definitions of members We discussed that out-of class default definitions can cause this situation, and at some time we should finish this. - Merge exception specifications - Handling of inheritable attributes (https://reviews.llvm.org/D68634) - Use PrevDecl in `GetImportedOrCreateDecl` ? We agreed that handling of attributes cannot be done with one patch because there are many different inheritable attributes and each must be handled differently. - ObjC/ObjC++ support and stabilization - No test cases (not interesting for E///, we don't have objc/c++ code) We agreed that there is a technical debt here, but we did not show much interest in fixing that. - ClassTemplateSpecializationDecl: merge instantiated default arguments, exceptions specifications We understood the problem (that took like 20 minutes and AST dumping) and recognized that the way we merge initializers of fields is good, and we should continue doing that with default arg and exceptions spec instantiations. We did not have time to discuss these other issues: - Structural Eq: - Polluted cache of nonequivalent declarations - Some diagnostics are completely missing, this is misleading - Several minor issues/fixmes with VarTemplateDecl - Check visibility/linkage for ClassTemplateDecl, VarTemplateDecl - Fix import of equivalent but repeated FriendDecls - Handle redecl chain of TypeDefNameDecl - Add Decls to their context in a unified way, and only if the "From"DC contains it (`AddDeclToContexts`) - VisitVarDecl: - Check for ODR error if the two definitions have different initializers? - Diagnose ODR error if the two initializers are different - Remove obsolate FIXMEs and TODOs - Import default arguments of templates Cheers, Gabor On Thu, Oct 17, 2019 at 4:11 PM Gábor Márton wrote: > Hi, > > At the upcoming LLVM Dev Conf, we will have a round table discussion for > ASTImporter, right after the ASTImporter Tutorial. > The time slot for the round table is Wednesday, Oct 23 2:55-4:00. > I have gathered things about possible future work and improvements, bring > your own topic to discuss! > > Thanks and see you at the conference, > Gabor > > Big stuff > - Error handling: rollback mechanism > - Replace `StructuralEquivalency` with ODRHash > - Step 0: StructuralEquivalency should not diagnose when called from the > Importer, diags should be coming from the importer. > - ODRViolation handling > - Class(Var)TemplateSpecializationDecl: Problem! We can't have more than > 1 spec > (https://reviews.llvm.org/D66999) > - VarTemplateSpecializationDecl: ODR violation is not even detected > -
Re: [lldb-dev] Slow expression evaluation (ASTImporter is too eager?)
Hi Jaroslav,
Thanks for working on this. Still, there are things that are unclear for me.
I see that `LayoutRecordType` is called superfluously during the second
evaluation of `c2.x`, ok.
But, could you explain why LLDB is calling that multiple times? Maybe it
thinks a type is not completed while it is? As far as I know we keep track
which RecordDecl needs completeion in LLDB. At least, we do not store that
info in clang::ASTImporter. Minimal import gives a CXXRecordDecl whose
`DefinitionData` is set, even though the members are not imported the
definition data is there! So, there is no way for clang::ASTImporter to
know which RecordDecl had been imported in a minimal way.
I suspect there are multiple invocations of ASTImporter::ImportDefinition
with C2, C1, C0 within ClangASTSource (could you please check that?). And
`ImportDefinition` will blindly import the full definitions recursively
even if the minimal import is set (see ASTNodeImporter::IDK_Everything).
The patch would change this behavior which I'd like to avoid if possible. I
think first we should discover why there are multiple invocations of
ASTImporter::ImportDefinition from within LLDB.
Gabor
On Wed, Nov 6, 2019 at 11:21 PM Raphael “Teemperor” Isemann via lldb-dev <
lldb-dev@lists.llvm.org> wrote:
> Can you post that patch on Phabricator with an '[ASTImporter]’ as a name
> prefix? This way the ASTImporter folks will see your patch (The ASTImporter
> is more of a Clang thing, so they might not read lldb-dev). Also it makes
> it easier to see/test your patch :)
>
> (And +Gabor just in case)
>
> > On Nov 6, 2019, at 10:25 PM, Jaroslav Sevcik via lldb-dev <
> lldb-dev@lists.llvm.org> wrote:
> >
> > Hello,
> >
> > I noticed that the AST importer is very eager to import classes/structs
> that were already completed, even if they are not needed. This is best
> illustrated with an example.
> >
> > struct C0 { int x = 0; };
> > struct C1 { int x = 1; C0* c0 = 0; };
> > struct C2 { int x = 2; C1* c1 = 0; };
> >
> > int main() {
> > C0 c0;
> > C1 c1;
> > C2 c2;
> >
> > return 0; // break here
> > }
> >
> > When we evaluate “c2.x” in LLDB, AST importer completes and imports only
> class C2. This is working as intended. Similarly, evaluating “c1.x” imports
> just C1 and “c0.x” imports C0. However, if we evaluate “c2.x” after
> evaluating “c1.x” and “c0.x”, the importer suddenly imports both C1 and C0
> (in addition to C2). See a log from a lldb session at the end of this email
> for illustration.
> >
> > I believe the culprit here is the following code at the end of the
> ASTNodeImporter::VisitRecordDecl method:
> >
> > if (D->isCompleteDefinition())
> > if (Error Err = ImportDefinition(D, D2, IDK_Default))
> > return std::move(Err);
> >
> > This will import a definition of class from LLDB if LLDB already happens
> to have a complete definition from before. For large programs, this can
> lead to importing very large chunks of ASTs even if they are not needed. I
> have tried to remove the code above from clang and test performance on
> several expressions in an Unreal engine sample - preliminary results show
> this could cut down evaluation time by roughly 50%.
> >
> > My prototype patch is below; note that couple of lldb tests are failing
> with a wrong error message, this is a work in progress. What would the
> experts here think? Is this a plausible direction?
> >
> > Cheers, Jaro
> >
> > diff --git a/clang/lib/AST/ASTImporter.cpp
> b/clang/lib/AST/ASTImporter.cpp
> > index 54acca7dc62..ebbce5c66c7 100644
> > --- a/clang/lib/AST/ASTImporter.cpp
> > +++ b/clang/lib/AST/ASTImporter.cpp
> > @@ -2799,7 +2799,7 @@ ExpectedDecl
> ASTNodeImporter::VisitRecordDecl(RecordDecl *D) {
> >if (D->isAnonymousStructOrUnion())
> > D2->setAnonymousStructOrUnion(true);
> >
> > - if (D->isCompleteDefinition())
> > + if (!Importer.isMinimalImport() && D->isCompleteDefinition())
> > if (Error Err = ImportDefinition(D, D2, IDK_Default))
> >return std::move(Err);
> >
> > @@ -3438,6 +3438,9 @@ ExpectedDecl
> ASTNodeImporter::VisitFieldDecl(FieldDecl *D) {
> >if (ToInitializer)
> > ToField->setInClassInitializer(ToInitializer);
> >ToField->setImplicit(D->isImplicit());
> > + if (CXXRecordDecl *FieldType = D->getType()->getAsCXXRecordDecl())
> > + if (Error Err = ImportDefinitionIfNeeded(FieldType))
> > + return std::move(Err);
> >LexicalDC->addDeclInternal(ToField);
> >return ToField;
> > }
> > @@ -5307,7 +5310,7 @@ ExpectedDecl
> ASTNodeImporter::VisitClassTemplateSpecializationDecl(
> >
> >D2->setTemplateSpecializationKind(D->getTemplateSpecializationKind());
> >
> > - if (D->isCompleteDefinition())
> > + if (!Importer.isMinimalImport() && D->isCompleteDefinition())
> > if (Error Err = ImportDefinition(D, D2))
> >return std::move(Err);
> >
> >
> >
> > —— lldb session illustrating the unnecessary imports —-
> > This shows that evaluation of “c2.x” after evaluation “c1.x” a
Re: [lldb-dev] Slow expression evaluation (ASTImporter is too eager?)
Hi Jaroslav,
Thank you for digging into this. I think it may be possible to not eagerly
import a RecordDecl. Perhaps we could control that from
clang::ASTNodeImporter::VisitPointerType and instead of a full import we
could just do a minimal import for the PointeeType always. Similarly to
references.
But I am not sure about whether we would know during a subsequent import
(where the full type is really needed) that the type had been minimally
imported before. This requires some more investigation, I'll come back to
you what I find, but it will take some time as this week is quite busy for
me.
Gabor
On Tue, Nov 12, 2019 at 11:14 AM Jaroslav Sevcik wrote:
>
> Hi,
>
> thanks for the feedback, I did some more investigation and now I understand
> a bit better where the LayoutRecordType calls are coming from. They are
> initiated by
> Clang's code generation for types, which insists on generating full LLVM
> types
> for any complete RecordDecl (see
>
> https://github.com/llvm-mirror/clang/blob/65acf43270ea2894dffa0d0b292b92402f80c8cb/lib/CodeGen/CodeGenTypes.cpp#L752
> ,
> the bailout for incomplete record decls is at
>
> https://github.com/llvm-mirror/clang/blob/master/lib/CodeGen/CodeGenTypes.cpp#L729
> ).
> Ideally, clang would only emit forward declarations for any type that is
> only
> used as a pointer, but that is not the case at the moment. It is not quite
> clear
> what else we could do on the lldb side to prevent clang from emitting
> types for all
> those structs. At the end of this email, I am pasting a stack trace that
> illustrates
> the recursive emit for a pointer-chain of structs.
>
> There is also one additional problem with importing base classes. This one
> is perhaps
> more related to AST importer. Again this is best illustrated with an
> example.
>
> struct F1 { int f1 = 30; };
> struct F0 { int f0 = 31; };
>
> struct B0 {
> F0 b0f;
> };
>
> struct C0 : B0 {
> };
>
> struct B1 {
> F1 b1f;
> int f(C0* c0);
> };
>
> struct C1 : B1 {};
> struct C2 { int x = 4; C1* c1 = 0; };
>
> int main() {
> C0 c0;
> C1 c1;
> C2 c2;
>
> return 0; // break here
> }
>
> If we evaluate c2.x in lldb after evaluating c1 and c0, the AST importer
> unnecessarily imports all the classes in this file. The main reason for
> this is
> that the import for structs needs to set the DefaultedDestructorIsConstexpr
> bit, which is computed by looking at destructors of base classes. Looking
> at
> the destructors triggers completion and import of the base classes, which
> in
> turn might trigger more imports of base classes. In the example above,
> importing C1 will cause completion and import of B1, which in turn causes
> import of C0, and recursively completion and import of B0. Below is a
> stack trace showing import of field F0::f0 from evaluation of c2.x. The
> stack traces going through the
> VisitRecordDecl-->ImportDefinition-->setBases path appear to be very hot
> in the workloads we see (debugging large codebases). Below is a stack
> trace that illustrates visiting F0::f0 from the evaluation of c2.x. The
> question is what is the right place to break this long chain of imports.
> Ideally, we would not have to import fields of C1 or B1 when evaluating
> c2.x. Any ideas?
>
> Stack trace visiting F0::f0 from eval of c2.x, I have tried to highlight
> what is imported/completed where.
>
> clang::ASTNodeImporter::VisitFieldDecl;; Visit F0::f0
> clang::declvisitor::Base<...>::Visit
> clang::ASTImporter::ImportImpl
> lldb_private::ClangASTImporter::ASTImporterDelegate::ImportImpl
> clang::ASTImporter::Import
> lldb_private::ClangASTImporter::CopyDecl
> lldb_private::ClangASTSource::CopyDecl
> lldb_private::ClangASTSource::FindExternalLexicalDecls
> lldb_private::ClangASTSource::ClangASTSourceProxy::FindExternalLexicalDecls
> clang::ExternalASTSource::FindExternalLexicalDecls
> clang::DeclContext::LoadLexicalDeclsFromExternalStorage const
> clang::DeclContext::buildLookup ;; lookup destructor
> of B0
> clang::DeclContext::lookup const
> clang::CXXRecordDecl::getDestructor const
> clang::CXXRecordDecl::hasConstexprDestructor const
> clang::CXXRecordDecl::addedClassSubobject
> clang::CXXRecordDecl::addedMember
> clang::DeclContext::addHiddenDecl
> clang::DeclContext::addDeclInternal
> clang::ASTNodeImporter::VisitFieldDecl;; visit B0::b0f
> clang::declvisitor::Base<...>::Visit
> clang::ASTImporter::ImportImpl
> lldb_private::ClangASTImporter::ASTImporterDelegate::ImportImpl
> clang::ASTImporter::Import
> lldb_private::ClangASTImporter::CopyDecl
> lldb_private::ClangASTSource::CopyDecl
> lldb_private::ClangASTSource::FindExternalLexicalDecls
> lldb_private::ClangASTSource::ClangASTSourceProxy::FindExternalLexicalDecls
> clang::RecordDecl::LoadFieldsFromExternalStorage const
> clang::RecordDecl::field_begin const
> clang::RecordDecl::field_empty const ;; fields of B0
> clang::CXXRecordDecl::setBases
> clang::ASTNodeImporter::ImportDefi
Re: [lldb-dev] Slow expression evaluation (ASTImporter is too eager?)
Ok, sounds good.
> In the base class case we already import minimally (I think), but even
minimal import causes full import
This is deliberate. During construction of ASTImporter the minimal import
flag is set. And then when ASTImporter::ImportDefinition is called, it will
do a full import, even though minimal import is set. Otherwise we would not
be able to get a full type for LLDB.
> In the codegen case, it seems to be clang that walks through pointers.
Yes, you are right, my idea probably would not solve that problem. But the
base class related problem may be handled with that because the stack trace
you sent does include clang::ASTNodeImporter::VisitPointerType. I will
elaborate the idea once I found that feasible.
Gabor
On Thu, Nov 14, 2019 at 11:49 AM Jaroslav Sevcik wrote:
> Hi Gabor,
>
> I was thinking about a different idea: perhaps we could try to track on
> lldb side which recorddecls were requested to be completed during a
> particular parsing session and only import full definitions if they were
> requested in this session. If completion was not requested in this session,
> we would only import forward declaration (even if we already happen to have
> full definition). Just like you, I have other things to attend to, so it
> might take some time to prototype that.
>
> I will think about your idea, it is not quite clear to me yet how it
> solves either of the problems. In the codegen case, it seems to be clang
> that walks through pointers. In the base class case we already import
> minimally (I think), but even minimal import causes full import of base
> classes (perhaps the const destructor check could only import the
> destructor, though).
>
> Thanks, Jaro
>
> On Thu, Nov 14, 2019 at 11:14 AM Gábor Márton
> wrote:
>
>> Hi Jaroslav,
>>
>> Thank you for digging into this. I think it may be possible to not
>> eagerly import a RecordDecl. Perhaps we could control that from
>> clang::ASTNodeImporter::VisitPointerType and instead of a full import we
>> could just do a minimal import for the PointeeType always. Similarly to
>> references.
>> But I am not sure about whether we would know during a subsequent import
>> (where the full type is really needed) that the type had been minimally
>> imported before. This requires some more investigation, I'll come back to
>> you what I find, but it will take some time as this week is quite busy for
>> me.
>>
>> Gabor
>>
>> On Tue, Nov 12, 2019 at 11:14 AM Jaroslav Sevcik
>> wrote:
>>
>>>
>>> Hi,
>>>
>>> thanks for the feedback, I did some more investigation and now I
>>> understand
>>> a bit better where the LayoutRecordType calls are coming from. They are
>>> initiated by
>>> Clang's code generation for types, which insists on generating full LLVM
>>> types
>>> for any complete RecordDecl (see
>>>
>>> https://github.com/llvm-mirror/clang/blob/65acf43270ea2894dffa0d0b292b92402f80c8cb/lib/CodeGen/CodeGenTypes.cpp#L752
>>> ,
>>> the bailout for incomplete record decls is at
>>>
>>> https://github.com/llvm-mirror/clang/blob/master/lib/CodeGen/CodeGenTypes.cpp#L729
>>> ).
>>> Ideally, clang would only emit forward declarations for any type that is
>>> only
>>> used as a pointer, but that is not the case at the moment. It is not
>>> quite clear
>>> what else we could do on the lldb side to prevent clang from emitting
>>> types for all
>>> those structs. At the end of this email, I am pasting a stack trace that
>>> illustrates
>>> the recursive emit for a pointer-chain of structs.
>>>
>>> There is also one additional problem with importing base classes. This
>>> one is perhaps
>>> more related to AST importer. Again this is best illustrated with an
>>> example.
>>>
>>> struct F1 { int f1 = 30; };
>>> struct F0 { int f0 = 31; };
>>>
>>> struct B0 {
>>> F0 b0f;
>>> };
>>>
>>> struct C0 : B0 {
>>> };
>>>
>>> struct B1 {
>>> F1 b1f;
>>> int f(C0* c0);
>>> };
>>>
>>> struct C1 : B1 {};
>>> struct C2 { int x = 4; C1* c1 = 0; };
>>>
>>> int main() {
>>> C0 c0;
>>> C1 c1;
>>> C2 c2;
>>>
>>> return 0; // break here
>>> }
>>>
>>> If we evaluate c2.x in lldb after evaluating c1 and c0, the AST importer
>>> unnecessarily imports all the classes in this file. The main reason for
>>> this is
>>> that the import for structs needs to set the
>>> DefaultedDestructorIsConstexpr
>>> bit, which is computed by looking at destructors of base classes.
>>> Looking at
>>> the destructors triggers completion and import of the base classes,
>>> which in
>>> turn might trigger more imports of base classes. In the example above,
>>> importing C1 will cause completion and import of B1, which in turn causes
>>> import of C0, and recursively completion and import of B0. Below is a
>>> stack trace showing import of field F0::f0 from evaluation of c2.x. The
>>> stack traces going through the
>>> VisitRecordDecl-->ImportDefinition-->setBases path appear to be very hot
>>> in the workloads we see (debugging large codebases). Below is a stack
>>> trace that illustrates visi
