Right, after learning way more than I ever wanted to know about xcrun,
I think I see the issue. When running with empty SDKROOT variable,
xcrun sets SDKROOT to "/" when running clang:
$ SDKROOT= xcrun --no-cache --log --verbose clang
/Users/lldb_build/lldbSlave/buildDir/llvm/tools/lldb/packages/Python/lldbsuite/test/crashinfo.c
-o
/Users/lldb_build/lldbSlave/buildDir/llvm/tools/lldb/packages/Python/lldbsuite/test/crashinfo.so
-framework Python -Xlinker -dylib -v
xcrun: note: PATH = '/usr/bin:/bin'
xcrun: note: SDKROOT = '/'
^^^^^ WRONG. there are no
python headers there
.....
env SDKROOT=/
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/clang
/Users/lldb_build/lldbSlave/buildDir/llvm/tools/lldb/packages/Python/lldbsuite/test/crashinfo.c
-o
/Users/lldb_build/lldbSlave/buildDir/llvm/tools/lldb/packages/Python/lldbsuite/test/crashinfo.so
-framework Python -Xlinker -dylib -v
...
#include "..." search starts here:
#include <...> search starts here:
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib/clang/9.0.0/include
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/include
/usr/include
/System/Library/Frameworks (framework directory)
/Library/Frameworks (framework directory)
^^^^ clang gets the include path wrong
End of search list.
/Users/lldb_build/lldbSlave/buildDir/llvm/tools/lldb/packages/Python/lldbsuite/test/crashinfo.c:15:10:
fatal error: 'Python/Python.h' file not found
#include <Python/Python.h>
^~~~~~~~~~~~~~~~~
1 error generated.
^^^^^ and errors out
On the other hand, if I invoke xcrun with SDKROOT=macosx, everything
works just fine:
$ SDKROOT=macosx xcrun --no-cache --log --verbose clang
/Users/lldb_build/lldbSlave/buildDir/llvm/tools/lldb/packages/Python/lldbsuite/test/crashinfo.c
-o
/Users/lldb_build/lldbSlave/buildDir/llvm/tools/lldb/packages/Python/lldbsuite/test/crashinfo.so
-framework Python -Xlinker -dylib -v
xcrun: note: looking up SDK with
'/Applications/Xcode.app/Contents/Developer/usr/bin/xcodebuild -sdk
macosx -version Path'
xcrun: note: PATH = '/usr/bin:/bin'
xcrun: note: SDKROOT = 'macosx'
xcrun: note: TOOLCHAINS = ''
xcrun: note: DEVELOPER_DIR = '/Applications/Xcode.app/Contents/Developer'
xcrun: note: XCODE_DEVELOPER_USR_PATH = ''
xcrun: note: xcrun_db =
'/var/folders/bt/tws6gynx0ws1cc4ss53_pvqm0000gq/T/xcrun_db'
xcrun: note: lookup resolved to:
'/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.13.sdk'
xcrun: note: PATH = '/usr/bin:/bin'
xcrun: note: SDKROOT =
'/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.13.sdk'
^^^^^^ correct. a valid SDK is located there
...
env
SDKROOT=/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.13.sdk
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/clang
/Users/lldb_build/lldbSlave/buildDir/llvm/tools/lldb/packages/Python/lldbsuite/test/crashinfo.c
-o
/Users/lldb_build/lldbSlave/buildDir/llvm/tools/lldb/packages/Python/lldbsuite/test/crashinfo.so
-framework Python -Xlinker -dylib -v
...
#include "..." search starts here:
#include <...> search starts here:
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib/clang/9.0.0/include
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/include
/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.13.sdk/usr/include
/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.13.sdk/System/Library/Frameworks
(framework directory)
^^^^^ include path is good and compilation succeeds
I've checked this on 10.12 and the xcrun behavior is same there (with
the difference that 10.12 does contain headers in
/System/Library/Frameworks.
It seems we could fix this by changing the invocation in dotest to
explicitly request SDKROOT=macosx. As far, as I can tell, this would
only break if someone tried to run dotest with iOS as host, but I
don't think that's ever going to be supported(?)
What do you think?
On 15 November 2017 at 20:25, Jim Ingham <jing...@apple.com> wrote:
> The build should be finding the version in the SDK within Xcode. I do have
> the CommandLineTools directory on my system, but it doesn’t have an SDK
> directory in it. I wonder if that is causing the problem?
>
> One thing to check, do:
>
> $ xcrun -find clang
>
> Does that find clang in the DeveloperTools directory? If so try:
>
> sudo xcode-select —switch /Applications/Xcode.app
>
> Then everything should point to the toolchain & SDK’s in Xcode. If that
> doesn’t help you might try moving the DeveloperTools aside and see if things
> work then. Xcode should not need that to work.
>
> Jim
>
>
>> On Nov 15, 2017, at 10:33 AM, Pavel Labath <lab...@google.com> wrote:
>>
>> Thanks for the reply, Jim.
>>
>> As far as I can tell, we already have Xcode on that machine (I only
>> have shell access there).
>> $ xcodebuild -version
>> Xcode 9.1
>> Build version 9B55
>> $ xcode-select -version
>> xcode-select version 2349.
>>
>>
>> BTW, this is the list of Python.h files on that machine:
>> /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/System/Library/Frameworks/Python.framework/Versions/2.7/include/python2.7/Python.h
>> /System/Library/Frameworks/Python.framework/Versions/2.7/include/python2.7/Python.h
>> /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/System/Library/Frameworks/Python.framework/Versions/2.7/include/python2.7/Python.h
>>
>> I guess you were referring to the third one. That one seems to have
>> the "Headers" symlink and everything, but for some reason clang is not
>> picking it up. Do I need to run some fancy xcode-select or set some
>> environment variables?
>>
>> I guess the reason I am able to compile lldb just fine is that cmake
>> is smart enough to find the right python framework. I'll try comparing
>> the cmake command lines with the one we run from dotest.
>>
>> Additional issue which may complicate things is that this was not a
>> clean install, but an upgrade.
>>
>>
>>
>> On 15 November 2017 at 18:16, Jim Ingham <jing...@apple.com> wrote:
>>> Xcode and general development on macOS has moved from using headers in the
>>> base OS (/System/Library/Frameworks…) for building programs to using SDK’s
>>> to contain all the header files. The installed tools will know how to find
>>> the correct SDK.
>>>
>>> On a clean install of 10.13 there are no headers anywhere in the
>>> /System/Library/Frameworks frameworks; I’m a little surprised you can build
>>> anything w/o an Xcode install. There’s a “command line tools” package in
>>> the Developer tools that puts some stuff back in /usr but that has had some
>>> issues, mostly it overwrites the /usr/bin/clang etc. tools that Xcode uses
>>> - which are just shims that find the correct version of the tools in the
>>> xcode-selected Xcode - with fixed versions of the tools which can lead to
>>> subtle errors if you ever try to use Xcode, so I don’t suggest that.
>>>
>>> I think you need to put an Xcode install on your bot.
>>>
>>> Jim
>>>
>>>> On Nov 15, 2017, at 8:20 AM, Pavel Labath via lldb-dev
>>>> <lldb-dev@lists.llvm.org> wrote:
>>>>
>>>> On a somewhat tangential note:
>>>>
>>>> Is anyone actually using this crashinfo hook? It looks like this could
>>>> be useful in the old dotest days, when we were running all of the
>>>> tests in a single python process, but now with the parallel test
>>>> runner spawning a new process for each test file and with piping test
>>>> results through a socket, it seems much less useful. If noone is using
>>>> that functionality, maybe we could "fix" the problem by deleting it.
>>>>
>>>>
>>>> On 15 November 2017 at 15:17, Pavel Labath <lab...@google.com> wrote:
>>>>> Hello lldb-dev,
>>>>>
>>>>> We've just updated our mac buildbot to 10.13.1 (from 10.10.x), and
>>>>> we're having trouble with the lldb test suite. All of the tests are
>>>>> failing with the following error:
>>>>>
>>>>> /Users/lldb_build/lldbSlave/buildDir/llvm/tools/lldb/packages/Python/lldbsuite/test/crashinfo.c:15:10:
>>>>> fatal error: 'Python/Python.h' file not found
>>>>> #include <Python/Python.h>
>>>>> ^~~~~~~~~~~~~~~~~
>>>>> 1 error generated.
>>>>> Traceback (most recent call last):
>>>>> File
>>>>> "/Users/lldb_build/lldbSlave/buildDir/scripts/../llvm/tools/lldb/test/dotest.py",
>>>>> line 7, in <module>
>>>>> lldbsuite.test.run_suite()
>>>>> File
>>>>> "/Users/lldb_build/lldbSlave/buildDir/llvm/tools/lldb/packages/Python/lldbsuite/test/dotest.py",
>>>>> line 1120, in run_suite
>>>>> configuration.setupCrashInfoHook()
>>>>> File
>>>>> "/Users/lldb_build/lldbSlave/buildDir/llvm/tools/lldb/packages/Python/lldbsuite/test/configuration.py",
>>>>> line 51, in setupCrashInfoHook
>>>>> raise Exception('command failed: "{}"'.format(cmd))
>>>>> Exception: command failed: "SDKROOT= xcrun clang
>>>>> /Users/lldb_build/lldbSlave/buildDir/llvm/tools/lldb/packages/Python/lldbsuite/test/crashinfo.c
>>>>> -o
>>>>> /Users/lldb_build/lldbSlave/buildDir/llvm/tools/lldb/packages/Python/lldbsuite/test/crashinfo.so
>>>>> -framework Python -Xlinker -dylib"
>>>>>
>>>>>
>>>>> It seems that this is happening because the buildbot is missing the
>>>>> /System/Library/Frameworks/Python.framework/Headers symlink (this link
>>>>> is present on my mac machine, which is still on 10.12). The rest of
>>>>> the framework seems to be there (e.g. the file
>>>>> /System/Library/Frameworks/Python.framework/Versions/Current/include/python2.7/Python.h
>>>>> is present), just this symlink is missing. I cannot even create it
>>>>> manually as System Integrity Protection will not let me do that.
>>>>>
>>>>> Do you have any idea what went wrong?
>>>>>
>>>>> thanks,
>>>>> pl
>>>>>
>>>>> PS: If it helps anything, the version reported by clang is:
>>>>> Apple LLVM version 9.0.0 (clang-900.0.38)
>>>>> Target: x86_64-apple-darwin17.2.0
>>>>> Thread model: posix
>>>>> InstalledDir:
>>>>> /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin
>>>> _______________________________________________
>>>> lldb-dev mailing list
>>>> lldb-dev@lists.llvm.org
>>>> http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev
>>>
>
_______________________________________________
lldb-dev mailing list
lldb-dev@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev
