We can’t separate building for ‘Build’ and building for an arbitrary ‘Host’.
For those who don’t know, this is how cross-compiling works with Swift (this
isn’t off-topic, I’m going to bring it back firmly to this proposal):
Currently, when you install the compiler, the standard library is in
(prefix)/lib/swift/(Target)/(arch). On OSX, we cross-compile the standard
library for the iOS targets, so you’ll see subfolders for ‘macosx’, ‘iphoneos’,
etc in there. All you need to cross-compile in Swift is the standard library;
swiftmodules don’t need ‘ar’, so libraries don’t even need any binutils to
cross-compile, which is refreshing. A cross-linker *is* unfortunately still
required for executables (come on, LLD!).
Grab the standard library from a linux machine and try:
echo 'print("hello, world")' | ./swiftc —target=x86-unknown-linux-gnu
-resource-dir {PATH_TO_LINUX_STDLIB} -emit-module -
So when we’re cross-compiling the swift portions of the standard-library, we’re
using the swift compiler we just made for Build, telling it to use Host’s
lib/swift directory, and there it will find the correct subdirectory for
Target. My native compiler for my OSX Build machine knows not to look for the
Linux host’s own Target in swift-macosx-x86_64/lib/swift, but in
swift-linux-armv7/lib/swift.
—
Okay, so that’s out of the way.
There is an actual problem with how we build/configure the target libraries
(Foundation, XCTest and libdispatch). We currently configure them like every
other product (for each host), instead of for each target. A consequence of
this is, for example, that we won’t try to build Foundation for cross-compile
targets such as Android. What we should do is this:
For each host:
-> Build cmark/llvm/swift
-> For each target of this host:
-> -> Build Foundation/XCTest/libdispatch
-> Build swiftpm
When you look at it like this, from a tools // target-libraries perspective
(swiftpm is an exception; it’s a host tool, but it depends on Foundation), I
came up with a different idea (possibly, I’m not sure), that might also help
the dependency issue, although I don’t know the specifics of it: I would
suggest that after building swift, we install it, then build the target
libraries and install them as we do so.
That would give us a stable standard-library (lib/swift) location which we
could treat like an installed system. So when Foundation builds (for each
Target now), it sets -resource-dir to the installed Host copy of swift, and the
Build swift compiler can find Target’s standard library (yeah, I know). It
installs itself in there. Then XCTest comes along, it sets the same
-resource-dir, so now it can find Foundation just with ‘import Foundation’ and
nothing else. In other words: every product after the swift compiler can assume
its dependencies are like they’ve been installed on the system.
I’m not sure if that’s what you’re referring to with a ‘toolchain-based’ build
process or not. The reason I think it might be is because I don’t think this
would require big changes. We’d just have to install products after swift, then
install the target libraries as we go. Foundation could even keep on generating
build.ninja in-tree, sadly.
There isn’t much difference between supporting one cross-host or supporting n
cross-hosts. I would prefer ’n’, if we could get a nice argument syntax for it,
because it’s more convenient to integrate with other tools, but it’s not a big
deal. Actually, I have an idea for a better build-script argument syntax which
would scale to NxM cross-compiled hosts and targets elegantly, but that’s a
topic for another day.
Karl
> On 3 Jun 2016, at 22:31, Daniel Dunbar via swift-dev
> wrote:
>
>
>> On Jun 3, 2016, at 1:14 PM, Saleem Abdulrasool > > wrote:
>>
>> On Wed, Jun 1, 2016 at 2:18 PM, Daniel Dunbar via swift-dev
>> mailto:[email protected]>> wrote:
>> Hi all,
>>
>> The current build process for the overall Swift project (i.e., the compiler
>> + associated projects like Foundation, XCTest, and SwiftPM) relies on each
>> project having dependencies on the built artifacts of previously built
>> projects. Those dependencies are currently communicated to each project in
>> the chain through an ad hoc set of arguments to the individual project's
>> build process. This has been painful to maintain, and makes it hard to
>> reason about the environment that each of the associated projects are
>> building within.
>>
>> Instead, I would like to move towards what I have been calling a
>> "toolchain-based" build process.
>>
>> In this model:
>>
>> 1. The entire build process will be organized as a sequential, incremental
>> construction of a complete toolchain.
>> - Each individual project will build and copy its products into a
>> staging area.
>> 2. The build script will always create a composed toolchain.
>> - It will start with an empty toolchain, and merge in the content
>> from each pr
