Re: [swift-corelibs-dev] [swift-evolution] Language or XCTest improvements to allow testing assert/precondition failures

2015-12-13 Thread Daniel Dunbar via swift-corelibs-dev 
> On Dec 13, 2015, at 2:03 AM, David Hart via swift-evolution 
>  wrote:
> 
> Currently, it is impossible for XCTest to unit test assert and precondition 
> failures because they kill the process. And those are important to unit test: 
> how would you test NSArray's objectAtIndex bound conditions? Currently, the 
> standard library tests those with StdlibUnittest, a small piece of code which 
> runs those tests in a forked process.
> 
> I want to start a proposal but I'm not sure if it should come as a XCTest 
> improvement which spawns a process or if the language should implement assert 
> and precondition as a special kind of throw?

One way to start a proposal is to try and work through what the implications of 
each approach are, decide which is best, and then write that up in a proposal 
explaining why you chose the approach you did.

If you are interested in working on this, I would definitely also survey how 
other frameworks do this. Googletest, for example, supports this via a "death 
tests" mechanism, which would be good to understand and compare to:
  
https://github.com/google/googletest/blob/master/googletest/docs/AdvancedGuide.md#death-tests

I suspect that spawning a process is the best approach, because it can cover 
more cases than the language built in features, and doesn't require language 
support, but I would be interested in seeing an in depth study of what this 
would mean for the framework and the APIs.

 - Daniel

> 
> David
> ___
> swift-evolution mailing list
> [email protected]
> https://lists.swift.org/mailman/listinfo/swift-evolution

___
swift-corelibs-dev mailing list
[email protected]
https://lists.swift.org/mailman/listinfo/swift-corelibs-dev

Re: [swift-corelibs-dev] [xctest] Terminology: What to call "swift-corelibs-xctest" and "Apple XCTest"

2015-12-24 Thread Daniel Dunbar via swift-corelibs-dev 
I believe we have been saying "Corelibs XCTest", but agree it would be nice to 
have a standard labelling.

 - Daniel

> On Dec 22, 2015, at 11:57 PM, Brian Gesiak via swift-corelibs-dev 
>  wrote:
> 
> Hello all,
> 
> I've been referring to the project hosted at
> https://github.com/apple/swift-corelibs-xctest as
> "swift-corelibs-xctest", and the project shipped alongside Xcode 7.2
> as "Apple XCTest".
> 
> What are the canonical names for these projects? I'd like to agree on
> common terminology to avoid confusion in the future.
> 
> Over the past few weeks I've seen several names for
> https://github.com/apple/swift-corelibs-xctest:
> 
> - Open source XCTest
> - Swift XCTest
> - Linux XCTest
> - swift-corelibs-xctest
> 
> And several names for the XCTest framework shipped alongside Xcode:
> 
> - Darwin XCTest
> - Apple XCTest
> 
> How do you refer to these projects? What do you think is the simplest
> way to refer to them, in a way that clearly differentiates them?
> 
> Personally, I prefer calling them "swift-corelibs-xctest" and "Apple
> XCTest", although I do think that "swift-corelibs-xctest" is a
> mouthful.
> 
> - Brian Gesiak
> ___
> swift-corelibs-dev mailing list
> [email protected]
> https://lists.swift.org/mailman/listinfo/swift-corelibs-dev

___
swift-corelibs-dev mailing list
[email protected]
https://lists.swift.org/mailman/listinfo/swift-corelibs-dev

Re: [swift-corelibs-dev] corelibs-xctest + swiftpm

2016-01-04 Thread Daniel Dunbar via swift-corelibs-dev 
I think this makes sense (and commented in your PR). For now we will still need 
the manual build process because we need XCTest to build first in CI, so that 
we can use it in building/testing the package manager itself, but it certainly 
makes sense to shoot for it to just be yet another package.

 - Daniel

> On Dec 31, 2015, at 11:42 AM, Keith Smiley via swift-corelibs-dev 
>  wrote:
> 
> Hey all,
> 
> I was working on integrating a project with corelibs-xctest and I noticed that
> it doesn't currently support building with swiftpm and instead provides
> `.xcodeproj` which generates a `.framework`. Is there any future plan to also
> include a `Package.swift` so that user's could build it either way? Because of
> how simple corelibs-xctest is this is just a matter of including a barebones
> `Package.swift` and moving the source files.
> 
> 
> Thanks!
> 
> --
> Keith Smiley
> 
> ___
> swift-corelibs-dev mailing list
> [email protected]
> https://lists.swift.org/mailman/listinfo/swift-corelibs-dev

___
swift-corelibs-dev mailing list
[email protected]
https://lists.swift.org/mailman/listinfo/swift-corelibs-dev

Re: [swift-corelibs-dev] [SE-0046] Implemented consistent function labels

2016-04-07 Thread Daniel Dunbar via swift-corelibs-dev 

> On Apr 7, 2016, at 10:15 AM, Daniel Eggert via swift-corelibs-dev 
>  wrote:
> 
> With this being merged
> 
> https://github.com/apple/swift-corelibs-foundation/pull/305
> 
> where do I find a compiler that works? Everything stopped working for me, 
> because utils/update-checkout currently checks out something that can't build 
> a toolchain.

Yes, this unfortunately just a reality of the language changing on trunk and 
the current development model -- if you want to track trunk you at times will 
need to build from trunk and not a snapshot.

> Does https://ci.swift.org/ have daily Toolchains?

No, not currently.

 - Daniel

> /Daniel
> 
> ___
> swift-corelibs-dev mailing list
> [email protected]
> https://lists.swift.org/mailman/listinfo/swift-corelibs-dev

___
swift-corelibs-dev mailing list
[email protected]
https://lists.swift.org/mailman/listinfo/swift-corelibs-dev

Re: [swift-corelibs-dev] NSURLSession & libcurl

2016-04-20 Thread Daniel Dunbar via swift-corelibs-dev 

> On Apr 20, 2016, at 9:43 AM, Daniel Eggert via swift-corelibs-dev 
>  wrote:
> 
> I think that would make a giant mess. I'm curious how far off we are for 
> having lob libdispatch on Linux. I think it would be worth waiting a bit 
> longer I n stead of making the code illegible. Just my 2¢.

+1, having another incentive to drive libdispatch to be part of the snapshots 
doesn't seem like a bad thing.

 - Daniel

> 
> /Daniel
> 
> 
> On Apr 20, 2016, at 04:23, Pushkar N Kulkarni  > wrote:
> 
>> Hi Tony, Daniel, 
>> 
>> Thanks for your responses. 
>> 
>> Would it be acceptable if, for now, we check for the availability of 
>> Dispatch and conditionally compile calls to it into the current 
>> implementation of NSURLSession* (something like the  _HAS_DISPATCH__ macro 
>> used in CoreFoundation)? This would make NSURLSession and related classes 
>> functional with a custom toolchain that has Dispatch, on Linux. It'd also 
>> make them functional on OS X.
>> 
>> I realise this may need some code restructuring.
>> 
>> Pushkar N Kulkarni,
>> IBM Runtimes
>> 
>> Simplicity is prerequisite for reliability - Edsger W. Dijkstra
>> 
>> 
>> 
>> -Daniel Eggert mailto:[email protected]>> wrote: 
>> -
>> To: Tony Parker mailto:[email protected]>>
>> From: Daniel Eggert mailto:[email protected]>>
>> Date: 04/19/2016 08:21PM
>> Cc: Pushkar N Kulkarni/India/IBM@IBMIN, Swift corelibs dev 
>> mailto:[email protected]>>
>> Subject: Re: [swift-corelibs-dev] NSURLSession & libcurl
>> 
>> I don't think it compiles, but I haven't tried, yet. I've been trying to 
>> keep it up-to-date with master. 
>> 
>> /Daniel
>> 
>> On Apr 18, 2016, at 13:15, Tony Parker > > wrote:
>> 
>>> I think that’s a fair approach - but does this even compile on Linux 
>>> without dispatch in place? I get “no such module ‘Dispatch’” errors when 
>>> compiling.
>>> 
>>> - Tony
>>> 
 On Apr 18, 2016, at 11:24 AM, Pushkar N Kulkarni >>> > wrote:
 
 Thanks for your great work on NSURLSession and friends, Daniel!
 
 I was wondering if we are only waiting for dispatch to be available on 
 Linux here. In that case, could we have the failing tests (if any) 
 excluded (only on Linux perhaps) and have this merged please? Some of us 
 at IBM would like to work with the current implementation and contribute 
 on top of it.
 
 Thanks!
 
 Pushkar N Kulkarni,
 IBM Runtimes
 
 Simplicity is prerequisite for reliability - Edsger W. Dijkstra
 
 
 
 [email protected] 
  wrote: -
 To: Swift corelibs dev >>> >
 From: Daniel Eggert via swift-corelibs-dev 
 Sent by: [email protected] 
 
 Date: 04/05/2016 12:14AM
 Subject: Re: [swift-corelibs-dev] NSURLSession & libcurl
 
 I won't be able to put too many more hours into this after next week.
 
 https://github.com/apple/swift-corelibs-foundation/pull/299 
 
 
 Feedback is very welcome.
 
 The tests show what's working, and there are "TODO:" markers throughout 
 the code where applicable.
 
 /Daniel
 
 ___
 swift-corelibs-dev mailing list
 [email protected] 
 https://lists.swift.org/mailman/listinfo/swift-corelibs-dev 
 
 
>>> 
>> 
> ___
> swift-corelibs-dev mailing list
> [email protected]
> https://lists.swift.org/mailman/listinfo/swift-corelibs-dev

___
swift-corelibs-dev mailing list
[email protected]
https://lists.swift.org/mailman/listinfo/swift-corelibs-dev

Re: [swift-corelibs-dev] swift-corelibs-xctest JIRA dashboard

2016-05-23 Thread Daniel Dunbar via swift-corelibs-dev 
Following up on Brian's JIRA dashboard for XCTest, I copied his model and 
created a dashboard for SwiftPM:
  https://bugs.swift.org/secure/Dashboard.jspa?selectPageId=10409
and defined several useful SwiftPM public filters:

 - Package Manager All Tasks: https://bugs.swift.org/issues/?filter=10477
 - Package Manager Open Tasks: https://bugs.swift.org/issues/?filter=10474
 - Package Manager Open 3.0 Tasks: https://bugs.swift.org/issues/?filter=10472
 - Package Manager Open Starter Tasks: 
https://bugs.swift.org/issues/?filter=10473

Thanks to Brian for the inspiration!

 - Daniel

> On May 22, 2016, at 12:47 PM, Brian Gesiak via swift-corelibs-dev 
>  wrote:
> 
> Hello all!
> 
> If you're like me, you might be curious how Core Libraries like 
> swift-corelibs-xctest are doing with regards to the looming Swift 3.0 
> release. Well, wonder no more -- this handy JIRA dashboard has the 
> information you need: 
> https://bugs.swift.org/secure/Dashboard.jspa?selectPageId=10408 
> 
> 
> The dashboard not only lists tasks that need to be resolved by Swift 3.0, but 
> also open starter tasks for new contributors.
> 
> Let me know if you find it useful! I personally think it'd be neat to have 
> one of these for all the swift-corelibs-* projects, what do you all think? :)
> 
> PS: For those interested, the dashboard is implemented using custom task 
> filters:
> 
> - corelibs-xctest Open Tasks: https://bugs.swift.org/issues/?filter=10469 
> 
> - corelibs-xctest Open 3.0 Tasks: https://bugs.swift.org/issues/?filter=10471 
> 
> - corelibs-xctest Open Starter Tasks: 
> https://bugs.swift.org/issues/?filter=10470 
> 
> 
> To track tasks related to Swift 3.0, I created a new "swift-3.0" label in 
> JIRA. I hope no one minds. (+cc Jordan Rose, I've seen him managing labels on 
> JIRA before.)
> 
> - Brian Gesiak
> 
> ___
> swift-corelibs-dev mailing list
> [email protected]
> https://lists.swift.org/mailman/listinfo/swift-corelibs-dev

___
swift-corelibs-dev mailing list
[email protected]
https://lists.swift.org/mailman/listinfo/swift-corelibs-dev

Re: [swift-corelibs-dev] Objective-C Foundation vs CoreLibs Foundation

2016-05-23 Thread Daniel Dunbar via swift-corelibs-dev 

> On May 23, 2016, at 3:34 PM, Philippe Hausler via swift-corelibs-dev 
>  wrote:
> 
> There are a few considerations for the package manager: we may have circular 
> build requirements, swift-corelibs-foundation does some squirrelly things 
> with linking and compilation like linker scripts and tacked on assembly data 
> segments. I am not certain those edge use cases are supported yet.

They are not, and I would view them as a stretch for 3.0 at this point.

The combination of this and the circular build problem makes me think that we 
should probably expect to maintain a Foundation-specific build solution for the 
time being.

 - Daniel

> 
> The Python config file is way too complex as it is and was only really 
> designed as a stopgap measure. If we can simplify I think it would definitely 
> improve the state of things: it is worth investigating.
> 
> Sent from my iPhone
> 
>> On May 23, 2016, at 3:03 PM, David Hart via swift-corelibs-dev 
>>  wrote:
>> 
>> Would you agree that the first step should be to have the project as a 
>> SwiftPM package so that we have a more consistent way to run tests on all 
>> platforms? Do you know if SwiftPM is far enough to support 
>> swift-corelibs-foundation? I might have a go at it once I finish 
>> implementing NSProgress (about half way through I think).
>> 
>>> On 23 May 2016, at 17:59, Tony Parker via swift-corelibs-dev 
>>>  wrote:
>>> 
>>> Hi David,
>>> 
 On May 22, 2016, at 8:15 AM, David Hart via swift-corelibs-dev 
  wrote:
 
 Hello,
 
 The discussion we had previously on this mailing list made it quite clear 
 that:
 
 - Objective-C Foundation is the framework that is supposed to be used on 
 all Darwin platforms,
 - swift-corelibs-foundation will be the Foundation framework for all other 
 platforms,
 - Both frameworks will evolve slowly together.
>>> 
>>> Yup.
>>> 
 
 Therefore, it makes sense that for code written against Foundation to be 
 portable, the swift-corelibs-foundation APIs and behavior needs to be 
 identical to Darwin Foundation. Hence the following questions?
 
 - Shouldn't we be writing tests in a way so that they can be run against 
 Darwin Foundation and have the CI Server run them? For example: while 
 working on NSProgress, I write a bunch of tests against Darwin Foundation, 
 make sure they pass, then copy-paste them in the CoreLibs project, and fix 
 the implementation until they pass. This makes sure that both APIs are 
 consistent with each other. I was thinking that we ought to automate this 
 and have the CI server test them.
>>> 
>>> That would be a great step. This is part of the reason we tried to set up 
>>> the dependencies of Foundation on XCTest the way we did, and provide the 
>>> Xcode project file; so we could share tests. I would welcome any help we 
>>> can get on improving our automation story here.
>>> 
 
 - How are we planning to reconcile the API differences between both 
 frameworks? For examples, one of my tests will run against CoreLibs but 
 not against Darwin because NSThread.init takes a closure as argument in 
 CoreLibs but a target+selector in Darwin. This is just one example, but I 
 guess they are other inconsistencies elsewhere. This seems to be 
 particularly the case with APIs that rely on the Objective-C runtime.
>>> 
>>> These should be marked as “experimental” in the documentation comments. If 
>>> not, we should do that.
>>> 
>>> There are some areas where we just don’t know what to do yet, because of 
>>> the lack of the ObjC runtime and implicit bridging on Linux. In some of 
>>> those places we’ve tried to provide a replacement API and mark it as 
>>> experimental until we can figure out the larger story.
>>> 
 In general, I'm starting to worry about the state of Foundation from a 
 portability point of view. Once Swift 3 is released, I want to start 
 writing portable swift code that relies on Foundation. And it seems like 
 this will require a huge amount of #if os() everywhere to cope with the 
 differences.
 
 David
>>> 
>>> Our long term goal is to enable developers to do this. I acknowledge that 
>>> we have a ways to go to get there. I’ve seen an uptick in contributions 
>>> recently, which gives me hope that we can get closer before the release of 
>>> Swift 3.
>>> 
>>> - Tony
>>> 
>>> ___
>>> swift-corelibs-dev mailing list
>>> [email protected]
>>> https://lists.swift.org/mailman/listinfo/swift-corelibs-dev
>> 
>> ___
>> swift-corelibs-dev mailing list
>> [email protected]
>> https://lists.swift.org/mailman/listinfo/swift-corelibs-dev
> ___
> swift-corelibs-dev mailing list
> [email protected]
> https://lists.swift.org/mailman/listinfo/swift-corelibs-dev

_

[swift-corelibs-dev] [RFC] Toolchain based build process

2016-06-01 Thread Daniel Dunbar via swift-corelibs-dev 
Hi all,

The current build process for the overall Swift project (i.e., the compiler + 
associated projects like Foundation, XCTest, and SwiftPM) relies on each 
project having dependencies on the built artifacts of previously built 
projects. Those dependencies are currently communicated to each project in the 
chain through an ad hoc set of arguments to the individual project's build 
process. This has been painful to maintain, and makes it hard to reason about 
the environment that each of the associated projects are building within.

Instead, I would like to move towards what I have been calling a 
"toolchain-based" build process.

In this model:

1. The entire build process will be organized as a sequential, incremental 
construction of a complete toolchain.
- Each individual project will build and copy its products into a 
staging area.
2. The build script will always create a composed toolchain.
- It will start with an empty toolchain, and merge in the content from 
each project as it builds.
- This will use rsync & hard-links to avoid needing to stay fast.
3. Each individual project build will just use the composed toolchain to build.
- This will replace the grab bag of options we use to communicate 
between the projects.
4. At the end of the build, we will have constructed a complete toolchain which 
can be installed (or used with Xcode).

Aside from simplifying the overall build process, this has a couple very nice 
upsides:

1. At the end of the build, the user has a complete toolchain. They can install 
it, or use it as they would a distributed snapshot. This is very beneficial for 
people who are only building the Swift project to get a more recent version of 
the compiler.

2. Each individual project can be built using the "official" build process with 
a downloaded snapshot (assuming it is of a new enough version). This is very 
beneficial for easing contribution to projects like Foundation, XCTest, and 
SwiftPM which are pure Swift and have fast build times, but which currently 
build in Swift CI using a very different process from the snapshot-based 
workflow.

Concrete details:

1. I do not plan to change the actual install process in the short term. The 
actual install process used today relies on the CMake-based install process for 
some projects (most importantly Swift) and isn't suitable for use in this 
fashion (where incremental development speed is of high importance). Instead, 
my plan is to teach the build script itself how to assemble the staging area 
for each individual project, with the long term goal of using that for the 
official install process instead of the CMake-based process.

2. My plan is that the build script would only support building one "primary 
product" (i.e. toolchain). That product may itself be a complete cross compiler 
toolchain with support for multiple platforms, but the expectation is that 
users would invoke the build script multiple times if building multiple 
toolchains. However, to support Canadian Cross [1] build scenarios the build 
script may need to manage the construction of two products, the primary 
toolchain and the intermediate (cross compiling) toolchain used to build the 
artifacts in the primary toolchain.

3. As a concrete example of a problem this solves, SwiftPM currently doesn't 
test its `swift-test` product in Swift CI, because that product requires using 
all of the toolchain stack (swift, Foundation, and XCTest), but the product 
itself is only intended to be used as part of a concrete toolchain. As such, it 
doesn't know how to operate correctly when given the piecemeal build products 
for each of those projects, and teaching it to do so would be very cumbersome 
for us to maintain.

Feedback welcome!

 - Daniel

[1] https://en.wikipedia.org/wiki/Cross_compiler#Canadian_Cross

___
swift-corelibs-dev mailing list
[email protected]
https://lists.swift.org/mailman/listinfo/swift-corelibs-dev

Re: [swift-corelibs-dev] [swift-dev] [RFC] Toolchain based build process

2016-06-03 Thread Daniel Dunbar via swift-corelibs-dev 

> On Jun 3, 2016, at 1:14 PM, Saleem Abdulrasool  wrote:
> 
> On Wed, Jun 1, 2016 at 2:18 PM, Daniel Dunbar via swift-dev 
> mailto:[email protected]>> wrote:
> Hi all,
> 
> The current build process for the overall Swift project (i.e., the compiler + 
> associated projects like Foundation, XCTest, and SwiftPM) relies on each 
> project having dependencies on the built artifacts of previously built 
> projects. Those dependencies are currently communicated to each project in 
> the chain through an ad hoc set of arguments to the individual project's 
> build process. This has been painful to maintain, and makes it hard to reason 
> about the environment that each of the associated projects are building 
> within.
> 
> Instead, I would like to move towards what I have been calling a 
> "toolchain-based" build process.
> 
> In this model:
> 
> 1. The entire build process will be organized as a sequential, incremental 
> construction of a complete toolchain.
> - Each individual project will build and copy its products into a 
> staging area.
> 2. The build script will always create a composed toolchain.
> - It will start with an empty toolchain, and merge in the content 
> from each project as it builds.
> - This will use rsync & hard-links to avoid needing to stay fast.
> 3. Each individual project build will just use the composed toolchain to 
> build.
> - This will replace the grab bag of options we use to communicate 
> between the projects.
> 4. At the end of the build, we will have constructed a complete toolchain 
> which can be installed (or used with Xcode).
> 
> Aside from simplifying the overall build process, this has a couple very nice 
> upsides:
> 
> 1. At the end of the build, the user has a complete toolchain. They can 
> install it, or use it as they would a distributed snapshot. This is very 
> beneficial for people who are only building the Swift project to get a more 
> recent version of the compiler.
> 
> 2. Each individual project can be built using the "official" build process 
> with a downloaded snapshot (assuming it is of a new enough version). This is 
> very beneficial for easing contribution to projects like Foundation, XCTest, 
> and SwiftPM which are pure Swift and have fast build times, but which 
> currently build in Swift CI using a very different process from the 
> snapshot-based workflow.
> 
> Concrete details:
> 
> 1. I do not plan to change the actual install process in the short term. The 
> actual install process used today relies on the CMake-based install process 
> for some projects (most importantly Swift) and isn't suitable for use in this 
> fashion (where incremental development speed is of high importance). Instead, 
> my plan is to teach the build script itself how to assemble the staging area 
> for each individual project, with the long term goal of using that for the 
> official install process instead of the CMake-based process.
> 
> 2. My plan is that the build script would only support building one "primary 
> product" (i.e. toolchain). That product may itself be a complete cross 
> compiler toolchain with support for multiple platforms, but the expectation 
> is that users would invoke the build script multiple times if building 
> multiple toolchains. However, to support Canadian Cross [1] build scenarios 
> the build script may need to manage the construction of two products, the 
> primary toolchain and the intermediate (cross compiling) toolchain used to 
> build the artifacts in the primary toolchain.
> 
> 3. As a concrete example of a problem this solves, SwiftPM currently doesn't 
> test its `swift-test` product in Swift CI, because that product requires 
> using all of the toolchain stack (swift, Foundation, and XCTest), but the 
> product itself is only intended to be used as part of a concrete toolchain. 
> As such, it doesn't know how to operate correctly when given the piecemeal 
> build products for each of those projects, and teaching it to do so would be 
> very cumbersome for us to maintain.
> 
> Feedback welcome!
> 
> Overall, I think that this would be a great change.
> 
> I am however slightly confused on why the build script needs to be concerned 
> about candian cross at all if it can be taught how to use a specified 
> toolchain.
> 
> Because these (standard at least as per auto tools) terms can be confusing:
> 
> [1] build - the platform where things are being built
> [2] host - the platform where the generated binaries will be run
> [3] target - the platform where the binaries generated by the compiler on the 
> host platform will run
> 
> It should be possible to shift the burden on the user.

Why would we _want_ to shift the burden to the user?

>From the user's perspective, they want to end up with a toolchain that 
>compiles `Host -> Target`... I see it as an implementation detail (of the 
>`build-script`) that producing what the user wants requires us to build the 
>intermediate `Build ->

Re: [swift-corelibs-dev] [swift-build-dev] [swift-dev] [RFC] Toolchain based build process

2016-06-10 Thread Daniel Dunbar via swift-corelibs-dev 
Hi Karl,

On Fri, Jun 10, 2016 at 2:38 AM, Karl via swift-build-dev <
[email protected]> wrote:

> We can’t separate building for ‘Build’ and building for an arbitrary
> ‘Host’. For those who don’t know, this is how cross-compiling works with
> Swift (this isn’t off-topic, I’m going to bring it back firmly to this
> proposal):
>
> Currently, when you install the compiler, the standard library is in
> (prefix)/lib/swift/(Target)/(arch). On OSX, we cross-compile the standard
> library for the iOS targets, so you’ll see subfolders for ‘macosx’,
> ‘iphoneos’, etc in there. All you need to cross-compile in Swift is the
> standard library; swiftmodules don’t need ‘ar’, so libraries don’t even
> need any binutils to cross-compile, which is refreshing. A cross-linker
> *is* unfortunately still required for executables (come on, LLD!).
>
> Grab the standard library from a linux machine and try:
> echo 'print("hello, world")' | ./swiftc —target=x86-unknown-linux-gnu
> -resource-dir {PATH_TO_LINUX_STDLIB} -emit-module -
>
> So when we’re cross-compiling the swift portions of the standard-library,
> we’re using the swift compiler we just made for Build, telling it to use
> Host’s lib/swift directory, and there it will find the correct subdirectory
> for Target. My native compiler for my OSX Build machine knows not to look
> for the Linux host’s own Target in swift-macosx-x86_64/lib/swift, but in
> swift-linux-armv7/lib/swift.
>
> —
>
> Okay, so that’s out of the way.
>
> There is an actual problem with how we build/configure the target
> libraries (Foundation, XCTest and libdispatch). We currently configure them
> like every other product (for each host), instead of for each target. A
> consequence of this is, for example, that we won’t try to build Foundation
> for cross-compile targets such as Android. What we should do is this:
>
> For each host:
> -> Build cmark/llvm/swift
> -> For each target of this host:
> -> -> Build Foundation/XCTest/libdispatch
> -> Build swiftpm
>
> When you look at it like this, from a tools // target-libraries
> perspective (swiftpm is an exception; it’s a host tool, but it depends on
> Foundation), I came up with a different idea (possibly, I’m not sure), that
> might also help the dependency issue, although I don’t know the specifics
> of it: I would suggest that after building swift, we install it, then build
> the target libraries and install them as we do so.
>
> That would give us a stable standard-library (lib/swift) location which we
> could treat like an installed system. So when Foundation builds (for each
> Target now), it sets -resource-dir to the installed Host copy of swift, and
> the Build swift compiler can find Target’s standard library (yeah, I know).
> It installs itself in there. Then XCTest comes along, it sets the same
> -resource-dir, so now it can find Foundation just with ‘import Foundation’
> and nothing else. In other words: every product after the swift compiler
> can assume its dependencies are like they’ve been installed on the system.
>
> I’m not sure if that’s what you’re referring to with a ‘toolchain-based’
> build process or not. The reason I think it might be is because I don’t
> think this would require big changes. We’d just have to install products
> after swift, then install the target libraries as we go. Foundation could
> even keep on generating build.ninja in-tree, sadly.
>

Yup, this is exactly what I was referring to (although for different
reasons).

 - Daniel


> There isn’t much difference between supporting one cross-host or
> supporting n cross-hosts. I would prefer ’n’, if we could get a nice
> argument syntax for it, because it’s more convenient to integrate with
> other tools, but it’s not a big deal. Actually, I have an idea for a better
> build-script argument syntax which would scale to NxM cross-compiled hosts
> and targets elegantly, but that’s a topic for another day.
>
> Karl
>
>
> On 3 Jun 2016, at 22:31, Daniel Dunbar via swift-dev 
> wrote:
>
>
> On Jun 3, 2016, at 1:14 PM, Saleem Abdulrasool 
> wrote:
>
> On Wed, Jun 1, 2016 at 2:18 PM, Daniel Dunbar via swift-dev <
> [email protected]> wrote:
>
>> Hi all,
>>
>> The current build process for the overall Swift project (i.e., the
>> compiler + associated projects like Foundation, XCTest, and SwiftPM) relies
>> on each project having dependencies on the built artifacts of previously
>> built projects. Those dependencies are currently communicated to each
>> project in the chain through an ad hoc set of arguments to the individual
>> project's build process. This has been painful to maintain, and makes it
>> hard to reason about the environment that each of the associated projects
>> are building within.
>>
>> Instead, I would like to move towards what I have been calling a
>> "toolchain-based" build process.
>>
>> In this model:
>>
>> 1. The entire build process will be organized as a sequential,
>> incremental construction of a complete toolchain.
>> - Each in

Re: [swift-corelibs-dev] Wrapping up Swift 3 for swift-corelibs

2016-08-03 Thread Daniel Dunbar via swift-corelibs-dev 
Can someone file a bug to track how we are going to resolve this issue, and 
outline our options? I want to make sure we have enough eyes on it to make sure 
we get what we need for Swift 3.

 - Daniel

> On Aug 1, 2016, at 3:08 AM, Chris Bailey  wrote:
> 
> I had a quick conversation with Daniel Dunbar on this. Adding -Xcc -fblocks 
> as a default option to Swift PM (on Linux). The wrinkle is the availability 
> of the blocks runtime - it would either need to become a pre-req for Swift, 
> or needs one to be built into the package. 
> 
> Chris
> 
> 
> 
> 
> From:David P Grove via swift-corelibs-dev 
>  
> To:Swift corelibs dev  
> Date:28/07/2016 18:50 
> Subject:Re: [swift-corelibs-dev] Wrapping up Swift 3 for 
> swift-corelibs 
> Sent by:[email protected] 
> 
> 
> 
> Tony Parker wrote on 07/28/2016 01:41:55 PM:
> > 
> > 1. Integrate swift-corelibs-dispatch into Foundation.
> 
> Hi Tony,
> 
> Hopefully this is on the task list already, but if it isn't we should add it 
> before it gets to be too late to change the compiler...
> 
> When compiling a Swift program on Linux that imports Dispatch (or Foundation 
> once the integration is done), the user has to give the extra compilation 
> flags -Xcc -fblocks to enable block support.
> 
> We really need to land a change somewhere so that either (1) blocks support 
> is always on for Linux or (2) importing Dispatch or Foundation automatically 
> turns on blocks support.
> 
> I have some time today and tomorrow that I could use to work on this if no 
> one is handling it already, but I'm not sure how best to tackle the problem.  
> Suggestions?
> 
> --dave
> ___
> swift-corelibs-dev mailing list
> [email protected]
> https://lists.swift.org/mailman/listinfo/swift-corelibs-dev 
> 
> 
> 

___
swift-corelibs-dev mailing list
[email protected]
https://lists.swift.org/mailman/listinfo/swift-corelibs-dev

Re: [swift-corelibs-dev] [swift-build-dev] JIRA workflow: "Resolved" -> "Verify"?

2016-08-15 Thread Daniel Dunbar via swift-corelibs-dev 
Do we anticipate Verify being used in practice much? 

Would it be better to simplify the workflow and just have a single 
"Resolved/Closed/Done" state? If the originator does test the bug and find it 
isn't fixed, they can reopen.

My guess is not that many people are going to actively look at their JIRA 
account to find bugs that they are supposed to be verifying.

 - Daniel

> On Aug 15, 2016, at 10:08 AM, Jordan Rose via swift-build-dev 
>  wrote:
> 
> Hi, swift-dev et al (but especially Ted). I’ve recently noticed that our JIRA 
> workflow has been a bit confusing. We currently have five “statuses":
> 
> 1. Opened: This bug has been filed.
> 2. In Progress: Someone is actively working on this bug. (Not everyone has 
> been bothering to set this, but it seems reasonable to have.)
> 3. Resolved: A fix has been merged to master.
> 4. Closed: The fix has been verified by the reporter.
> 5. Reopened: The “fix" doesn’t actually fix the reporter’s problem.
> 
> The problem is that the “Resolved” and “Closed” statuses aren’t really 
> distinguished on the site itself—it’s unclear for a contributor which one 
> they’re supposed to use when they get something merged, and it’s unclear for 
> the reporter what they’re supposed to say. Therefore, I suggest changing the 
> “Resolved” status to “Verify” (like we use in Radar) and the “Resolve Issue” 
> button to “Mark Resolved”.
> 
> (There are other things unspecified here, such as how to track what release a 
> fix gets into, or what the story is for the Assignee field, but we can 
> discuss those later.)
> 
> What do you think?
> Thanks,
> Jordan
> ___
> swift-build-dev mailing list
> [email protected]
> https://lists.swift.org/mailman/listinfo/swift-build-dev

___
swift-corelibs-dev mailing list
[email protected]
https://lists.swift.org/mailman/listinfo/swift-corelibs-dev

Re: [swift-corelibs-dev] [swift-dev] [Swift CI] Build Still Failing: 0. OSS - Swift Incremental RA - Ubuntu 14.04 - Long Test (swift 3.0) #302

2016-09-24 Thread Daniel Dunbar via swift-corelibs-dev 
Alternately, we could change their build scripts to generate llbuild manifest 
files and build using llbuild (which tracks the Swift compiler version 
automatically as a dependency). This is why SwiftPM doesn't usually hit this 
problem, for example.

 - Daniel

> On Sep 24, 2016, at 7:57 PM, Michael Gottesman via swift-dev 
>  wrote:
> 
> 
>> On Sep 23, 2016, at 10:02 AM, Jordan Rose via swift-dev > > wrote:
>> 
>>> Foundation/NSData.swift:19:8: error: module file was created by an older 
>>> version of the compiler; rebuild 'Dispatch' and try again: 
>>> /home/buildnode/jenkins/workspace/oss-swift-3.0-incremental-RA-linux-ubuntu-14_04-long-test/buildbot_incremental/libdispatch-linux-x86_64/src/swift/Dispatch.swiftmodule
>>> 
>>> import Dispatch
>>>^
>> 
>> I thought we fixed this—we need to build Swift, then libdispatch, then the 
>> Dispatch overlay, then Foundation. Has no one set that up yet?
> 
> I wonder if it would just make sense to create a cmake build system for 
> dispatch and have ninja just understand the dependencies. If we update to a 
> newer cmake (3.6), we can use subninja's to just have ninja understand the 
> dependencies directly.
> 
>> 
>> 
>>> On Sep 23, 2016, at 0:51, [email protected]  
>>> wrote:
>>> 
>>> New issue found!
>>> 
>>> [FAILURE] oss-swift-3.0-incremental-RA-linux-ubuntu-14_04-long-test [#302]
>>> 
>>> Build URL:  
>>> https://ci.swift.org/job/oss-swift-3.0-incremental-RA-linux-ubuntu-14_04-long-test/302/
>>>  
>>> 
>>> Project:oss-swift-3.0-incremental-RA-linux-ubuntu-14_04-long-test
>>> Date of build:  Fri, 23 Sep 2016 00:42:00 -0700
>>> Build duration: 9 min 45 sec
>>> Identified problems:
>>> 
>>> Compile Error: This build failed because of a compile error. Below is a 
>>> list of all errors in the build log:
>>> Indication 1 
>>> 
>>> Tests:
>>> 
>>> Name: Swift(linux-x86_64)
>>> Failed: 0 test(s), Passed: 8245 test(s), Total: 8245 test(s)
>>> Name: Swift-Unit
>>> Failed: 0 test(s), Passed: 230 test(s), Total: 230 test(s)
>>> 
>>> Changes
>>> 
>>> Commit d8a375727247d3c9e3295a9af1fec079485c5310 by jordan_rose:
>>> Merge pull request #4682 from jrose-apple/generic-param-fix-it
>>> 
>>> edit: test/Constraints/construction.swift
>>> edit: test/Constraints/generics.swift
>>> edit: include/swift/AST/DiagnosticsSema.def
>>> edit: lib/FrontendTool/FrontendTool.cpp
>>> edit: lib/Sema/TypeChecker.h
>>> edit: lib/Sema/MiscDiagnostics.cpp
>>> edit: lib/Sema/TypeCheckType.cpp
>>> edit: test/Constraints/bridging.swift
>>> edit: test/Constraints/incomplete_function_ref.swift
>>> edit: test/FixCode/fixits-apply.swift
>>> edit: test/decl/typealias/generic.swift
>>> edit: test/FixCode/fixits-apply.swift.result
>>> edit: lib/Sema/CSDiag.cpp
>>> 
>>> Commit 509ee4c7c8113e5865a22ffb9053b0236d7160ec by jordan_rose:
>>> Don't use construction to convert literals in rawValue fix-it. (#4934)
>>> 
>>> edit: test/FixCode/fixits-apply.swift.result
>>> edit: lib/Sema/CSDiag.cpp
>>> edit: test/FixCode/fixits-apply.swift
>> 
>> ___
>> swift-dev mailing list
>> [email protected] 
>> https://lists.swift.org/mailman/listinfo/swift-dev
> 
> ___
> swift-dev mailing list
> [email protected]
> https://lists.swift.org/mailman/listinfo/swift-dev

___
swift-corelibs-dev mailing list
[email protected]
https://lists.swift.org/mailman/listinfo/swift-corelibs-dev

Re: [swift-corelibs-dev] [swift-dev] [Swift CI] Build Still Failing: 0. OSS - Swift Incremental RA - Ubuntu 14.04 - Long Test (swift 3.0) #302

2016-09-24 Thread Daniel Dunbar via swift-corelibs-dev 

> On Sep 24, 2016, at 9:33 PM, Michael Gottesman  wrote:
> 
> 
>> On Sep 24, 2016, at 8:15 PM, Daniel Dunbar > > wrote:
>> 
>> Alternately, we could change their build scripts to generate llbuild 
>> manifest files and build using llbuild (which tracks the Swift compiler 
>> version automatically as a dependency). This is why SwiftPM doesn't usually 
>> hit this problem, for example.
> 
> I do not understand how swiftpm could help here given that part of the issue 
> is in the dispatch overlay. This is currently built by Swift's cmake build 
> system. Could you be more explicit about what you are suggesting?

I didn't say swiftpm, I said an llbuild manifest. The actual build system 
SwiftPM uses is provided by llbuild's swift-build-tool, and doesn't depend on 
Swift.

I haven't kept up with how libdispatch is built, I assumed it was using a 
custom build process like Foundation and XCTest.

 - Daniel

> 
> Michael
> 
>> 
>>  - Daniel
>> 
>>> On Sep 24, 2016, at 7:57 PM, Michael Gottesman via swift-dev 
>>> mailto:[email protected]>> wrote:
>>> 
>>> 
 On Sep 23, 2016, at 10:02 AM, Jordan Rose via swift-dev 
 mailto:[email protected]>> wrote:
 
> Foundation/NSData.swift:19:8: error: module file was created by an older 
> version of the compiler; rebuild 'Dispatch' and try again: 
> /home/buildnode/jenkins/workspace/oss-swift-3.0-incremental-RA-linux-ubuntu-14_04-long-test/buildbot_incremental/libdispatch-linux-x86_64/src/swift/Dispatch.swiftmodule
> 
> import Dispatch
>^
 
 I thought we fixed this—we need to build Swift, then libdispatch, then the 
 Dispatch overlay, then Foundation. Has no one set that up yet?
>>> 
>>> I wonder if it would just make sense to create a cmake build system for 
>>> dispatch and have ninja just understand the dependencies. If we update to a 
>>> newer cmake (3.6), we can use subninja's to just have ninja understand the 
>>> dependencies directly.
>>> 
 
 
> On Sep 23, 2016, at 0:51, [email protected]  
> wrote:
> 
> New issue found!
> 
> [FAILURE] oss-swift-3.0-incremental-RA-linux-ubuntu-14_04-long-test [#302]
> 
> Build URL:
> https://ci.swift.org/job/oss-swift-3.0-incremental-RA-linux-ubuntu-14_04-long-test/302/
>  
> 
> Project:  oss-swift-3.0-incremental-RA-linux-ubuntu-14_04-long-test
> Date of build:Fri, 23 Sep 2016 00:42:00 -0700
> Build duration:   9 min 45 sec
> Identified problems:
> 
> Compile Error: This build failed because of a compile error. Below is a 
> list of all errors in the build log:
> Indication 1 
> 
> Tests:
> 
> Name: Swift(linux-x86_64)
> Failed: 0 test(s), Passed: 8245 test(s), Total: 8245 test(s)
> Name: Swift-Unit
> Failed: 0 test(s), Passed: 230 test(s), Total: 230 test(s)
> 
> Changes
> 
> Commit d8a375727247d3c9e3295a9af1fec079485c5310 by jordan_rose:
> Merge pull request #4682 from jrose-apple/generic-param-fix-it
> 
> edit: test/Constraints/construction.swift
> edit: test/Constraints/generics.swift
> edit: include/swift/AST/DiagnosticsSema.def
> edit: lib/FrontendTool/FrontendTool.cpp
> edit: lib/Sema/TypeChecker.h
> edit: lib/Sema/MiscDiagnostics.cpp
> edit: lib/Sema/TypeCheckType.cpp
> edit: test/Constraints/bridging.swift
> edit: test/Constraints/incomplete_function_ref.swift
> edit: test/FixCode/fixits-apply.swift
> edit: test/decl/typealias/generic.swift
> edit: test/FixCode/fixits-apply.swift.result
> edit: lib/Sema/CSDiag.cpp
> 
> Commit 509ee4c7c8113e5865a22ffb9053b0236d7160ec by jordan_rose:
> Don't use construction to convert literals in rawValue fix-it. (#4934)
> 
> edit: test/FixCode/fixits-apply.swift.result
> edit: lib/Sema/CSDiag.cpp
> edit: test/FixCode/fixits-apply.swift
 
 ___
 swift-dev mailing list
 [email protected] 
 https://lists.swift.org/mailman/listinfo/swift-dev 
 
>>> 
>>> ___
>>> swift-dev mailing list
>>> [email protected] 
>>> https://lists.swift.org/mailman/listinfo/swift-dev 
>>> 
>> 
> 

___
swift-corelibs-dev mailing list
[email protected]
https://lists.swift.org/mailman/listinfo/swift-corelibs-dev

Re: [swift-corelibs-dev] Better integration with UNIX tools

2017-12-01 Thread Daniel Dunbar via swift-corelibs-dev 

> On Dec 1, 2017, at 6:28 AM, Ian Partridge via swift-corelibs-dev 
>  wrote:
> 
> Hi Nick,
> 
> You might be interested in the new Utility project that the Package
> Manager team have published.  It has a bunch of Foundation-esque
> features including subprocess support, temporary file, progress bars
> and more.
> 
> There's a good blog post about it here:
> https://www.hackingwithswift.com/articles/44/apple-s-new-utility-library-will-power-up-command-line-apps
> 
> I hope it gets more publicity as there's some great functionality
> here, which has already been used "for real" by the package manager.

Just to be clear, the APIs in that product are developed for SwiftPM’s own use. 
They are definitely useful to us, but they are also unstable and unsupported… 
you are welcome to reuse them if you find them helpful, but they are very much 
designed specifically for SwiftPM and may change unexpectedly. Use at your own 
risk!

 - Daniel

> 
> Thanks,
> Ian Partridge
> 
> On 30 November 2017 at 08:43, Nick Keets via swift-corelibs-dev
>  wrote:
>> In that case I think the discussion is kind of moot, scripts are
>> fundamentally different than apps, being terse is important and almost
>> always you want to block.
>> 
>> If better scripting support is a non-goal for Foundation then `Process` is
>> mostly fine as it is. My only wish would be to somehow make it easier to
>> read and write `Data` to stdin/stdout/stderr.
>> 
>> 
>> On Tue, Nov 28, 2017 at 11:30 PM, Tony Parker 
>> wrote:
>>> 
>>> Of course, Foundation API has no way of distinguishing if the caller is
>>> considered a script or not.
>>> 
>>> If the API is a bad idea for other kinds of apps then we simply wouldn’t
>>> add it. So, I think this proposed convenience API needs to consider all of
>>> the varied clients of Foundation.
>>> 
>>> - Tony
>>> 
>>> 
>>> On Nov 28, 2017, at 12:24 PM, Brent Royal-Gordon 
>>> wrote:
>>> 
>>> On Nov 28, 2017, at 8:00 AM, Tony Parker  wrote:
>>> 
>>> Why does it imply a run loop rather than one of many multithreading
>>> possibilities (dispatch queue, starting one more thread, etc)? And even if
>>> it did use run loops, why is that a problem?
>>> 
>>> 
>>> The problem is simply that we're discussing using this feature in Swift
>>> *scripts*. Swift scripts don't typically invoke `RunLoop.run()` or
>>> `dispatch_main()`, and without changing the way they generate main()
>>> functions to intrinsically do so (and then schedule main.swift's body on the
>>> main runloop/queue), I don't see a good way for it to do so. So an async API
>>> would be inconvenient to use from a Swift script.
>>> 
>>> --
>>> Brent Royal-Gordon
>>> Architechies
>>> 
>>> 
>> 
>> 
>> ___
>> swift-corelibs-dev mailing list
>> [email protected]
>> https://lists.swift.org/mailman/listinfo/swift-corelibs-dev
>> 
> 
> 
> 
> -- 
> Ian Partridge
> ___
> swift-corelibs-dev mailing list
> [email protected]
> https://lists.swift.org/mailman/listinfo/swift-corelibs-dev

___
swift-corelibs-dev mailing list
[email protected]
https://lists.swift.org/mailman/listinfo/swift-corelibs-dev