To close the loop on this pitch, I’ve gotten a lot of useful feedback from the discussion. I’ll prepare a revised version and do pitch #2 when I have time, perhaps later this week. Thanks!
-Chris > On Nov 10, 2017, at 9:37 AM, Chris Lattner <[email protected]> wrote: > > Hello all, > > I have a couple of proposals cooking in a quest to make Swift interoperate > with dynamically typed languages like Python better. Instead of baking in > hard coded support for one language or the other, I’m preferring to add a few > small but general purpose capabilities to Swift. This is the first, which > allows a Swift type to become “callable”. > > The proposal is here: > https://gist.github.com/lattner/a6257f425f55fe39fd6ac7a2354d693d > <https://gist.github.com/lattner/a6257f425f55fe39fd6ac7a2354d693d> > > I’ve also attached a snapshot below, but it will drift out of date as the > proposal is refined. Feedback and thoughts are appreciated, thanks! > > -Chris > > > > > > Introduce user-defined dynamically "callable" types > > Proposal: SE-NNNN <https://gist.github.com/lattner/NNNN-DynamicCallable.md> > Author: Chris Lattner <https://github.com/lattner> > Review Manager: TBD > Status: Awaiting implementation > > <https://gist.github.com/lattner/a6257f425f55fe39fd6ac7a2354d693d#introduction>Introduction > > This proposal introduces a new DynamicCallable protocol to the standard > library. Types that conform to it are "callable" with the function call > syntax. It is simple syntactic sugar which allows the user to write: > > a = someValue(keyword1: 42, "foo", keyword2: 19) > and have it be interpreted by the compiler as: > > a = someValue.dynamicCall(arguments: [ > ("keyword1", 42), ("", "foo"), ("keyword2", 19) > ]) > Other languages have analogous features (e.g. Python "callables"), but the > primary motivation of this proposal is to allow elegant and natural > interoperation with dynamic languages in Swift. > > Swift-evolution thread: Discussion thread topic for that proposal > <https://lists.swift.org/pipermail/swift-evolution/> > > <https://gist.github.com/lattner/a6257f425f55fe39fd6ac7a2354d693d#motivation>Motivation > > Swift is well known for being exceptional at interworking with existing C and > Objective-C APIs, but its support for calling APIs written in scripting > langauges like Python, Perl, and Ruby is quite lacking. These languages > provide an extremely dynamic programming model where almost everything is > discovered at runtime. > > Through the introduction of this proposal, and the related > DynamicMemberLookupProtocol proposal, we seek to fix this problem. We believe > we can make many common APIs feel very natural to use directly from Swift > without all the complexity of implementing something like the Clang importer. > For example, consider this Python code: > > class Dog: > def __init__(self, name): > self.name = name > self.tricks = [] # creates a new empty list for each dog > > def add_trick(self, trick): > self.tricks.append(trick) > we would like to be able to use this from Swift like this (the comments show > the corresponding syntax you would use in Python): > > // import DogModule > // import DogModule.Dog as Dog // an alternate > let Dog = Python.import(“DogModule.Dog") > > // dog = Dog("Brianna") > let dog = Dog("Brianna") > > // dog.add_trick("Roll over") > dog.add_trick("Roll over") > > // dog2 = Dog("Kaylee").add_trick("snore") > let dog2 = Dog("Kaylee").add_trick("snore") > Of course, this would also apply to standard Python APIs as well. Here is an > example working with the Python pickleAPI and the builtin Python function > open: > > // import pickle > let pickle = Python.import("pickle") > > // file = open(filename) > let file = Python.open(filename) > > // blob = file.read() > let blob = file.read() > > // result = pickle.loads(blob) > let result = pickle.loads(blob) > This can all be expressed today as library functionality written in Swift, > but without this proposal, the code required is unnecessarily verbose and > gross. Without it (but with the related dynamic member lookup proposal) the > code would have a method name (like call) all over the code: > > // import pickle > let pickle = Python.import("pickle") // normal method in Swift, no change. > > // file = open(filename) > let file = Python.open.call(filename) > > // blob = file.read() > let blob = file.read.call() > > // result = pickle.loads(blob) > let result = pickle.loads.call(blob) > > // dog2 = Dog("Kaylee").add_trick("snore") > let dog2 = Dog.call("Kaylee").add_trick.call("snore") > While this is a syntactic sugar proposal, we believe that this expands Swift > to be usable in important new domains. This sort of capability is also highly > precedented in other languages, and is a generally useful language feature > that could be used for other purposes as well. > > > <https://gist.github.com/lattner/a6257f425f55fe39fd6ac7a2354d693d#proposed-solution>Proposed > solution > > We propose introducing this protocol to the standard library: > > protocol DynamicCallable { > associatedtype DynamicCallableArgument > associatedtype DynamicCallableResult > > func dynamicCall(arguments: [(String, DynamicCallableArgument)]) throws -> > DynamicCallableResult > } > It also extends the language such that function call syntax - when applied to > a value of DynamicCallable type - is accepted and transformed into a call to > the dynamicCall member. The dynamicCall method takes a list of tuples: the > first element is the keyword label (or an empty string if absent) and the > second value is the formal parameter specified at the call site. > > Before this proposal, the Swift language has two types that participate in > call syntax: functions and metatypes (for initialization). Neither of those > may conform to protocols at the moment, so this introduces no possible > ambiguity into the language. > > It is worth noting that this does not introduce the ability to provide > dynamicly callable static/class members. We don't believe that this is > important given the goal of supporting dynamic languages like Python, but if > there is a usecase discovered in the future, it could be explored as future > work. Such future work should keep in mind that call syntax on metatypes is > already meaningful, and that ambiguity would have to be resolved somehow. > > > <https://gist.github.com/lattner/a6257f425f55fe39fd6ac7a2354d693d#discussion>Discussion > > While the signature for dynamicCall is highly general we expect the most > common use will be clients who are programming against concrete types that > implement this proposal. One very nice aspect of this is that, as a result of > Swift's existing subtyping mechanics, implementations of this type can choose > whether they can actually throw an error or not. For example, consider this > silly implementation: > > struct ParameterSummer : DynamicCallable { > func dynamicCall(arguments: [(String, Int)]) -> Int { > return arguments.reduce(0) { $0+$1.1 } > } > } > > let x = ParameterSummer() > print(x(1, 7, 12)) // prints 20 > Because ParameterSummer's implementation of dynamicCall does not throw, the > call site is known not to throw either, so the print doesn't need to be > marked with try. > > > <https://gist.github.com/lattner/a6257f425f55fe39fd6ac7a2354d693d#example-usage>Example > Usage > > A more realistic (and motivating) example comes from a prototype Python > interop layer. While the concrete details of this use case are subject to > change and not important for this proposal, it is perhaps useful to have a > concrete example to see how this comes together. > > That prototype currently has two types which model Python values, one of > which handles Python exceptions and one of which does not. Their conformances > would look like this, enabling the use cases described in the Motivation > section above: > > extension ThrowingPyRef: DynamicCallable { > func dynamicCall(arguments: [(String, PythonConvertible)]) throws > -> PythonConvertible { > // Make sure state errors are not around. > assert(PyErr_Occurred() == nil, "Python threw an error but wasn't > handled") > > // Count how many keyword arguments are in the list. > let numKeywords = arguments.reduce(0) { > $0 + ($1.0.isEmpty ? 0 : 1) > } > > let kwdict = numKeywords != 0 ? PyDict_New() : nil > > // Non-keyword arguments are passed as a tuple of values. > let argTuple = PyTuple_New(arguments.count-numKeywords)! > var nonKeywordIndex = 0 > for (keyword, argValue) in arguments { > if keyword.isEmpty { > PyTuple_SetItem(argTuple, nonKeywordIndex, argValue.toPython()) > nonKeywordIndex += 1 > } else { > PyDict_SetItem(kwdict!, keyword.toPython(), argValue.toPython()) > } > } > > // Python calls always return a non-null value when successful. If the > // Python function produces the equivalent of C "void", it returns the > None > // value. A null result of PyObjectCall happens when there is an error, > // like 'self' not being a Python callable. > guard let resultPtr = PyObject_Call(state, argTuple, kwdict) else { > throw PythonError.invalidCall(self) > } > > let result = PyRef(owned: resultPtr) > > // Translate a Python exception into a Swift error if one was thrown. > if let exception = PyErr_Occurred() { > PyErr_Clear() > throw PythonError.exception(PyRef(borrowed: exception)) > } > > return result > } > } > > extension PyRef: DynamicCallable { > func dynamicCall(arguments: [(String, PythonConvertible)]) > -> PythonConvertible { > // Same as above, but internally aborts instead of throwing Swift > // errors. > } > } > > <https://gist.github.com/lattner/a6257f425f55fe39fd6ac7a2354d693d#source-compatibility>Source > compatibility > > This is a strictly additive proposal with no source breaking changes. > > > <https://gist.github.com/lattner/a6257f425f55fe39fd6ac7a2354d693d#effect-on-abi-stability>Effect > on ABI stability > > This is a strictly additive proposal with no ABI breaking changes. > > > <https://gist.github.com/lattner/a6257f425f55fe39fd6ac7a2354d693d#effect-on-api-resilience>Effect > on API resilience > > This has no impact on API resilience which is not already captured by other > language features. > > > <https://gist.github.com/lattner/a6257f425f55fe39fd6ac7a2354d693d#alternatives-considered>Alternatives > considered > > A few alternatives were considered: > > > <https://gist.github.com/lattner/a6257f425f55fe39fd6ac7a2354d693d#add-ability-to-reject-parameter-labels>Add > ability to reject parameter labels > > The implementation above does not allow an implementation to staticly reject > argument labels. If this was important to add, we could add another protocol > to model this, along the lines of: > > /// A type conforming just to this protocol would not accept parameter > /// labels in its calls. > protocol DynamicCallable { > associatedtype DynamicCallableArgument > associatedtype DynamicCallableResult > > func dynamicCall(arguments: [DynamicCallableArgument]) throws -> > DynamicCallableResult > } > > /// A type conforming to this protocol does allow optional parameter > /// labels. > protocol DynamicCallableWithKeywordsToo : DynamicCallable { > func dynamicCall(arguments: [(String, DynamicCallableArgument)]) throws -> > DynamicCallableResult > } > This would allow a type to implement one or the other based on their > capabilities. This proposal is going with a very simple design, but if there > is demand for this, the author is happy to switch. > > > <https://gist.github.com/lattner/a6257f425f55fe39fd6ac7a2354d693d#staticly-checking-for-exact-signatures>Staticly > checking for exact signatures > > This protocol does not allow a type to specify an exact signature for the > callable - a specific number of parameters with specific types. If we went > down that route, the best approach would be to introduce a new declaration > kind (which would end up being very similar to get-only subscripts) since, in > general, a type could want multiple concrete callable signatures, and those > signatures should participate in overload resolution. > > While such a feature could be interesting for some use cases, it is almost > entirely orthogonal from this proposal: it addresses different use cases and > does not solve the needs of this proposal. It does not address our needs > because even a variadic callable declaration would not provide access to the > keyword argument labels we need. > > > <https://gist.github.com/lattner/a6257f425f55fe39fd6ac7a2354d693d#direct-language-support-for-python>Direct > language support for Python > > We considered implementing something analogous to the Clang importer for > Python, which would add a first class Python specific type(s) to Swift > language or standard library. We rejected this option because it would be > significantly more invasive in the compiler, would set the precedent for all > other dynamic languages to get first class language support, and because that > first class support doesn't substantially improve the experience of working > with Python over existing Swift with a couple small "generally useful" > extensions like this one. > > > <https://gist.github.com/lattner/a6257f425f55fe39fd6ac7a2354d693d#naming>Naming > > The most fertile ground for bikeshedding is the naming of the protocol and > the members. We welcome other ideas and suggestions for naming, but here are > some thoughts on obvious options to consider: > > We considered but rejected the name CustomCallable, because the existing > Custom* protocols in the standard library (CustomStringConvertible, > CustomReflectable, etc) provide a way to override and custom existing builtin > abilities of Swift. In contrast, this feature grants a new capability to a > type. > > We considered but rejected a name like ExpressibleByCalling to fit with the > ExpressibleBy* family of protocols (like ExpressibleByFloatLiteral, > ExpressibleByStringLiteral, etc). This name family is specifically used by > literal syntax, and calls are not literals. Additionally the type itself is > not "expressible by calling" - instead, instances of the type may be called. > > On member and associated type naming, we intentionally gave these long and > verbose names so they stay out of the way of user code completion. The > members of this protocol are really just compiler interoperability glue. If > there was a Swift attribute to disable the members from showing up in code > completion, we would use it (such an attribute would also be useful for the > LiteralConvertible and other compiler magic protocols). > >
_______________________________________________ swift-evolution mailing list [email protected] https://lists.swift.org/mailman/listinfo/swift-evolution
