I don't like the idea of some calls having wildly different semantics from others; it's difficult enough to tell what exactly a call might be doing already. Since we also lack the more obvious static "Callable" protocol idea to give even well-typed call syntax to user-defined types, this also seems like it'd be easily abused for that purpose too.
I think a much better general solution to the problem of "make dynamic systems interact with type systems" is something like F#'s type providers which lets you write your own importers that look at dynamic information from a database, dynamic language VM, or some other system and generate type information usable by the compiler. Integration at the importer level could let you produce more well-typed Swift declarations by looking at the runtime information you get by importing a Python module. -Joe > On Nov 10, 2017, at 9:37 AM, Chris Lattner via swift-evolution > <[email protected]> wrote: > > Hello all, > > I have a couple of proposals cooking in a quest to make Swift interoperate > with dynamically typed languages like Python better. Instead of baking in > hard coded support for one language or the other, I’m preferring to add a few > small but general purpose capabilities to Swift. This is the first, which > allows a Swift type to become “callable”. > > The proposal is here: > https://gist.github.com/lattner/a6257f425f55fe39fd6ac7a2354d693d > > I’ve also attached a snapshot below, but it will drift out of date as the > proposal is refined. Feedback and thoughts are appreciated, thanks! > > -Chris > > > > > > Introduce user-defined dynamically "callable" types > > • Proposal: SE-NNNN > • Author: Chris Lattner > • Review Manager: TBD > • Status: Awaiting implementation > Introduction > > This proposal introduces a new DynamicCallable protocol to the standard > library. Types that conform to it are "callable" with the function call > syntax. It is simple syntactic sugar which allows the user to write: > > a = someValue(keyword1: 42, "foo", keyword2: 19) > and have it be interpreted by the compiler as: > > a = someValue.dynamicCall(arguments > : [ > ( > "keyword1", 42), ("", "foo"), ("keyword2", 19 > ) > ]) > > Other languages have analogous features (e.g. Python "callables"), but the > primary motivation of this proposal is to allow elegant and natural > interoperation with dynamic languages in Swift. > > Swift-evolution thread: Discussion thread topic for that proposal > > Motivation > > Swift is well known for being exceptional at interworking with existing C and > Objective-C APIs, but its support for calling APIs written in scripting > langauges like Python, Perl, and Ruby is quite lacking. These languages > provide an extremely dynamic programming model where almost everything is > discovered at runtime. > > Through the introduction of this proposal, and the related > DynamicMemberLookupProtocol proposal, we seek to fix this problem. We believe > we can make many common APIs feel very natural to use directly from Swift > without all the complexity of implementing something like the Clang importer. > For example, consider this Python code: > > class Dog > : > > def __init__(self, name > ): > > self.name = > name > > self.tricks = [] # creates a new empty list for each dog > > > > def add_trick(self, trick > ): > > self.tricks.append(trick) > we would like to be able to use this from Swift like this (the comments show > the corresponding syntax you would use in Python): > > // import DogModule > // import DogModule.Dog as Dog // an alternate > let Dog = Python.import(“DogModule.Dog") > > // dog = Dog("Brianna") > let dog = Dog("Brianna") > > // dog.add_trick("Roll over") > dog.add_trick("Roll over") > > // dog2 = Dog("Kaylee").add_trick("snore") > let dog2 = Dog("Kaylee").add_trick("snore") > Of course, this would also apply to standard Python APIs as well. Here is an > example working with the Python pickleAPI and the builtin Python function > open: > > // import pickle > let pickle = Python.import("pickle" > ) > > > // file = open(filename) > let file = Python.open > (filename) > > > // blob = file.read() > let blob = file.read > () > > > // result = pickle.loads(blob) > let result = pickle.loads(blob) > This can all be expressed today as library functionality written in Swift, > but without this proposal, the code required is unnecessarily verbose and > gross. Without it (but with the related dynamic member lookup proposal) the > code would have a method name (like call) all over the code: > > // import pickle > let pickle = Python.import("pickle") // normal method in Swift, no change. > > > // file = open(filename) > let file = Python.open.call > (filename) > > > // blob = file.read() > let blob = file.read.call > () > > > // result = pickle.loads(blob) > let result = pickle.loads.call > (blob) > > > // dog2 = Dog("Kaylee").add_trick("snore") > let dog2 = Dog.call("Kaylee").add_trick.call("snore") > While this is a syntactic sugar proposal, we believe that this expands Swift > to be usable in important new domains. This sort of capability is also highly > precedented in other languages, and is a generally useful language feature > that could be used for other purposes as well. > > Proposed solution > > We propose introducing this protocol to the standard library: > > protocol DynamicCallable > { > > associatedtype DynamicCallableArgument > > > associatedtype DynamicCallableResult > > > > func dynamicCall(arguments: [(String, DynamicCallableArgument)]) throws -> > DynamicCallableResult > } > > It also extends the language such that function call syntax - when applied to > a value of DynamicCallable type - is accepted and transformed into a call to > the dynamicCall member. The dynamicCall method takes a list of tuples: the > first element is the keyword label (or an empty string if absent) and the > second value is the formal parameter specified at the call site. > > Before this proposal, the Swift language has two types that participate in > call syntax: functions and metatypes (for initialization). Neither of those > may conform to protocols at the moment, so this introduces no possible > ambiguity into the language. > > It is worth noting that this does not introduce the ability to provide > dynamicly callable static/class members. We don't believe that this is > important given the goal of supporting dynamic languages like Python, but if > there is a usecase discovered in the future, it could be explored as future > work. Such future work should keep in mind that call syntax on metatypes is > already meaningful, and that ambiguity would have to be resolved somehow. > > Discussion > > While the signature for dynamicCall is highly general we expect the most > common use will be clients who are programming against concrete types that > implement this proposal. One very nice aspect of this is that, as a result of > Swift's existing subtyping mechanics, implementations of this type can choose > whether they can actually throw an error or not. For example, consider this > silly implementation: > > struct ParameterSummer : DynamicCallable > { > > func dynamicCall(arguments: [(String, Int)]) -> Int > { > > return arguments.reduce(0) { $0+$1 > .1 } > } > } > > > let x = ParameterSummer > () > > print(x(1, 7, 12)) // prints 20 > Because ParameterSummer's implementation of dynamicCall does not throw, the > call site is known not to throw either, so the print doesn't need to be > marked with try. > > Example Usage > > A more realistic (and motivating) example comes from a prototype Python > interop layer. While the concrete details of this use case are subject to > change and not important for this proposal, it is perhaps useful to have a > concrete example to see how this comes together. > > That prototype currently has two types which model Python values, one of > which handles Python exceptions and one of which does not. Their conformances > would look like this, enabling the use cases described in the Motivation > section above: > > extension ThrowingPyRef: DynamicCallable > { > > func dynamicCall(arguments: [(String, PythonConvertible)]) throws > > > -> > PythonConvertible { > > // Make sure state errors are not around. > assert(PyErr_Occurred() == nil, "Python threw an error but wasn't handled" > ) > > > // Count how many keyword arguments are in the list. > let numKeywords = arguments.reduce(0 > ) { > > $0 + ($1.0.isEmpty ? 0 : 1 > ) > } > > > let kwdict = numKeywords != 0 ? PyDict_New() : nil > > > > // Non-keyword arguments are passed as a tuple of values. > let argTuple = PyTuple_New(arguments.count-numKeywords)! > > > var nonKeywordIndex = 0 > > > for (keyword, argValue) in > arguments { > > if keyword.isEmpty > { > > PyTuple_SetItem(argTuple, nonKeywordIndex, argValue.toPython > ()) > nonKeywordIndex > += 1 > > } > else > { > > PyDict_SetItem(kwdict!, keyword.toPython(), argValue.toPython > ()) > } > } > > > // Python calls always return a non-null value when successful. If the > // Python function produces the equivalent of C "void", it returns the > None > // value. A null result of PyObjectCall happens when there is an error, > // like 'self' not being a Python callable. > guard let resultPtr = PyObject_Call(state, argTuple, kwdict) else > { > > throw PythonError.invalidCall(self > ) > } > > > let result = PyRef(owned > : resultPtr) > > > // Translate a Python exception into a Swift error if one was thrown. > if let exception = PyErr_Occurred > () { > > PyErr_Clear > () > > throw PythonError.exception(PyRef(borrowed > : exception)) > } > > > return > result > } > } > > > extension PyRef: DynamicCallable > { > > func dynamicCall(arguments: [(String > , PythonConvertible)]) > > -> > PythonConvertible { > > // Same as above, but internally aborts instead of throwing Swift > // errors. > } > } > > Source compatibility > > This is a strictly additive proposal with no source breaking changes. > > Effect on ABI stability > > This is a strictly additive proposal with no ABI breaking changes. > > Effect on API resilience > > This has no impact on API resilience which is not already captured by other > language features. > > Alternatives considered > > A few alternatives were considered: > > Add ability to reject parameter labels > > The implementation above does not allow an implementation to staticly reject > argument labels. If this was important to add, we could add another protocol > to model this, along the lines of: > > /// A type conforming just to this protocol would not accept parameter > /// labels in its calls. > protocol DynamicCallable > { > > associatedtype DynamicCallableArgument > > > associatedtype DynamicCallableResult > > > > func dynamicCall(arguments: [DynamicCallableArgument]) throws -> > DynamicCallableResult > } > > > /// A type conforming to this protocol does allow optional parameter > /// labels. > protocol DynamicCallableWithKeywordsToo : DynamicCallable > { > > func dynamicCall(arguments: [(String, DynamicCallableArgument)]) throws -> > DynamicCallableResult > } > > This would allow a type to implement one or the other based on their > capabilities. This proposal is going with a very simple design, but if there > is demand for this, the author is happy to switch. > > Staticly checking for exact signatures > > This protocol does not allow a type to specify an exact signature for the > callable - a specific number of parameters with specific types. If we went > down that route, the best approach would be to introduce a new declaration > kind (which would end up being very similar to get-only subscripts) since, in > general, a type could want multiple concrete callable signatures, and those > signatures should participate in overload resolution. > > While such a feature could be interesting for some use cases, it is almost > entirely orthogonal from this proposal: it addresses different use cases and > does not solve the needs of this proposal. It does not address our needs > because even a variadic callable declaration would not provide access to the > keyword argument labels we need. > > Direct language support for Python > > We considered implementing something analogous to the Clang importer for > Python, which would add a first class Python specific type(s) to Swift > language or standard library. We rejected this option because it would be > significantly more invasive in the compiler, would set the precedent for all > other dynamic languages to get first class language support, and because that > first class support doesn't substantially improve the experience of working > with Python over existing Swift with a couple small "generally useful" > extensions like this one. > > Naming > > The most fertile ground for bikeshedding is the naming of the protocol and > the members. We welcome other ideas and suggestions for naming, but here are > some thoughts on obvious options to consider: > > We considered but rejected the name CustomCallable, because the existing > Custom* protocols in the standard library (CustomStringConvertible, > CustomReflectable, etc) provide a way to override and custom existing builtin > abilities of Swift. In contrast, this feature grants a new capability to a > type. > > We considered but rejected a name like ExpressibleByCalling to fit with the > ExpressibleBy* family of protocols (like ExpressibleByFloatLiteral, > ExpressibleByStringLiteral, etc). This name family is specifically used by > literal syntax, and calls are not literals. Additionally the type itself is > not "expressible by calling" - instead, instances of the type may be called. > > On member and associated type naming, we intentionally gave these long and > verbose names so they stay out of the way of user code completion. The > members of this protocol are really just compiler interoperability glue. If > there was a Swift attribute to disable the members from showing up in code > completion, we would use it (such an attribute would also be useful for the > LiteralConvertible and other compiler magic protocols). > > > _______________________________________________ > swift-evolution mailing list > [email protected] > https://lists.swift.org/mailman/listinfo/swift-evolution _______________________________________________ swift-evolution mailing list [email protected] https://lists.swift.org/mailman/listinfo/swift-evolution
