How would this work for a property access that caused a python exception to be thrown? Are we assuming that only "good" non-throwing properties will be bridged in this way?
``` >>> x = "hello" >>> x.foobar Traceback (most recent call last): File "<stdin>", line 1, in <module> AttributeError: 'str' object has no attribute 'foobar' ``` On Tue, Nov 14, 2017 at 11:29 PM, Chris Lattner via swift-evolution < [email protected]> wrote: > Hi All, > > As a peer to the DynamicCallable proposal (https://gist.github.com/ > lattner/a6257f425f55fe39fd6ac7a2354d693d), I’d like to get your feedback > on making member lookup dynamically extensible. My primary motivation is > to improve interoperability with dynamic languages like Python, Perl, Ruby, > Javascript, etc, but there are other use cases (e.g. when working with > untyped JSON). > > In addition to being a high impact on expressivity of Swift, I believe an > implementation can be done in a way with changes that are localized, and > thus not have significant impact on the maintainability of the compiler as > a whole. Once the pitch phase of this proposal helps refine the details, > I’ll be happy to prepare an implementation for consideration. > > In case it is useful, I’m working on cleaning up my current prototype > Python bindings. I’ll share them in the next day or two in case they are > useful to provide context. It is amazingly simple: less than 500 lines of > Swift code (plus some small additional C header glue to work around clang > importer limitations) enables impressive interoperability. The only > problems are the verbosity addressed by this proposal and the peer > DynamicCallable proposal. > > > Here is the canonical proposal URL: > https://gist.github.com/lattner/b016e1cf86c43732c8d82f90e5ae5438 > > A snapshot of the proposal is included below in case it is useful. Thanks > in advance for help improving the proposal! > > -Chris > > > Introduce User-defined "Dynamic Member Lookup" Types > > - Proposal: SE-NNNN > <https://gist.github.com/lattner/NNNN-DynamicMemberLookup.md> > - Author: Chris Lattner <https://github.com/lattner> > - Review Manager: TBD > - Status: Awaiting implementation > > > <https://gist.github.com/lattner/b016e1cf86c43732c8d82f90e5ae5438#introduction> > Introduction > > This proposal introduces a new DynamicMemberLookupProtocol type to the > standard library. Types that conform to it provide "dot" syntax for > arbitrary names which are resolved at runtime. It is simple syntactic sugar > which allows the user to write: > > a = someValue.someMember > someValue.someMember = a > > and have it be interpreted by the compiler as: > > a = someValue[dynamicMember: "someMember"] > someValue[dynamicMember: "someMember"] = a > > Many other languages have analogous features (e.g. the composition of > Objective-C's explicit properties > <https://developer.apple.com/library/content/documentation/General/Conceptual/DevPedia-CocoaCore/DeclaredProperty.html> > and > underlying messaging infrastructure > <https://developer.apple.com/library/content/documentation/Cocoa/Conceptual/ObjCRuntimeGuide/Articles/ocrtHowMessagingWorks.html>). > This sort of functionality is great for implementing dynamic language > interoperability, dynamic proxy APIs > <https://developer.apple.com/library/content/documentation/Cocoa/Conceptual/ObjCRuntimeGuide/Articles/ocrtForwarding.html>, > and other APIs (e.g. for JSON processing). > > Swift-evolution thread: Discussion thread topic for that proposal > <https://lists.swift.org/pipermail/swift-evolution/> > > <https://gist.github.com/lattner/b016e1cf86c43732c8d82f90e5ae5438#motivation-and-context>Motivation > and Context > > Swift is well known for being exceptional at interworking with existing C > and Objective-C APIs, but its support for calling APIs written in scripting > languages like Python, Perl, and Ruby is quite lacking. > > C and Objective-C are integrated into Swift by expending a heroic amount > of effort into integrating Clang ASTs, remapping existing APIs in an > attempt to feel "Swifty", and by providing a large number of attributes and > customization points for changing the behavior of this integration when > writing an Objective-C header. The end result of this massive investment of > effort is that Swift provides a *better* experience when programming > against these legacy APIs than Objective-C itself did. > > When considering the space of dynamic languages, three things are clear: > 1) there are several different languages of interest, and they each have > significant interest in different quarters: for example, Python is big in > data science and machine learning, Ruby is popular for building server side > apps, and even Perl is in still widely used. 2) These languages have > decades of library building behind them, sometimes with significant > communities <https://pandas.pydata.org/> and 3) there are one or two > orders of magnitude more users of these libraries than there are people > currently using Swift. > > While it is theoretically possible to expend the same level of effort on > each of these languages and communities as has been spent on Objective-C, > it is quite clear that this would both ineffective as well as bad for > Swift: It would be ineffective, because the Swift community has not > leverage over these communities to force auditing and annotation of their > APIs. It would be bad for Swift because it would require a ton of > language-specific support (and a number of third-party dependencies) onto > the compiler and runtime, each of which makes the implementation > significantly more complex, difficult to reason about, difficult to > maintain, and difficult to test the supported permutations. In short, we'd > end up with a mess. > > Fortunately for us, these scripting languages provide an extremely dynamic > programming model where almost everything is discovered at runtime, and > many of them are explicitly designed to be embedded into other languages > and applications. This aspect allows us to embed APIs from these languages > directly into Swift with no language support at all - without not the level > of effort, integration, and invasiveness that Objective-C has benefited > from. Instead of invasive importer work, we can write some > language-specific Swift APIs, and leave the interop details to that library. > > This offers a significant opportunity for us - the Swift community can > "embrace" these dynamic language APIs (making them directly available in > Swift) which reduces the pain of someone moving from one of those languages > into Swift. It is true that the APIs thus provided will not feel "Swifty", > but if that becomes a significant problem for any one API, then the > community behind it can evaluate the problem and come up with a solution > (either a Swift wrapper for the dynamic language, or a from-scratch Swift > reimplementation of the desired API). In any case, if/when we face this > challenge, it will be a good thing: we'll know that we've won a significant > new community of Swift developers. > > While it is possible today to import (nearly) arbitrary dynamic language > APIs into Swift today, the resultant API is unusable for two major reasons: > member lookup is too verbose to be acceptable, and calling behavior is > similarly too verbose to be acceptable. As such, we seek to provide two > "syntactic sugar" features that solve this problem. These sugars are > specifically designed to be dynamic language independent and, indeed, > independent of dynamic languages at all: we can imagine other usage for the > same primitive capabilities. > > The two proposals in question are the introduction of the DynamicCallable > <https://gist.github.com/lattner/a6257f425f55fe39fd6ac7a2354d693d> protocol > and a related DynamicMemberLookupProtocol proposal (this proposal). With > these two extensions, we think we can eliminate the need for invasive > importer magic by making interoperability with dynamic languages ergonomic > enough to be acceptable. > > For example, consider this Python code: > > class Dog: > def __init__(self, name): > self.name = name > self.tricks = [] # creates a new empty list for each dog > > def add_trick(self, trick): > self.tricks.append(trick) > > we would like to be able to use this from Swift like this (the comments > show the corresponding syntax you would use in Python): > > // import DogModule // import DogModule.Dog as Dog // an alternate let > Dog = Python.import(“DogModule.Dog") // dog = Dog("Brianna") let dog = > Dog("Brianna") // dog.add_trick("Roll over") dog.add_trick("Roll over") // > dog2 = Dog("Kaylee").add_trick("snore") let dog2 = > Dog("Kaylee").add_trick("snore") > > Of course, this would also apply to standard Python APIs as well. Here is > an example working with the Python pickleAPI and the builtin Python > function open: > > // import pickle let pickle = Python.import("pickle") > > // file = open(filename) let file = Python.open(filename) > > // blob = file.read() let blob = file.read() > > // result = pickle.loads(blob) let result = pickle.loads(blob) > > This can all be expressed today as library functionality written in Swift, > but without this proposal, the code required is unnecessarily verbose and > gross. Without it (but *with* the related DynamicCallable proposal > <https://gist.github.com/lattner/a6257f425f55fe39fd6ac7a2354d693d> the > code would have explicit member lookups all over the place: > > // import pickle let pickle = Python.get(member: "import")("pickle") > > // file = open(filename) let file = Python.get(member: "open")(filename) > > // blob = file.read() let blob = file.get(member: "read")() > > // result = pickle.loads(blob) let result = pickle.get(member: > "loads")(blob) > > // dog2 = Dog("Kaylee").add_trick("snore") let dog2 = > Dog("Kaylee").get(member: "add_trick")("snore") > > While this is a syntactic sugar proposal, we believe that this expands > Swift to be usable in important new domains. In addition to dynamic > language interoperability, this sort of functionality is useful for other > APIs, e.g. when working with dynamically typed unstructured data like JSON, > which could provide an API like jsonValue?.jsonField1?.jsonField2where > each field is dynamically looked up. > > <https://gist.github.com/lattner/b016e1cf86c43732c8d82f90e5ae5438#proposed-solution>Proposed > solution > > We propose introducing this protocol to the standard library: > > protocol DynamicMemberLookupProtocol { > associatedtype DynamicMemberLookupValue > > subscript(dynamicMember name: String) -> DynamicMemberLookupValue { get set > } > } > > It also extends the language such that member lookup syntax (x.y) - when > it otherwise fails (because there is no member y defined on the type of x) > and when applied to a value which conforms to DynamicMemberLookupProtocol- > is accepted and transformed into a call to the subscript in the protocol. > This ensures that no member lookup on such a type ever fails. > > <https://gist.github.com/lattner/b016e1cf86c43732c8d82f90e5ae5438#example-usage>Example > Usage > > While there are many potential uses of this sort of API (e.g. resolving > JSON members to named results, producing optional bindings) a motivating > example comes from a prototype Python interoperability layer. There are > many ways to implement this, and the details are not particularly > important, but it is perhaps useful to know that this is directly useful to > address the motivation section described above. Given a currency type of > PyVal (and a conforming implementation named PyRef), an implementation > may look like: > > extension PyVal { > subscript(dynamicMember member: String) -> PyVal { > get { > let result = PyObject_GetAttrString(borrowedPyObject, member)! > return PyRef(owned: result) // PyObject_GetAttrString returns +1 > result. } > set { > PyObject_SetAttrString(borrowedPyObject, member, > newValue.toPython().borrowedPyObject) > } > } > } > > > <https://gist.github.com/lattner/b016e1cf86c43732c8d82f90e5ae5438#source-compatibility>Source > compatibility > > This is a strictly additive proposal with no source breaking changes. > > <https://gist.github.com/lattner/b016e1cf86c43732c8d82f90e5ae5438#effect-on-abi-stability>Effect > on ABI stability > > This is a strictly additive proposal with no ABI breaking changes. > > <https://gist.github.com/lattner/b016e1cf86c43732c8d82f90e5ae5438#effect-on-api-resilience>Effect > on API resilience > > This has no impact on API resilience which is not already captured by > other language features. > > <https://gist.github.com/lattner/b016e1cf86c43732c8d82f90e5ae5438#alternatives-considered>Alternatives > considered > > A few alternatives were considered: > > <https://gist.github.com/lattner/b016e1cf86c43732c8d82f90e5ae5438#add-ability-to-provide-read-only-members>Add > ability to provide read-only members > > The implementation above does not allow an implementation to statically > reject members that are read-only. If this was important to add, we could > add another protocol to model this, along the lines of: > > protocol DynamicMemberLookupGettableProtocol { > associatedtype DynamicMemberLookupValue > > // gettable only subscript(dynamicMember name: String) -> > DynamicMemberLookupValue { get } > } > protocol DynamicMemberLookupProtocol : DynamicMemberLookupGettableProtocol { > // gettable and settable. subscript(dynamicMember name: String) -> > DynamicMemberLookupValue { get set } > } > > This would allow a type to implement one or the other based on their > capabilities. This proposal starts with a very simple design based on the > requirements of dynamic languages (which have no apparent immutability > model), but if there is demand for this (e.g. because we want input JSON > values to be gettable but not settalbe), the author is happy to switch to > this more general model. > <https://gist.github.com/lattner/b016e1cf86c43732c8d82f90e5ae5438#naming> > Naming > There is a lot of grounds to debate naming of the protocol and methods in > this type. Suggestions (along with rationale to support them) are more than > welcome. > > > _______________________________________________ > swift-evolution mailing list > [email protected] > https://lists.swift.org/mailman/listinfo/swift-evolution > > -- Functional Programmer, iOS Developer, Surfs Poorly http://twitter.com/n8gray
_______________________________________________ swift-evolution mailing list [email protected] https://lists.swift.org/mailman/listinfo/swift-evolution
