In Java you can have annotation processors, user supplied compiler extensions,
that are extensively used for making frameworks easier to use and for general
interfacing to ‘foreign stuff’.
Would a limited form of user supplied compiler extension be an alternative, an
import processor. Special import syntax, import beginning with ‘Foreign’, would
be recognised by the compiler and a user supplied import processor would be
called, in the case of the example for Python. The import processor would
generate the glue code. Below is a basic example to give a feel for what I am
proposing.
(Syntax, names, etc. in code below just the 1st that came into my head!)
— Howard.
======================================================
import Foreign.Python.Foo // Special import tag `Foreign` calls a user supplied
compiler extension that parses the imported Python and generates something
along these lines (its up to the importer the exact code generated!):
protocol _ForeignPythonCallable {
func call(method: Any, arguments: Any...) throws -> Any
}
extension _ForeignPythonCallable {
func call(method: Any, arguments: Any...) throws -> Any {
return "method: \(method), arguments: \(arguments)" // Method lookup
and calling code for Python.
}
}
class Foo: _ForeignPythonCallable {
// Could override `call` if more efficient than complete general lookup was
possible.
enum _ForeignPythonMethodNames { // This might be a C struct so that it
matches perfectly what the python interpreter is expecting.
case bar
}
enum _ForeignPythonMethodBarArguments { // This might be a C struct so that
it matches perfectly what the python interpreter is expecting.
case x(Any)
case noName1(Any)
case y(Any)
}
func bar(x: Any, _ noName1: Any, y: Any) -> Any {
do {
return try call(method: _ForeignPythonMethodNames.bar, arguments:
_ForeignPythonMethodBarArguments.x(x),
_ForeignPythonMethodBarArguments.noName1(noName1),
_ForeignPythonMethodBarArguments.y(y))
} catch {
fatalError("Method `bar` does not throw, therefore Python importer
bug.")
}
}
}
// Then the consumer of `Foo` uses as per normal Swift:
let foo = Foo()
foo.bar(x: 1, 23, y: 17)
class TypedFoo: Foo { // For many dynamic languages this wouldn't be possible
to automatically generate via the import, therefore would have to be hand
written.
func bar(x: Int, _ noName1: Int, y: Int) -> String {
return super.bar(x: x, noName1, y: y) as! String
}
}
let typedFoo = TypedFoo()
typedFoo.bar(x: 1, 23, y: 17)
> On 12 Nov 2017, at 11:15 am, Andrew Bennett via swift-evolution
> <[email protected]> wrote:
>
> HI, this proposal looks really interesting!
>
> I have a few questions:
>
> Clarity on the proposal's intent
> Nice cheap bridges, or lowering barriers to bridging?
>
> I can see this providing a nice quick interface to Python from Swift, but I'm
> not sure if the exposed interface will be very Swifty (you probably have a
> much better idea of what is Swifty ;) than I do though). It seems you want it
> to be possible for everything to be dynamically exposed, I've used similar
> with Lua's meta methods, and I found it to be very powerful, you could
> basically implement inheritance in the language, which wasn't necessarily a
> good thing in retrospect.
>
> Is it common for the the argument labels in other languages to be open ended,
> or are labels typically finite? If the answer is finite, why not use a Swift
> method as the wrapper?
> Do you want duck typing, and would it be better to expose this via a protocol?
>
> It seems like in almost every case you could do something like this:
>
> func myMethod<X: PythonConvertible & CanQuack, Y: PythonConvertible>(a: X? =
> nil, b: Y) {
> pythonBridge.call("myMethod", arguments: ["a": X, "b": Y])
> }
>
> It might be good to add some use-cases (a popular Python library perhaps) to
> the proposal where this type of bridge would be insufficient :).
>
> It seems like this proposal pushes the responsibility of Swifty-ness and
> type-safety to the caller. At some point you'll have to write a type-safe
> bridging layer, or write your entire program in non-Swifty code ("The most
> obvious way to write code should also behave in a safe manner"). Is the main
> goal to lower the barrier to Python and other dynamic languages? or is it to
> provide a cheap nice Swifty bridge? I have the above concerns about the
> latter.
>
> Alternative sugar
>
> Ruby has Keyword Arguments for similar sugar:
>
> def foo(regular, hash={})
> puts "the hash: #{hash}"
>
>
> I'm sure you're aware of it, but I'll explain for completeness, any trailing
> argument labels are stored and passed as a hash:
>
> foo(regular, bar: "hello", bas: 123) # outputs 'the hash: [bar: "hello", bas:
> 123]'
> Have you considered an alternative like this? For example:
>
> func myMethod(regular: Int, store: @argcapture [String: PythonConvertible])
> -> PythonConvertible
>
> I'm sure you have good reasons, it might make the implementation bleed out
> into other parts of the codebase. It would be good to include it in the
> proposal alternatives section though. At the moment most of the
> "alternatives" in the proposal just seem to be extensions to the same
> solution :)
>
> Clarity
> Perhaps just that things are more clear to me now
>
> If my extrapolation is correct a user will implement a single type that will
> allow a subset of a good portion of another language to be exposed (object
> method and property bridging). I'm guessing that the dynamic member proposal
> you're planning will not work with methods, it will require a property, I
> think this helps explain some of the motivations. It might be nice to have a
> more complete example that includes dynamic members. I didn't find it clear
> from the proposal that it would only be necessary to implement this protocol
> once per language.
>
> Thanks for considering my questions,
> Andrew Bennett
>
>
>> On Sat, Nov 11, 2017 at 4:37 AM, Chris Lattner via swift-evolution
>> <[email protected]> wrote:
>> Hello all,
>>
>> I have a couple of proposals cooking in a quest to make Swift interoperate
>> with dynamically typed languages like Python better. Instead of baking in
>> hard coded support for one language or the other, I’m preferring to add a
>> few small but general purpose capabilities to Swift. This is the first,
>> which allows a Swift type to become “callable”.
>>
>> The proposal is here:
>> https://gist.github.com/lattner/a6257f425f55fe39fd6ac7a2354d693d
>>
>> I’ve also attached a snapshot below, but it will drift out of date as the
>> proposal is refined. Feedback and thoughts are appreciated, thanks!
>>
>> -Chris
>>
>>
>>
>>
>>
>> Introduce user-defined dynamically "callable" types
>> Proposal: SE-NNNN
>> Author: Chris Lattner
>> Review Manager: TBD
>> Status: Awaiting implementation
>> Introduction
>>
>> This proposal introduces a new DynamicCallable protocol to the standard
>> library. Types that conform to it are "callable" with the function call
>> syntax. It is simple syntactic sugar which allows the user to write:
>>
>> a = someValue(keyword1: 42, "foo", keyword2: 19)
>> and have it be interpreted by the compiler as:
>>
>> a = someValue.dynamicCall(arguments: [
>> ("keyword1", 42), ("", "foo"), ("keyword2", 19)
>> ])
>> Other languages have analogous features (e.g. Python "callables"), but the
>> primary motivation of this proposal is to allow elegant and natural
>> interoperation with dynamic languages in Swift.
>>
>> Swift-evolution thread: Discussion thread topic for that proposal
>>
>> Motivation
>>
>> Swift is well known for being exceptional at interworking with existing C
>> and Objective-C APIs, but its support for calling APIs written in scripting
>> langauges like Python, Perl, and Ruby is quite lacking. These languages
>> provide an extremely dynamic programming model where almost everything is
>> discovered at runtime.
>>
>> Through the introduction of this proposal, and the related
>> DynamicMemberLookupProtocol proposal, we seek to fix this problem. We
>> believe we can make many common APIs feel very natural to use directly from
>> Swift without all the complexity of implementing something like the Clang
>> importer. For example, consider this Python code:
>>
>> class Dog:
>> def __init__(self, name):
>> self.name = name
>> self.tricks = [] # creates a new empty list for each dog
>>
>> def add_trick(self, trick):
>> self.tricks.append(trick)
>> we would like to be able to use this from Swift like this (the comments show
>> the corresponding syntax you would use in Python):
>>
>> // import DogModule
>> // import DogModule.Dog as Dog // an alternate
>> let Dog = Python.import(“DogModule.Dog")
>>
>> // dog = Dog("Brianna")
>> let dog = Dog("Brianna")
>>
>> // dog.add_trick("Roll over")
>> dog.add_trick("Roll over")
>>
>> // dog2 = Dog("Kaylee").add_trick("snore")
>> let dog2 = Dog("Kaylee").add_trick("snore")
>> Of course, this would also apply to standard Python APIs as well. Here is an
>> example working with the Python pickleAPI and the builtin Python function
>> open:
>>
>> // import pickle
>> let pickle = Python.import("pickle")
>>
>> // file = open(filename)
>> let file = Python.open(filename)
>>
>> // blob = file.read()
>> let blob = file.read()
>>
>> // result = pickle.loads(blob)
>> let result = pickle.loads(blob)
>> This can all be expressed today as library functionality written in Swift,
>> but without this proposal, the code required is unnecessarily verbose and
>> gross. Without it (but with the related dynamic member lookup proposal) the
>> code would have a method name (like call) all over the code:
>>
>> // import pickle
>> let pickle = Python.import("pickle") // normal method in Swift, no change.
>>
>> // file = open(filename)
>> let file = Python.open.call(filename)
>>
>> // blob = file.read()
>> let blob = file.read.call()
>>
>> // result = pickle.loads(blob)
>> let result = pickle.loads.call(blob)
>>
>> // dog2 = Dog("Kaylee").add_trick("snore")
>> let dog2 = Dog.call("Kaylee").add_trick.call("snore")
>> While this is a syntactic sugar proposal, we believe that this expands Swift
>> to be usable in important new domains. This sort of capability is also
>> highly precedented in other languages, and is a generally useful language
>> feature that could be used for other purposes as well.
>>
>> Proposed solution
>>
>> We propose introducing this protocol to the standard library:
>>
>> protocol DynamicCallable {
>> associatedtype DynamicCallableArgument
>> associatedtype DynamicCallableResult
>>
>> func dynamicCall(arguments: [(String, DynamicCallableArgument)]) throws ->
>> DynamicCallableResult
>> }
>> It also extends the language such that function call syntax - when applied
>> to a value of DynamicCallable type - is accepted and transformed into a call
>> to the dynamicCall member. The dynamicCall method takes a list of tuples:
>> the first element is the keyword label (or an empty string if absent) and
>> the second value is the formal parameter specified at the call site.
>>
>> Before this proposal, the Swift language has two types that participate in
>> call syntax: functions and metatypes (for initialization). Neither of those
>> may conform to protocols at the moment, so this introduces no possible
>> ambiguity into the language.
>>
>> It is worth noting that this does not introduce the ability to provide
>> dynamicly callable static/class members. We don't believe that this is
>> important given the goal of supporting dynamic languages like Python, but if
>> there is a usecase discovered in the future, it could be explored as future
>> work. Such future work should keep in mind that call syntax on metatypes is
>> already meaningful, and that ambiguity would have to be resolved somehow.
>>
>> Discussion
>>
>> While the signature for dynamicCall is highly general we expect the most
>> common use will be clients who are programming against concrete types that
>> implement this proposal. One very nice aspect of this is that, as a result
>> of Swift's existing subtyping mechanics, implementations of this type can
>> choose whether they can actually throw an error or not. For example,
>> consider this silly implementation:
>>
>> struct ParameterSummer : DynamicCallable {
>> func dynamicCall(arguments: [(String, Int)]) -> Int {
>> return arguments.reduce(0) { $0+$1.1 }
>> }
>> }
>>
>> let x = ParameterSummer()
>> print(x(1, 7, 12)) // prints 20
>> Because ParameterSummer's implementation of dynamicCall does not throw, the
>> call site is known not to throw either, so the print doesn't need to be
>> marked with try.
>>
>> Example Usage
>>
>> A more realistic (and motivating) example comes from a prototype Python
>> interop layer. While the concrete details of this use case are subject to
>> change and not important for this proposal, it is perhaps useful to have a
>> concrete example to see how this comes together.
>>
>> That prototype currently has two types which model Python values, one of
>> which handles Python exceptions and one of which does not. Their
>> conformances would look like this, enabling the use cases described in the
>> Motivation section above:
>>
>> extension ThrowingPyRef: DynamicCallable {
>> func dynamicCall(arguments: [(String, PythonConvertible)]) throws
>> -> PythonConvertible {
>> // Make sure state errors are not around.
>> assert(PyErr_Occurred() == nil, "Python threw an error but wasn't
>> handled")
>>
>> // Count how many keyword arguments are in the list.
>> let numKeywords = arguments.reduce(0) {
>> $0 + ($1.0.isEmpty ? 0 : 1)
>> }
>>
>> let kwdict = numKeywords != 0 ? PyDict_New() : nil
>>
>> // Non-keyword arguments are passed as a tuple of values.
>> let argTuple = PyTuple_New(arguments.count-numKeywords)!
>> var nonKeywordIndex = 0
>> for (keyword, argValue) in arguments {
>> if keyword.isEmpty {
>> PyTuple_SetItem(argTuple, nonKeywordIndex, argValue.toPython())
>> nonKeywordIndex += 1
>> } else {
>> PyDict_SetItem(kwdict!, keyword.toPython(), argValue.toPython())
>> }
>> }
>>
>> // Python calls always return a non-null value when successful. If the
>> // Python function produces the equivalent of C "void", it returns the
>> None
>> // value. A null result of PyObjectCall happens when there is an error,
>> // like 'self' not being a Python callable.
>> guard let resultPtr = PyObject_Call(state, argTuple, kwdict) else {
>> throw PythonError.invalidCall(self)
>> }
>>
>> let result = PyRef(owned: resultPtr)
>>
>> // Translate a Python exception into a Swift error if one was thrown.
>> if let exception = PyErr_Occurred() {
>> PyErr_Clear()
>> throw PythonError.exception(PyRef(borrowed: exception))
>> }
>>
>> return result
>> }
>> }
>>
>> extension PyRef: DynamicCallable {
>> func dynamicCall(arguments: [(String, PythonConvertible)])
>> -> PythonConvertible {
>> // Same as above, but internally aborts instead of throwing Swift
>> // errors.
>> }
>> }
>> Source compatibility
>>
>> This is a strictly additive proposal with no source breaking changes.
>>
>> Effect on ABI stability
>>
>> This is a strictly additive proposal with no ABI breaking changes.
>>
>> Effect on API resilience
>>
>> This has no impact on API resilience which is not already captured by other
>> language features.
>>
>> Alternatives considered
>>
>> A few alternatives were considered:
>>
>> Add ability to reject parameter labels
>>
>> The implementation above does not allow an implementation to staticly reject
>> argument labels. If this was important to add, we could add another protocol
>> to model this, along the lines of:
>>
>> /// A type conforming just to this protocol would not accept parameter
>> /// labels in its calls.
>> protocol DynamicCallable {
>> associatedtype DynamicCallableArgument
>> associatedtype DynamicCallableResult
>>
>> func dynamicCall(arguments: [DynamicCallableArgument]) throws ->
>> DynamicCallableResult
>> }
>>
>> /// A type conforming to this protocol does allow optional parameter
>> /// labels.
>> protocol DynamicCallableWithKeywordsToo : DynamicCallable {
>> func dynamicCall(arguments: [(String, DynamicCallableArgument)]) throws ->
>> DynamicCallableResult
>> }
>> This would allow a type to implement one or the other based on their
>> capabilities. This proposal is going with a very simple design, but if there
>> is demand for this, the author is happy to switch.
>>
>> Staticly checking for exact signatures
>>
>> This protocol does not allow a type to specify an exact signature for the
>> callable - a specific number of parameters with specific types. If we went
>> down that route, the best approach would be to introduce a new declaration
>> kind (which would end up being very similar to get-only subscripts) since,
>> in general, a type could want multiple concrete callable signatures, and
>> those signatures should participate in overload resolution.
>>
>> While such a feature could be interesting for some use cases, it is almost
>> entirely orthogonal from this proposal: it addresses different use cases and
>> does not solve the needs of this proposal. It does not address our needs
>> because even a variadic callable declaration would not provide access to the
>> keyword argument labels we need.
>>
>> Direct language support for Python
>>
>> We considered implementing something analogous to the Clang importer for
>> Python, which would add a first class Python specific type(s) to Swift
>> language or standard library. We rejected this option because it would be
>> significantly more invasive in the compiler, would set the precedent for all
>> other dynamic languages to get first class language support, and because
>> that first class support doesn't substantially improve the experience of
>> working with Python over existing Swift with a couple small "generally
>> useful" extensions like this one.
>>
>> Naming
>>
>> The most fertile ground for bikeshedding is the naming of the protocol and
>> the members. We welcome other ideas and suggestions for naming, but here are
>> some thoughts on obvious options to consider:
>>
>> We considered but rejected the name CustomCallable, because the existing
>> Custom* protocols in the standard library (CustomStringConvertible,
>> CustomReflectable, etc) provide a way to override and custom existing
>> builtin abilities of Swift. In contrast, this feature grants a new
>> capability to a type.
>>
>> We considered but rejected a name like ExpressibleByCalling to fit with the
>> ExpressibleBy* family of protocols (like ExpressibleByFloatLiteral,
>> ExpressibleByStringLiteral, etc). This name family is specifically used by
>> literal syntax, and calls are not literals. Additionally the type itself is
>> not "expressible by calling" - instead, instances of the type may be called.
>>
>> On member and associated type naming, we intentionally gave these long and
>> verbose names so they stay out of the way of user code completion. The
>> members of this protocol are really just compiler interoperability glue. If
>> there was a Swift attribute to disable the members from showing up in code
>> completion, we would use it (such an attribute would also be useful for the
>> LiteralConvertible and other compiler magic protocols).
>>
>>
>>
>> _______________________________________________
>> swift-evolution mailing list
>> [email protected]
>> https://lists.swift.org/mailman/listinfo/swift-evolution
>>
>
> _______________________________________________
> swift-evolution mailing list
> [email protected]
> https://lists.swift.org/mailman/listinfo/swift-evolution
_______________________________________________
swift-evolution mailing list
[email protected]
https://lists.swift.org/mailman/listinfo/swift-evolution
