Hi Cory,
I think we're dealing with two separate issues here.
1) that all forward declared struct pointers get imported as an OpaquePointer
which makes us lose all type-safety
2) that it's a fairly frequent case that C libraries evolve from 'pointers to
fully declared structs' to 'pointers to forward declared structs'
Regarding 1)
------------
I fully agree that is pretty bad and I believe I have an idea what the Swift
importer should do for that case:
For the following C types
struct foo_s;
typedef foo_s *foo;
struct bar_s;
typedef bar_s *bar;
the C compiler today imports both `foo` and `bar` as `OpaquePointer` which
isn't very helpful. Instead I believe the importer should declare to phantom
types
enum foo_s {}
enum bar_s {}
and import `foo` as `typealias foo = OpaquePointer<foo_s>` and `bar` as
`typealias bar = OpaquePointer<bar_s>`.
That seems to preserve the right semantics:
- we can't have any values of `foo_s` or `bar_s` as they're phantom
- we can however have pointers to them
- in this example the pointers are `OpaquePointer<foo_s>` and
`OpaquePointer<bar_s>`, deliberately not `UnsafePointer<foo_s>` and
`UnsafePointer<bar_s>` so we don't have an issue that the user can do
'unsafePtr.pointee'
How do people think about this proposed change?
Regarding 2)
------------
I feel you'd prefer if we'd import the above as `typealias foo =
UnsafePointer<foo_s>` which I would be happy with but I can also see why
`OpaquePointer` exists: It stops us from dereferencing the pointer at compile
time (just like C does).
So yes, I agree we should fix (1). For the time being I believe I have
something that might work for you use-case even today (and would work even
nicer if the above changes are implemented):
Today, we already have the following initialisers
struct UnsafePointer<T> {
init?(_ ptr: OpaquePointer)
}
and
struct OpaquePointer {
init?<T>(_ ptr: UnsafePointer<T>)
}
by just adding the following two trivial ones:
extension UnsafePointer<T> {
init?(_ ptr: UnsafePointer<T>) { return ptr }
}
extension OpaquePointer {
init?<T>(_ ptr: OpaquePointer) { return ptr }
}
this should solve your problem:
- when you receive a pointer from the C library, store it as OpaquePointer
let myOpaquePointer: OpaquePointer =
OpaquePointer(c_library_create_function())
which now should work regardless of whether you're linking the old or the new
version of the C library
- when you pass a pointer to the C library:
c_library_consuming_function(.init(myOpaquePointer))
which should (modulo it doesn't right now
https://bugs.swift.org/browse/SR-6211) select the right initialiser for you,
only it doesn't 😲
But fortunately we can work around it:
--- SNIP ---
extension UnsafePointer {
static func make(_ ptr: UnsafePointer<Pointee>) -> UnsafePointer<Pointee> {
return ptr
}
static func make(_ ptr: OpaquePointer) -> UnsafePointer<Pointee> {
return UnsafePointer(ptr)
}
}
extension OpaquePointer {
static func make<T>(_ ptr: UnsafePointer<T>) -> OpaquePointer {
return OpaquePointer(ptr)!
}
static func make(_ ptr: OpaquePointer) -> OpaquePointer {
return ptr
}
}
func mockCLibraryCreateOld() -> UnsafePointer<Int> {
return UnsafePointer(UnsafeMutablePointer<Int>.allocate(capacity: 1))
}
func mockCLibraryCreateNew() -> OpaquePointer {
return OpaquePointer(mockCLibraryCreateOld())
}
func mockCLibraryConsumeOld(_ x: UnsafePointer<Int>) {}
func mockCLibraryConsumeNew(_ x: OpaquePointer) {}
let fromCold: OpaquePointer = .make(mockCLibraryCreateOld())
let fromCnew: OpaquePointer = .make(mockCLibraryCreateNew())
mockCLibraryConsumeOld(.make(fromCold))
mockCLibraryConsumeNew(.make(fromCnew))
mockCLibraryConsumeOld(.make(fromCnew))
mockCLibraryConsumeNew(.make(fromCold))
--- SNAP ---
HTH
-- Johannes
> On 24 Oct 2017, at 9:14 am, Cory Benfield via swift-evolution
> <[email protected]> wrote:
>
> I wanted to discuss a recent difficulty I’ve encountered while writing a
> Swift program that uses a C library that has recently changed its API to use
> opaque pointers, with an eye towards asking whether there are suggestions for
> ways to tackle the problem that I haven’t considered, or whether some
> enhancement to Swift should be proposed to provide a solution.
>
> A common trend in modern C code is to encapsulate application data by using
> pointers to “opaque” data structures: that is, data structures whose complete
> definition is not available in the header files for the library. This has
> many benefits from the perspective of library developers, mostly notably
> because it limits the ABI of the library, making it easier to change the
> internals without requiring recompilation or breaking changes. Pointers to
> these structures are translated into Swift code in the form of the
> OpaquePointer type.
>
> Older C libraries frequently have non-opaque structures: that is, the
> structure definition is available in the header files for the library. When
> using code like this from Swift, pointers to these structures are translated
> to Unsafe[Mutable]Pointer<T>.
>
> Both of these cases are well-handled by Swift today: opaque pointers
> correctly can do absolutely nothing, whereas typed pointers have the option
> of having behaviour based on knowing about the size of the data structure to
> which they point. All very good.
>
> A problem arises if a C dependency chooses to transition from non-opaque to
> opaque structures. This is a transition that well-maintained C libraries are
> strongly incentivised to make, but if you want to write Swift code that will
> compile against both the old and new version of the library you run into
> substantial issues. To illustrate the issue I’ll construct a small problem
> based on the most widely-used library to recently make this transition,
> OpenSSL.
>
> In OpenSSL 1.1.0 almost all of the previously-open data structures were made
> opaque, including the heavily used SSL_CTX structure. In terms of C code, the
> header file declaration changed from
>
>
> struct ssl_ctx_st {
> const SSL_METHOD *method;
> // snip 250 lines of structure declaration
> }
> typedef struct ssl_ctx_st SSL_CTX;
>
> to
>
> typedef struct ssl_ctx_st SSL_CTX;
>
>
> At an API level, any function that worked on the SSL_CTX structure that
> existed before this change was unaffected. For example, the function
> SSL_CTX_use_certificate has the same C API in both versions:
>
> int SSL_CTX_use_certificate(SSL_CTX *ctx, X509 *x);
>
> Unfortunately, in Swift the API for this function changes dramatically, from
>
> func SSL_CTX_use_certificate(_ ctx: UnsafeMutablePointer<SSL_CTX>!,
> _ x:
> UnsafeMutablePointer<X509>!) -> Int32
>
> to
>
> func SSL_CTX_use_certificate(_ ctx: OpaquePointer!,
> _ x: OpaquePointer!)
> -> Int32
>
> The reason this is problematic is that there is no implicit cast in either
> direction between UnsafeMutablePointer<T> and OpaquePointer. This means the
> API here has changed in an incompatible way: types that are valid before the
> structure was made opaque are not valid afterwards. This adds a pretty
> substantial burden to supporting multiple versions of the same library from
> Swift code.
>
> So far I have thought of the following solutions to this problem that I can
> implement today:
>
> 1. Write a C wrapper library that exposes a third, consistent type that is
> the same on all versions. Most likely this would be done by re-exposing all
> these methods with arguments that take `void *` and performing the cast in C
> code. This, unfortunately, loses some of the Swift compiler’s ability to
> enforce type safety, as all these arguments will now be UnsafeRawPointer.
> This is not any worse than OpaquePointer, but it’s objectively worse than the
> un-opaqued version.
>
> 2. Write a C wrapper library that embeds these pointers in single, non-opaque
> structures with separate types. This allows us to keep the type safety at the
> cost of verbosity and an additional layer of indirection.
>
> 3. Write two different Swift wrappers for each of these versions that expose
> the same outer types, and transform them internally. Not ideal: the
> conditional compilation story here isn’t good and distributing this library
> via Swift PM would be hard.
>
> I’d be really interested in hearing whether there is a solution I’m missing
> that can be implemented today. If there is *not* such a solution, is there
> interest in attempting to tackle this problem in Swift more directly? There
> are plenty of language changes that could be made to solve this solution
> (e.g. changing OpaquePointer to OpaquePointer<T> but making it impossible to
> dereference, then treating UnsafePointer<T> as a subclass of
> OpaquePointer<T>, or making OpaquePointer a protocol implemented by
> UnsafePointer<T>, or all kinds of other things), but I wanted to hear from
> the community about suggested approach.
>
> I’d love not to have to manually maintain a C wrapper just to escape Swift’s
> type system here.
>
> Thanks,
>
> Cory
> _______________________________________________
> swift-evolution mailing list
> [email protected]
> https://lists.swift.org/mailman/listinfo/swift-evolution
_______________________________________________
swift-evolution mailing list
[email protected]
https://lists.swift.org/mailman/listinfo/swift-evolution
