nmusset opened a new issue, #288:
URL: https://github.com/apache/pulsar-dotpulsar/issues/288
### Description
DotPulsar is not compatible with .NET Native AOT publishing. Attempting to
use it in a project built with `<PublishAot>true</PublishAot>` produces
multiple trim and dynamic-code warnings at build time, and the produced binary
may crash at runtime when compression or schema features are exercised.
The root causes fall into three areas:
1. **Compression plugins** — all five compression backends
(`BuiltinZlibCompression`, `Lz4Compression`, `SnappyCompression`,
`ZstdCompression`, `ZstdSharpCompression`) discover and invoke the compression
libraries entirely through `Assembly.Load()` + runtime reflection. These
patterns are fundamentally incompatible with AOT and trimming.
2. **Avro schemas** — `AvroISpecificRecordSchema<T>` and
`AvroGenericRecordSchema` also use `Assembly.Load("Avro")`, `DefinedTypes`,
`MakeGenericType()`, and `Activator.CreateInstance()` to bind to the Apache
Avro library at runtime.
3. **JSON schema** — `JsonSchemaDefinitionGenerator` inspects types via
`GetProperties()` / `GetCustomAttribute()` / `Enum.GetNames()` at runtime, and
`JsonSchema<T>` calls `JsonSerializer.Deserialize<T>` / `SerializeToUtf8Bytes`
without a source-generated `JsonSerializerContext`, which is trim-unsafe.
The library also declares no AOT-compatibility stance (`<IsAotCompatible>`
is absent from the `.csproj`) and carries no `[RequiresDynamicCode]` /
`[RequiresUnreferencedCode]` annotations to alert consumers.
### Reproduction Steps
1. Create a new .NET 8/9/10 console application.
2. Reference DotPulsar (latest, 5.3.1 at time of writing).
3. Add `<PublishAot>true</PublishAot>` to the project file.
4. Call any API that touches compression or a JSON/Avro schema (or simply
add `using DotPulsar;`).
5. Publish: `dotnet publish -r linux-x64 -c Release`
Build output (verified with `dotnet publish -r win-x64 -c Release` against
DotPulsar 5.3.1 / .NET 9 SDK) includes **39 warnings** — a representative
sample:
```
AvroISpecificRecordSchema.cs(114): Trim analysis warning IL2026:
...Assembly.DefinedTypes.get which has 'RequiresUnreferencedCodeAttribute'...
AvroISpecificRecordSchema.cs(136): AOT analysis warning IL3050:
...System.Type.MakeGenericType(Type[]) which has
'RequiresDynamicCodeAttribute'...
AvroISpecificRecordSchema.cs(136): Trim analysis warning IL2055: Call to
'System.Type.MakeGenericType(Type[])' can not be statically analyzed...
BuiltinZlibCompression.cs(28): Trim analysis warning IL2026:
...Assembly.GetTypes() which has 'RequiresUnreferencedCodeAttribute'...
BuiltinZlibCompression.cs(35): AOT analysis warning IL3050:
...System.Type.GetEnumValues() which has 'RequiresDynamicCodeAttribute'...
Lz4Compression.cs(34): Trim analysis warning IL2026:
...Assembly.DefinedTypes.get which has 'RequiresUnreferencedCodeAttribute'...
SnappyCompression.cs(32): Trim analysis warning IL2026:
...Assembly.DefinedTypes.get which has 'RequiresUnreferencedCodeAttribute'...
ZstdCompression.cs(33): Trim analysis warning IL2026:
...Assembly.DefinedTypes.get which has 'RequiresUnreferencedCodeAttribute'...
ZstdSharpCompression.cs(33): Trim analysis warning IL2026:
...Assembly.DefinedTypes.get which has 'RequiresUnreferencedCodeAttribute'...
JsonSchemaDefinitionGenerator.cs(53): Trim analysis warning IL2070:
...'this' argument does not satisfy 'PublicProperties' in call to
'System.Type.GetProperties(BindingFlags)'...
```
The full list is in the **Other information** section below.
At runtime (depending on trimming aggressiveness), the compressor factory
initialization will throw `FileNotFoundException` (trimmed assembly) or
`MissingMethodException`.
### Expected behavior
The library should either:
- Be fully AOT-compatible for the core messaging path (at minimum) and
annotate any optional features (Avro, dynamic compression loading) with
`[RequiresDynamicCode]` / `[RequiresUnreferencedCode]` so the compiler can warn
consumers; **or**
- Declare `<IsAotCompatible>false</IsAotCompatible>` in the `.csproj` and
document the limitation.
Ideally the core path (connect, produce, consume with `byte[]` / primitive
schemas) would be AOT-safe, with opt-in AOT-unsafe features clearly annotated.
### Actual behavior
All five compression loaders use `Assembly.Load()` and extensive reflection
with no annotations. The Avro schema implementations do the same. The JSON
schema implementation uses un-annotated `System.Text.Json` reflection APIs. The
project carries zero `[RequiresDynamicCode]` / `[RequiresUnreferencedCode]`
attributes and no `<IsAotCompatible>` marker.
### Regression?
Not a regression — AOT compatibility has not been declared or targeted in
any prior release.
### Known Workarounds
- Build without `<PublishAot>` and instead use `<PublishSingleFile>` with
`<EnableCompressionInSingleFile>true</EnableCompressionInSingleFile>` (dynamic
loading still works).
- Suppress individual warnings with `<NoWarn>` / `<TrimmerRootDescriptor>`
in the consuming project (fragile, requires intimate knowledge of the library
internals).
- For the JSON schema specifically, pass a pre-built `jsonSchemaDefinition`
string to `JsonSchema<T>(options, jsonSchemaDefinition)` and supply a
source-generated `JsonSerializerContext` via `JsonSerializerOptions`.
### Configuration
- DotPulsar version: 5.3.1
- .NET SDK: 9.0 / 10.0 (both affected)
- OS: Windows 11, Linux x64 (both affected)
- Architecture: x64, ARM64
### Other information
### Detailed breakdown of AOT-incompatible code
#### 1. Compression — `Assembly.Load()` + runtime reflection
All five files under `src/DotPulsar/Internal/Compression/` follow the same
pattern: load an optional assembly at runtime, enumerate its types by full
name, and create delegates or instances via `Activator.CreateInstance` /
`MethodInfo.CreateDelegate`. None of these patterns survive trimming.
| File | Key problematic APIs |
|------|----------------------|
| `BuiltinZlibCompression.cs:27` | `Assembly.Load("System.IO.Compression")`,
`assembly.GetTypes()`, `Activator.CreateInstance(zlibStreamType!, ...)` (lines
60, 71) |
| `Lz4Compression.cs:32` | `Assembly.Load("K4os.Compression.LZ4")`,
`assembly.DefinedTypes.ToArray()`, `lz4Codec.GetMethods(BindingFlags.Public \|
BindingFlags.Static)`, `method.CreateDelegate(...)` |
| `SnappyCompression.cs:30` | `Assembly.Load("IronSnappy")`,
`snappy.GetMethods(BindingFlags.Public \| BindingFlags.Static)`,
`method.CreateDelegate(...)` |
| `ZstdCompression.cs:31` | `Assembly.Load("ZstdNet")`,
`GetMethods(BindingFlags.Public \| BindingFlags.Instance)`,
`Activator.CreateInstance(...)`, `wrapMethod.CreateDelegate(...)` |
| `ZstdSharpCompression.cs:31` | `Assembly.Load("ZstdSharp")`, same pattern
as above |
**Suggested fix:** Take compile-time dependencies on the optional
compression packages instead of loading them dynamically. Gate each
implementation behind a `#if` / multi-targeting or expose a registration API
(`builder.UseSnappy()`) that accepts a concrete `ICompressorFactory` /
`IDecompressorFactory` supplied by the consumer. This is the pattern used by
libraries like `Microsoft.Extensions.Caching.StackExchangeRedis`.
#### 2. Avro schemas — `Assembly.Load()` + `MakeGenericType` +
`Activator.CreateInstance`
`AvroISpecificRecordSchema<T>` (lines 54, 57, 59, 71–72, 113–114, 119–120,
136, 149, 156, 159) and `AvroGenericRecordSchema` use an identical late-binding
strategy against the `Apache.Avro` NuGet package.
Notable calls:
- `Assembly.Load("Avro")` — the assembly is trimmed away unless explicitly
preserved
- `assembly.DefinedTypes.ToArray()` — all types must survive trimming for
this to work
- `type.MakeGenericType(typeof(T))` — `[RequiresDynamicCode]`
- `Activator.CreateInstance(_avroWriterTypeInfo, _avroSchema)` —
`[RequiresUnreferencedCode]`
- `MethodInfo.Invoke(...)` in `Decode` / `Encode` hot paths — defeats
inlining and is trim-unsafe
**Suggested fix:** Take a direct compile-time reference to `Apache.Avro` and
use the concrete `SpecificDatumWriter<T>` / `SpecificDatumReader<T>` types. If
the dependency must remain optional, annotate both schema classes with
`[RequiresDynamicCode("Avro schema binding uses runtime reflection.")]` and
`[RequiresUnreferencedCode(...)]`.
#### 3. `JsonSchemaDefinitionGenerator` — runtime type inspection
`src/DotPulsar/Schemas/JsonSchemaDefinitionGenerator.cs` calls
`GetProperties(BindingFlags.Public | BindingFlags.Instance)` (line 53),
`GetCustomAttribute<JsonIgnoreAttribute>()` (line 58),
`GetCustomAttribute<JsonPropertyNameAttribute>()` (line 81),
`Enum.GetNames(type)` (line 135), `type.GetElementType()` (line 154),
`type.GetGenericTypeDefinition()` (line 161), and `type.GetGenericArguments()`
(lines 170, 178) — all trim-unsafe when the concrete `T` is not statically
known.
**Suggested fix:** Replace with a `System.Text.Json` source-generated schema
approach (available in .NET 9 via `JsonSchemaExporter` in `System.Text.Json`)
or accept an explicit schema string and deprecate the reflection-based
generator. Short-term: annotate `Generate(Type, ...)` with
`[RequiresUnreferencedCode]`.
#### 4. `JsonSchema<T>` — trim-unsafe `JsonSerializer` calls
`src/DotPulsar/Schemas/JsonSchema.cs` calls
`JsonSerializer.Deserialize<T>(array, _options)` (line 59) and
`JsonSerializer.SerializeToUtf8Bytes(message, _options)` (line 70). Both
overloads without a `JsonTypeInfo<T>` parameter are decorated with
`[RequiresUnreferencedCode]` in the BCL.
**Suggested fix:** Add an overload that accepts `JsonTypeInfo<T>` (or a
`JsonSerializerContext`) and route serialization through it. Mark the existing
reflection-based constructors with `[RequiresUnreferencedCode]`.
#### 5. No `<IsAotCompatible>` declaration
The absence of `<IsAotCompatible>true</IsAotCompatible>` in
`DotPulsar.csproj` means the .NET SDK trim analyzer is **not** run during
normal builds of the library itself. Adding
`<IsAotCompatible>true</IsAotCompatible>` (even as a goal/aspiration) would
surface all of the above as build-time errors in the library's own CI, making
it impossible to accidentally re-introduce these patterns.
---
### Full verified warning list
Obtained by running `dotnet publish -r win-x64 -c Release` against a minimal
consumer project referencing DotPulsar 5.3.1 source (the native link step fails
because MSVC is not on `PATH`, but all ILLink/AOT analyzer warnings are emitted
before that step).
**39 warnings across 8 source files:**
| # | File | Line | Warning | Description |
|---|------|------|---------|-------------|
| 1 | `Schemas/AvroISpecificRecordSchema.cs` | 54 | IL2080 |
`GetInterfaces()` — field `_typeT` not annotated with
`DynamicallyAccessedMemberTypes.Interfaces` |
| 2 | `Schemas/AvroISpecificRecordSchema.cs` | 57 | IL2080 | `GetField()` —
field `_typeT` not annotated with `PublicFields` |
| 3 | `Schemas/AvroISpecificRecordSchema.cs` | 71 | IL2075 | `GetProperty()`
— return of `GetType()` not annotated with `PublicProperties` |
| 4 | `Schemas/AvroISpecificRecordSchema.cs` | 72 | IL2075 | `GetMethod()` —
return of `GetType()` not annotated with `PublicMethods` |
| 5 | `Schemas/AvroISpecificRecordSchema.cs` | 114 | IL2026 |
`Assembly.DefinedTypes` has `[RequiresUnreferencedCode]` — types may be removed
|
| 6 | `Schemas/AvroISpecificRecordSchema.cs` | 119 | IL2065 | `GetMethods()`
— `this` value not statically determinable |
| 7 | `Schemas/AvroISpecificRecordSchema.cs` | 120 | IL2065 | `GetMethods()`
— same as above (second call) |
| 8 | `Schemas/AvroISpecificRecordSchema.cs` | 136 | IL3050 |
`MakeGenericType()` has `[RequiresDynamicCode]` — native code for instantiation
may be absent |
| 9 | `Schemas/AvroISpecificRecordSchema.cs` | 136 | IL2055 |
`MakeGenericType()` — call cannot be statically analyzed |
| 10 | `Schemas/AvroISpecificRecordSchema.cs` | 149 | IL3050 |
`MakeGenericType()` — same (reader type) |
| 11 | `Schemas/AvroISpecificRecordSchema.cs` | 149 | IL2055 |
`MakeGenericType()` — same (reader type) |
| 12 | `Schemas/AvroISpecificRecordSchema.cs` | 156 | IL2077 |
`Activator.CreateInstance()` — field `_avroWriterTypeInfo` not annotated with
`PublicConstructors` |
| 13 | `Schemas/AvroISpecificRecordSchema.cs` | 159 | IL2077 |
`Activator.CreateInstance()` — field `_avroReaderTypeInfo` not annotated with
`PublicConstructors` |
| 14 | `Internal/Compression/BuiltinZlibCompression.cs` | 28 | IL2026 |
`Assembly.GetTypes()` has `[RequiresUnreferencedCode]` |
| 15 | `Internal/Compression/BuiltinZlibCompression.cs` | 35 | IL3050 |
`Type.GetEnumValues()` has `[RequiresDynamicCode]` |
| 16 | `Internal/Compression/BuiltinZlibCompression.cs` | 37 | IL3050 |
`Type.GetEnumValues()` — same (second enum type) |
| 17 | `Internal/Compression/BuiltinZlibCompression.cs` | 60 | IL2067 |
`Activator.CreateInstance()` — parameter `zlibStreamType` not annotated with
`PublicConstructors` |
| 18 | `Internal/Compression/BuiltinZlibCompression.cs` | 71 | IL2067 |
`Activator.CreateInstance()` — same (decompressor lambda) |
| 19 | `Internal/Compression/Lz4Compression.cs` | 34 | IL2026 |
`Assembly.DefinedTypes` has `[RequiresUnreferencedCode]` |
| 20 | `Internal/Compression/Lz4Compression.cs` | 39 | IL2075 |
`GetMethods()` — return of `FindLZ4Codec()` not annotated with `PublicMethods` |
| 21 | `Internal/Compression/SnappyCompression.cs` | 32 | IL2026 |
`Assembly.DefinedTypes` has `[RequiresUnreferencedCode]` |
| 22 | `Internal/Compression/SnappyCompression.cs` | 36 | IL2075 |
`GetMethods()` — return of `FindSnappy()` not annotated with `PublicMethods` |
| 23 | `Internal/Compression/ZstdCompression.cs` | 33 | IL2026 |
`Assembly.DefinedTypes` has `[RequiresUnreferencedCode]` |
| 24 | `Internal/Compression/ZstdCompression.cs` | 36 | IL2075 |
`GetMethods()` — return of `Find()` not annotated with `PublicMethods` |
| 25 | `Internal/Compression/ZstdCompression.cs` | 40 | IL2075 |
`GetMethods()` — return of `Find()` not annotated with `PublicMethods` (second
call) |
| 26 | `Internal/Compression/ZstdCompression.cs` | 45 | IL2072 |
`Activator.CreateInstance()` — return of `Find()` not annotated with
`PublicParameterlessConstructor` |
| 27 | `Internal/Compression/ZstdCompression.cs` | 55 | IL2072 |
`Activator.CreateInstance()` — same (decompressor lambda) |
| 28 | `Internal/Compression/ZstdCompression.cs` | 83 | IL2075 |
`TypeInfo.ImplementedInterfaces` — enumerator current not annotated with
`Interfaces` |
| 29 | `Internal/Compression/ZstdCompression.cs` | 83 | IL2075 |
`Type.GetConstructor()` — enumerator current not annotated with
`PublicParameterlessConstructor` |
| 30 | `Internal/Compression/ZstdSharpCompression.cs` | 33 | IL2026 |
`Assembly.DefinedTypes` has `[RequiresUnreferencedCode]` |
| 31 | `Internal/Compression/ZstdSharpCompression.cs` | 36 | IL2075 |
`GetMethods()` — return of `FindDecompressor()` not annotated with
`PublicMethods` |
| 32 | `Internal/Compression/ZstdSharpCompression.cs` | 40 | IL2075 |
`GetMethods()` — return of `FindCompressor()` not annotated with
`PublicMethods` |
| 33 | `Internal/Compression/ZstdSharpCompression.cs` | 45 | IL2072 |
`Activator.CreateInstance()` — return of `FindCompressor()` not annotated with
`PublicConstructors` |
| 34 | `Internal/Compression/ZstdSharpCompression.cs` | 55 | IL2072 |
`Activator.CreateInstance()` — return of `FindDecompressor()` not annotated
with `PublicParameterlessConstructor` |
| 35 | `Internal/Compression/ZstdSharpCompression.cs` | 83 | IL2075 |
`TypeInfo.ImplementedInterfaces` — enumerator current not annotated with
`Interfaces` (FindDecompressor) |
| 36 | `Internal/Compression/ZstdSharpCompression.cs` | 83 | IL2075 |
`Type.GetConstructor()` — enumerator current not annotated with
`PublicParameterlessConstructor` (FindDecompressor) |
| 37 | `Internal/Compression/ZstdSharpCompression.cs` | 103 | IL2075 |
`TypeInfo.ImplementedInterfaces` — enumerator current not annotated with
`Interfaces` (FindCompressor) |
| 38 | `Internal/Compression/ZstdSharpCompression.cs` | 103 | IL2075 |
`Type.GetConstructor()` — enumerator current not annotated with
`PublicConstructors` (FindCompressor) |
| 39 | `Schemas/JsonSchemaDefinitionGenerator.cs` | 53 | IL2070 |
`GetProperties(BindingFlags)` — parameter `type` not annotated with
`PublicProperties` |
> **Note:** `AvroGenericRecordSchema` and the `JsonSchema<T>.Decode` /
`Encode` paths were not reachable from the probe's entry point, so their
warnings do not appear in this run. The actual warning count in a real
application touching those code paths will be higher.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
