jamesnetherton opened a new issue, #9392:
URL: https://github.com/apache/camel-quarkus/issues/9392
### Bug description
In native mode, every Tika producer start logs:
```
WARN [org.apache.tika.config.ServiceLoader] (main) failed to load service
provider org.apache.tika.ml.chardetect.MojibusterEncodingDetector for interface
org.apache.tika.detect.EncodingDetector: java.lang.RuntimeException:
java.lang.reflect.InvocationTargetException
...
Caused by: java.io.IOException: UTF-16 specialist model resource not found
at /org/apache/tika/ml/chardetect/utf16-specialist.bin. The specialist must be
trained and the model file bundled on the classpath before this detector can be
instantiated.
at
org.apache.tika.ml.chardetect.Utf16SpecialistEncodingDetector.loadModel(Utf16SpecialistEncodingDetector.java:170)
at
org.apache.tika.ml.chardetect.Utf16SpecialistEncodingDetector.<init>(Utf16SpecialistEncodingDetector.java:132)
at
org.apache.tika.ml.chardetect.MojibusterEncodingDetector.<init>(MojibusterEncodingDetector.java:200)
```
Tika 4.1.0's `tika-parser-text-module` depends on
`tika-encoding-detector-mojibuster`, which `camel-tika` pulls in transitively:
```
camel-quarkus-tika
\- org.apache.camel:camel-tika:jar:4.23.0
\- org.apache.tika:tika-parser-text-module:jar:4.1.0
\- org.apache.tika:tika-encoding-detector-mojibuster:jar:4.1.0
```
`TikaProcessor.registerTikaServices` registers every `EncodingDetector`
provider found in `META-INF/services`, so `MojibusterEncodingDetector` is
instantiated in the native image. Its constructor loads two model files via
`Class.getResourceAsStream`:
- `org/apache/tika/ml/chardetect/nb-bigram.bin` (~680 KB)
- `org/apache/tika/ml/chardetect/utf16-specialist.bin`
Neither is registered as a native image resource, so the lookup returns
`null`, the constructor throws and Tika's service loader skips the detector
with the warning above.
The warning is visible in the current `main` native CI runs of the `tika`
and `langchain4j-ingest` integration tests.
### Impact
Text extraction still works because Tika falls back to its other encoding
detectors, so the tests pass. However charset detection for non-UTF-8 documents
can differ between JVM and native mode, and the log is noisy (one stack trace
per Tika producer start).
### Suggested fix
In `TikaProcessor`, when
`org.apache.tika.ml.chardetect.MojibusterEncodingDetector` is present at
runtime, produce a `NativeImageResourceDirectoryBuildItem` for
`org/apache/tika/ml/chardetect` (or `NativeImageResourceBuildItem`s for the two
`.bin` files). The models are read with `DataInputStream` /
`org.apache.tika.ml.LinearModel.load`, not Java serialization, so no further
native configuration should be needed.
_Claude Code on behalf of James Netherton_
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
