pawel-big-lebowski commented on issue #7879:
URL: https://github.com/apache/iceberg/issues/7879#issuecomment-1611303688
Hi @nastra @stevenzwu , I think I found the reason. So, for some reason we
were creating Uber-jar with shadow plugin which contained Flink libraries
including `org.apache.flink:flink-streaming-java`. This was not necessary and
we can get rid of it and this solves the problem on our side.
So the scenario is:
* Someone tries to create Uber-jar containing
`org.apache.flink:flink-streaming-java`,
* While using gradle's `shadowJar` plugin, manifests of dependencies get
overwritten,
* The code
```
DataStream.class.getPackage().getImplementationVersion();
```
returns `null` and `NullPointerException` is thrown when doing
`EnvironmentContext.put("engine-version", FlinkPackage.version());`
The bug is nasty to understand because one does have `DataStream.class` on
the classpath but the manifest with `implementationVersion` is missing. And the
only thing observed on the top is `NullPointerException`.
So, if you don't mind, I could prepare a PR which prevents putting `null` in
`org.apache.iceberg.flink.FlinkEnvironmentContext#init` and throws more
meaningful Exception. It would have been even better if you were able to find
flink version some other way. Alternatively, we can close the issue if it so
rare and uncommon to happen elsewhere.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
