AllenWee1106 opened a new issue, #9188:
URL: https://github.com/apache/iceberg/issues/9188
### Query engine
flink --> icebergsource
### Question
I have a difficult point and I need everyone's assistance, THX!
flink on yarn throw exception:
`java.util.concurrent.CompletionException:
org.apache.flink.client.deployment.application.ApplicationExecutionException:
Could not execute application.
at
java.util.concurrent.CompletableFuture.encodeThrowable(CompletableFuture.java:292)
~[?:1.8.0_392]
at
java.util.concurrent.CompletableFuture.completeThrowable(CompletableFuture.java:308)
~[?:1.8.0_392]
at
java.util.concurrent.CompletableFuture.uniCompose(CompletableFuture.java:957)
~[?:1.8.0_392]
at
java.util.concurrent.CompletableFuture$UniCompose.tryFire(CompletableFuture.java:940)
~[?:1.8.0_392]
at
java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:488)
~[?:1.8.0_392]
at
java.util.concurrent.CompletableFuture.completeExceptionally(CompletableFuture.java:1990)
~[?:1.8.0_392]
at
org.apache.flink.client.deployment.application.ApplicationDispatcherBootstrap.runApplicationEntryPoint(ApplicationDispatcherBootstrap.java:337)
~[recoveryData-1.0-SNAPSHOT.jar:?]
at
org.apache.flink.client.deployment.application.ApplicationDispatcherBootstrap.lambda$runApplicationAsync$2(ApplicationDispatcherBootstrap.java:254)
~[recoveryData-1.0-SNAPSHOT.jar:?]
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
~[?:1.8.0_392]
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
~[?:1.8.0_392]
at
org.apache.flink.runtime.concurrent.akka.ActorSystemScheduledExecutorAdapter$ScheduledFutureTask.run(ActorSystemScheduledExecutorAdapter.java:171)
~[flink-rpc-akka_d5e8496b-09b8-47d8-9224-6b04949f84c9.jar:1.16.2]
at
org.apache.flink.runtime.concurrent.akka.ClassLoadingUtils.runWithContextClassLoader(ClassLoadingUtils.java:68)
~[flink-rpc-akka_d5e8496b-09b8-47d8-9224-6b04949f84c9.jar:1.16.2]
at
org.apache.flink.runtime.concurrent.akka.ClassLoadingUtils.lambda$withContextClassLoader$0(ClassLoadingUtils.java:41)
~[flink-rpc-akka_d5e8496b-09b8-47d8-9224-6b04949f84c9.jar:1.16.2]
at akka.dispatch.TaskInvocation.run(AbstractDispatcher.scala:49)
[flink-rpc-akka_d5e8496b-09b8-47d8-9224-6b04949f84c9.jar:1.16.2]
at
akka.dispatch.ForkJoinExecutorConfigurator$AkkaForkJoinTask.exec(ForkJoinExecutorConfigurator.scala:48)
[flink-rpc-akka_d5e8496b-09b8-47d8-9224-6b04949f84c9.jar:1.16.2]
at java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:289)
[?:1.8.0_392]
at
java.util.concurrent.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1056)
[?:1.8.0_392]
at java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1692)
[?:1.8.0_392]
at
java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:175)
[?:1.8.0_392]
Caused by:
org.apache.flink.client.deployment.application.ApplicationExecutionException:
Could not execute application.
... 13 more
Caused by: org.apache.flink.client.program.ProgramInvocationException: The
main method caused an error: null
at
org.apache.flink.client.program.PackagedProgram.callMainMethod(PackagedProgram.java:372)
~[recoveryData-1.0-SNAPSHOT.jar:?]
at
org.apache.flink.client.program.PackagedProgram.invokeInteractiveModeForExecution(PackagedProgram.java:222)
~[recoveryData-1.0-SNAPSHOT.jar:?]
at
org.apache.flink.client.ClientUtils.executeProgram(ClientUtils.java:98)
~[recoveryData-1.0-SNAPSHOT.jar:?]
at
org.apache.flink.client.deployment.application.ApplicationDispatcherBootstrap.runApplicationEntryPoint(ApplicationDispatcherBootstrap.java:301)
~[recoveryData-1.0-SNAPSHOT.jar:?]
... 12 more
Caused by: java.lang.NullPointerException
at
java.util.concurrent.ConcurrentHashMap.putVal(ConcurrentHashMap.java:1011)
~[?:1.8.0_392]
at
java.util.concurrent.ConcurrentHashMap.put(ConcurrentHashMap.java:1006)
~[?:1.8.0_392]
at
org.apache.iceberg.EnvironmentContext.put(EnvironmentContext.java:53)
~[recoveryData-1.0-SNAPSHOT.jar:?]
at
org.apache.iceberg.flink.FlinkEnvironmentContext.init(FlinkEnvironmentContext.java:29)
~[recoveryData-1.0-SNAPSHOT.jar:?]
at
org.apache.iceberg.flink.TableLoader$CatalogTableLoader.loadTable(TableLoader.java:120)
~[recoveryData-1.0-SNAPSHOT.jar:?]
at
org.apache.iceberg.flink.source.IcebergSource$Builder.build(IcebergSource.java:430)
~[recoveryData-1.0-SNAPSHOT.jar:?]
at
com.uisee.recovery.es.tracking.RecoveryEsDataTracking.main(RecoveryEsDataTracking.java:59)
~[recoveryData-1.0-SNAPSHOT.jar:?]
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
~[?:1.8.0_392]
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
~[?:1.8.0_392]
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
~[?:1.8.0_392]
at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_392]
at
org.apache.flink.client.program.PackagedProgram.callMainMethod(PackagedProgram.java:355)
~[recoveryData-1.0-SNAPSHOT.jar:?]
at
org.apache.flink.client.program.PackagedProgram.invokeInteractiveModeForExecution(PackagedProgram.java:222)
~[recoveryData-1.0-SNAPSHOT.jar:?]
at
org.apache.flink.client.ClientUtils.executeProgram(ClientUtils.java:98)
~[recoveryData-1.0-SNAPSHOT.jar:?]
at
org.apache.flink.client.deployment.application.ApplicationDispatcherBootstrap.runApplicationEntryPoint(ApplicationDispatcherBootstrap.java:301)
~[recoveryData-1.0-SNAPSHOT.jar:?]
... 12 more`
Flink and Iceberg versions as follows:
`<flink.version>1.16.2</flink.version>
<icerberg.version>1.3.1</icerberg.version>
`
tip:
But when I wrote iceberg table, I used version 1.2.1 of iceberg.
so, Is it possible that the error mentioned above was caused by writing data
to the iceberg table using version 1.2.1, but reading it using version 1.3.1
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org
For additional commands, e-mail: issues-h...@iceberg.apache.org
