silentsokolov opened a new issue, #13482:
URL: https://github.com/apache/iceberg/issues/13482
### Apache Iceberg version
1.9.1 (latest release)
### Query engine
Spark
### Please describe the bug 🐞
```
set gcs.oauth2.refresh-credentials-enabled = "true"
```
```python
spark_config = SparkConf().setMaster("local").setAppName("Iceberg")
for k, v in config.items():
spark_config = spark_config.set(k, v)
spark = SparkSession.builder.config(conf=spark_config).getOrCreate()
df = spark.sql("""
select *
from pg.default.table
limit 10
""")
df.writeTo("iceberg.default.table",).using("iceberg").append()
```
```
25/07/07 09:48:26 WARN RewriteDataFilesSparkAction: Failure during rewrite
process for group FileGroupInfo{globalIndex=5, partitionIndex=1,
partition=PartitionData{fact_ts_day=20198, fact_name_bucket=8}}
java.io.NotSerializableException:
org.apache.iceberg.gcp.gcs.OAuth2RefreshCredentialsHandler
Serialization stack:
- object not serializable (class:
org.apache.iceberg.gcp.gcs.OAuth2RefreshCredentialsHandler, value:
org.apache.iceberg.gcp.gcs.OAuth2RefreshCredentialsHandler@40915c3a)
- field (class: org.apache.iceberg.gcp.gcs.GCSFileIO, name:
refreshHandler, type: class
org.apache.iceberg.gcp.gcs.OAuth2RefreshCredentialsHandler)
- object (class org.apache.iceberg.gcp.gcs.GCSFileIO,
org.apache.iceberg.gcp.gcs.GCSFileIO@48827edf)
- writeObject data (class: java.util.concurrent.ConcurrentHashMap)
- object (class java.util.concurrent.ConcurrentHashMap,
{org.apache.iceberg.gcp.gcs.GCSFileIO=org.apache.iceberg.gcp.gcs.GCSFileIO@48827edf})
- field (class: org.apache.iceberg.io.ResolvingFileIO, name:
ioInstances, type: interface java.util.Map)
- object (class org.apache.iceberg.io.ResolvingFileIO,
org.apache.iceberg.io.ResolvingFileIO@19ceaa02)
- field (class: org.apache.iceberg.SerializableTable, name: io,
type: interface org.apache.iceberg.io.FileIO)
- object (class
org.apache.iceberg.spark.source.SerializableTableWithSize,
org.apache.iceberg.spark.source.SerializableTableWithSize@646767c7)
at
org.apache.spark.serializer.SerializationDebugger$.improveException(SerializationDebugger.scala:41)
at
org.apache.spark.serializer.JavaSerializationStream.writeObject(JavaSerializer.scala:49)
at
org.apache.spark.broadcast.TorrentBroadcast$.$anonfun$blockifyObject$4(TorrentBroadcast.scala:365)
at
org.apache.spark.util.SparkErrorUtils.tryWithSafeFinally(SparkErrorUtils.scala:64)
at
org.apache.spark.util.SparkErrorUtils.tryWithSafeFinally$(SparkErrorUtils.scala:61)
at org.apache.spark.util.Utils$.tryWithSafeFinally(Utils.scala:94)
at
org.apache.spark.broadcast.TorrentBroadcast$.blockifyObject(TorrentBroadcast.scala:367)
at
org.apache.spark.broadcast.TorrentBroadcast.writeBlocks(TorrentBroadcast.scala:161)
at
org.apache.spark.broadcast.TorrentBroadcast.<init>(TorrentBroadcast.scala:99)
at
org.apache.spark.broadcast.TorrentBroadcastFactory.newBroadcast(TorrentBroadcastFactory.scala:38)
at
org.apache.spark.broadcast.BroadcastManager.newBroadcast(BroadcastManager.scala:78)
```
### Willingness to contribute
- [ ] I can contribute a fix for this bug independently
- [ ] I would be willing to contribute a fix for this bug with guidance from
the Iceberg community
- [x] I cannot contribute a fix for this bug at this time
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org
For additional commands, e-mail: issues-h...@iceberg.apache.org
