huaxingao commented on code in PR #14297:
URL: https://github.com/apache/iceberg/pull/14297#discussion_r3185274222
##########
spark/v4.1/spark/src/main/java/org/apache/iceberg/spark/SparkSQLProperties.java:
##########
@@ -114,4 +114,12 @@ private SparkSQLProperties() {}
public static final String ASYNC_MICRO_BATCH_PLANNING_ENABLED =
"spark.sql.iceberg.async-micro-batch-planning-enabled";
public static final boolean ASYNC_MICRO_BATCH_PLANNING_ENABLED_DEFAULT =
false;
+
+ // Controls whether to shred variant columns during write operations
+ public static final String SHRED_VARIANTS =
"spark.sql.iceberg.shred-variants";
+
+ // Controls the buffer size for variant schema inference during writes
+ // This determines how many rows are buffered before inferring shredded
schema
+ public static final String VARIANT_INFERENCE_BUFFER_SIZE =
+ "spark.sql.iceberg.variant.inference.buffer-size";
Review Comment:
nit: `spark.sql.iceberg.variant-inference-buffer-size`?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
