tcguanshuhuai opened a new issue, #10102:
URL: https://github.com/apache/iceberg/issues/10102
### Apache Iceberg version
1.5.0 (latest release)
### Query engine
Spark
### Please describe the bug 🐞
Step1: start iceberg
spark-sql --jars /root/myjars/iceberg-spark-runtime-3.3_2.12-1.5.0.jar \
--conf
spark.sql.extensions=org.apache.iceberg.spark.extensions.IcebergSparkSessionExtensions
\
--conf
spark.sql.catalog.spark_catalog=org.apache.iceberg.spark.SparkSessionCatalog \
--conf spark.sql.catalog.spark_catalog.type=hive \
--conf spark.sql.catalog.hive_prod=org.apache.iceberg.spark.SparkCatalog
\
--conf spark.sql.catalog.hive_prod.type=hive \
--conf
spark.sql.catalog.hadoop_prod=org.apache.iceberg.spark.SparkCatalog \
--conf spark.sql.catalog.hadoop_prod.type=hadoop \
--conf
spark.sql.catalog.hadoop_prod.warehouse=hdfs://ns:8020/user/iceberg/warehouse/hadoop_prod
Step2: create a iceberg table
CREATE TABLE hadoop_prod.iceberg_db.sample_hidden_partition (
id bigint,
data string,
ts timestamp)
USING iceberg
PARTITIONED BY (days(ts));
Step3: insert into a record
//Timestamp: 1718899200 is 2024-06-21 00:00:00
insert into table hadoop_prod.iceberg_db.sample_hidden_partition values(2,
'2021-01-02', cast(1718899200 as timestamp) );
We can find use select sql I can get the correct value:
spark-sql (default)> select * from
hadoop_prod.iceberg_db.sample_hidden_partition;
id data ts
2 2021-01-02 2024-06-21 00:00:00
However, the hadoop url is wrong:
spark-sql (default)> dfs -ls
/user/iceberg/warehouse/hadoop_prod/iceberg_db/sample_hidden_partition/data;
**Expect result:**
drwxr-xr-x - root supergroup 0 2024-04-08 14:36
/user/iceberg/warehouse/hadoop_prod/iceberg_db/sample_hidden_partition/data/ts_day=2024-06-21
**Actual result:**
drwxr-xr-x - root supergroup 0 2024-04-08 14:36
/user/iceberg/warehouse/hadoop_prod/iceberg_db/sample_hidden_partition/data/ts_day=2024-06-20
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org
For additional commands, e-mail: issues-h...@iceberg.apache.org
