[I] Spark SQL DESCRIBE not showing proper schema on a branch [iceberg]

[via GitHub]

cccs-eric opened a new issue, #9026:
URL: https://github.com/apache/iceberg/issues/9026

   ### Apache Iceberg version
   
   1.3.1
   
   ### Query engine
   
   Spark
   
   ### Please describe the bug 🐞
   
   This started with a 
[question](https://apache-iceberg.slack.com/archives/C025PH0G1D4/p1699543073213439)
 on Slack and it was mentioned that this could be a bug with `DESCRIBE` not 
being "branch-aware".
   
   The use-case is I have a table where I'd like to use the [W-A-P 
workflow](https://apache-iceberg.slack.com/archives/C025PH0G1D4/p1699543073213439),
 but instead of doing a simple `INSERT INTO`, I'd like to add a new column.  
Here are the SQL statements used:
   ```SQL
   CREATE TABLE my_namespace.wap_table
       using iceberg
       tblproperties ('write.parquet.compression-codec' = 'zstd',
                      'write.wap.enabled'='true'
       )
         AS
           SELECT uuid() AS table_id,
                  "First row" AS value
   
   ALTER TABLE my_namespace.wap_table CREATE branch wap_branch
   ALTER TABLE my_namespace.wap_table SET TBLPROPERTIES 
('write.wap.enabled'='true')
   
   SET spark.wap.branch = wap_branch
   
   ALTER TABLE my_namespace.wap_table ADD COLUMN nop string
   ```
   
   From my understanding, the new column `nop` should only be visible from the 
`wab_branch` branch and not from `main`.
   
   But both blocks of SQL will show the `nop` column:
   ```SQL
   SET spark.wap.branch = wap_branch
   DESCRIBE my_namespace.wap_table
   ```
   ```SQL
   SET spark.wap.branch = main
   DESCRIBE my_namespace.wap_table
   ```
   both will return:  
   | col_name   | data_type     | comment
   |----------------|----------------|-------------
   |table_id    |string |None
   |value       |string |None
   |nop |string |None
   
   Is there a problem with my syntax (issuing DDL statements on a branch), or 
is there a problem with the Spark implementation?
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org
For additional commands, e-mail: issues-h...@iceberg.apache.org