GitHub user shashihoushengqia edited a discussion: When using "Enable Authorization and Dynamic Config Provider", an unexpected output occurred.
The problems encountered: I obtained the iceberg catalog configuration from the gravitino server through the "Dynamic Config Provider". The "show tables" command produced inconsistent outputs when "Authorization" was enabled and when it was not enabled. Reproduction steps： 1. Start the gravitino server with the following configuration file： ``` # THE CONFIGURATION FOR Gravitino SERVER gravitino.server.shutdown.timeout = 3000 # THE CONFIGURATION FOR Gravitino WEB SERVER gravitino.server.webserver.host = 0.0.0.0 gravitino.server.webserver.httpPort = 8090 gravitino.server.webserver.minThreads = 24 gravitino.server.webserver.maxThreads = 200 gravitino.server.webserver.stopTimeout = 30000 gravitino.server.webserver.idleTimeout = 30000 gravitino.server.webserver.threadPoolWorkQueueSize = 100 gravitino.server.webserver.requestHeaderSize = 131072 gravitino.server.webserver.responseHeaderSize = 131072 # Comma-separated list of filter class names to apply to the API. gravitino.server.webserver.customFilters = # Comma-separated list of REST API packages to expand gravitino.server.rest.extensionPackages = # THE CONFIGURATION FOR Gravitino ENTITY STORE gravitino.entity.store = relational gravitino.entity.store.maxTransactionSkewTimeMs = 2000 gravitino.entity.store.deleteAfterTimeMs = 604800000 gravitino.entity.store.versionRetentionCount = 1 gravitino.entity.store.relational = JDBCBackend gravitino.entity.store.relational.jdbcUrl = jdbc:mysql://localhost:3306/gravitino_db gravitino.entity.store.relational.jdbcDriver = com.mysql.cj.jdbc.Driver gravitino.entity.store.relational.jdbcUser = **** gravitino.entity.store.relational.jdbcPassword = ******** gravitino.entity.store.relational.storagePath = /root/gravitino/data/jdbc # THE CONFIGURATION FOR Gravitino CATALOG gravitino.catalog.cache.evictionIntervalMs = 3600000 # THE CONFIGURATION FOR Gravitino Entity Cache gravitino.cache.enabled = true gravitino.cache.maxEntries = 10000 gravitino.cache.expireTimeInMs = 3600000 gravitino.cache.enableStats = false gravitino.cache.enableWeigher = true gravitino.cache.implementation = caffeine # THE CONFIGURATION FOR authorization gravitino.authorization.enable = false gravitino.authorization.serviceAdmins = anonymous gravitino.authenticators = simple gravitino.authenticator.oauth.serviceAudience = test gravitino.authenticator.oauth.defaultSignKey = gravitino.authenticator.oauth.serverUri = gravitino.authenticator.oauth.tokenPath = /realms/myrealm/protocol/openid-connect/token gravitino.authenticator.oauth.provider = gravitino.authenticator.oauth.clientId = gravitino.authenticator.oauth.authority = gravitino.authenticator.oauth.scope = gravitino.authenticator.oauth.jwksUri = gravitino.authenticator.oauth.tokenValidatorClass = gravitino.authenticator.oauth.principalFields = # THE CONFIGURATION FOR AUXILIARY SERVICE gravitino.auxService.names = iceberg-rest gravitino.iceberg-rest.classpath = iceberg-rest-server/libs, iceberg-rest-server/conf gravitino.iceberg-rest.host = 0.0.0.0 gravitino.iceberg-rest.httpPort = 9001 gravitino.iceberg-rest.catalog-backend = memory gravitino.iceberg-rest.warehouse = /tmp/ # Audit log configuration gravitino.audit.enabled = false gravitino.audit.writer.className = org.apache.gravitino.audit.FileAuditWriter gravitino.audit.formatter.className = org.apache.gravitino.audit.SimpleFormatter gravitino.audit.writer.file.fileName = gravitino_audit.log gravitino.audit.writer.file.flushIntervalSecs = 10 gravitino.audit.writer.file.append = true # Metrics configuration gravitino.metrics.timeSlidingWindowSecs = 60 gravitino.iceberg-rest.catalog-config-provider = dynamic-config-provider gravitino.iceberg-rest.default-catalog-name = iceberg_catalog_yizhihangmu gravitino.iceberg-rest.gravitino-auth-type = simple gravitino.iceberg-rest.gravitino-metalake = lakehouse_metalake gravitino.iceberg-rest.gravitino-simple.user-name = anonymous gravitino.iceberg-rest.gravitino-uri = http://localhost:8090 gravitino.iceberg-rest.jdbc.schema-version = V1 ``` 3. Create a "Metalake" named "lakehouse_metalake" in the web UI, and under this "Metalake" create a "catalog" named "iceberg_catalog_yizhihangmu". The configuration is as follows： <img width="1096" height="1146" alt="image" src="https://github.com/user-attachments/assets/b67c8b8c-9575-417b-99fd-2b0f9044e719"; /> 4. Place iceberg-spark-runtime-3.5_2.12-1.10.1.jar, iceberg-aws-bundle-1.10.1.jar and iceberg-spark-runtime-3.5_2.12-1.10.1.jar into the $SPARK_HOME/jars directory. 5. Create a spark configuration file named spark-defaults.conf under the $SPARK_HOME/conf directory. ``` spark.sql.extensions org.apache.iceberg.spark.extensions.IcebergSparkSessionExtensions spark.sql.catalog.default_rest_catalog org.apache.iceberg.spark.SparkCatalog spark.sql.catalog.default_rest_catalog.type rest spark.sql.catalog.default_rest_catalog.uri http://192.168.0.105:31002/iceberg/ spark.sql.catalog.default_rest_catalog.header.X-Iceberg-Access-Delegation vended-credentials spark.sql.catalog.iceberg_catalog_yizhihangmu org.apache.iceberg.spark.SparkCatalog spark.sql.catalog.iceberg_catalog_yizhihangmu.type rest spark.sql.catalog.iceberg_catalog_yizhihangmu.uri http://192.168.0.105:31002/iceberg/ spark.sql.catalog.iceberg_catalog_yizhihangmu.header.X-Iceberg-Access-Delegation vended-credentials ``` 6. Execute the SQL ``` $SPARK_HOME/bin/beeline -u jdbc:hive2://localhost:10000/ -n sparkbeeline ``` ``` use iceberg_catalog_yizhihangmu; create database db_dev; use db_dev; create table tab_test01 ( stu_id bigint comment '学生id', stu_name string comment '学生姓名', stu_sex string comment '学生性别' ) using iceberg; show tables; ``` 7. The result is: +------------+-------------+--------------+ | namespace | tableName | isTemporary | +------------+-------------+--------------+ | db_dev | tab_test01 | false | +------------+-------------+--------------+ 8. Modify the "gravitino" configuration, enable authorization, and restart the gravitino service. ``` gravitino.authorization.enable = true gravitino.authorization.serviceAdmins = anonymous ``` 9. Repeat step 6 to start Spark SQL 10. The result is: +---------------------+-------------+--------------+ | namespace | tableName | isTemporary | +---------------------+-------------+--------------+ | lakehouse_metalake | tab_test01 | false | +---------------------+-------------+--------------+ The problem has arisen： "namespace" should be "db_dev"，However, the "namespace" shown here is "lakehouse_metalake". Applications that will be affected: Using "dbeaver" and "datagrip" to connect to "spark thriftserver" fails to display the table names. GitHub link: https://github.com/apache/gravitino/discussions/9829 ---- This is an automatically sent email for [email protected]. To unsubscribe, please send an email to: [email protected]
