GitHub user krystian-blaszczyk added a comment to the discussion: Connect to Iceberg table on GCS
I am experiencing several issues while attempting to connect Amoro AMS to an external Iceberg REST catalog (Apache Polaris) using Google Cloud Storage (GCS) as the storage layer and Polaris vended-credentials for authentication. Despite adding the necessary JAR files to the lib directory (including gcs-connector-hadoop3-shaded and iceberg-gcp-bundle), I have encountered the following roadblocks: 1. Classloader / ClassNotFoundException: Amoro AMS reports java.lang.ClassNotFoundException: org.apache.iceberg.gcp.gcs.GCSFileIO, even when the JAR files are present in /usr/local/amoro/lib. In a containerized environment, the AMS process seems to look for libraries only in specific subdirectories (like /ams/lib). Forcing io-impl in the catalog configuration prevents the entire AMS dashboard from starting (resulting in an "AMS start error"). 2. Fallback to HadoopFileIO Failure: When the io-impl parameter is not explicitly set, Amoro attempts to use HadoopFileIO by default. This leads to the error: org.apache.hadoop.fs.UnsupportedFileSystemException: No FileSystem for scheme "gs". This indicates that the native GCS support from the iceberg-gcp-bundle is not being activated automatically for gs:// paths. 3. Metadata Server Timeout with vended-credentials: When using Polaris, the REST server should delegate temporary credentials. However, the Amoro logs show that the GCS connector ignores the credentials provided by Polaris and instead attempts to connect to the Google Cloud metadata server (http://169.254.169.254/...), which results in a SocketTimeoutException. Simply adding the gcs-connector-shaded library (as suggested in issue #3135) does not resolve the issue in a Polaris + Amoro architecture. The system struggles to correctly load GCP classes into the Classpath and fails to prioritize GCSFileIO over HadoopFileIO when handling delegated access tokens. GitHub link: https://github.com/apache/amoro/discussions/3135#discussioncomment-15369774 ---- This is an automatically sent email for [email protected]. To unsubscribe, please send an email to: [email protected]
