KodaiD commented on PR #16512:
URL: https://github.com/apache/iceberg/pull/16512#issuecomment-4509143493
```
$ docker exec spark-iceberg curl -sL -o
/opt/spark/jars/sqlite-jdbc-3.53.1.0.jar \
https://repo1.maven.org/maven2/org/xerial/sqlite-jdbc/3.53.1.0/sqlite-jdbc-3.53.1.0.jar
$
$ docker exec -it spark-iceberg spark-sql \
--packages org.xerial:sqlite-jdbc:3.53.1.0 \
--conf spark.sql.catalog.local=org.apache.iceberg.spark.SparkCatalog \
--conf spark.sql.catalog.local.type=jdbc \
--conf spark.sql.catalog.local.jdbc-driver=org.sqlite.JDBC \
--conf
spark.sql.catalog.local.uri=jdbc:sqlite:/home/iceberg/warehouse/iceberg_catalog_db.sqlite
\
--conf spark.sql.catalog.local.warehouse=/home/iceberg/warehouse \
--conf spark.sql.defaultCatalog=local
:: loading settings :: url =
jar:file:/opt/spark/jars/ivy-2.5.1.jar!/org/apache/ivy/core/settings/ivysettings.xml
Ivy Default Cache set to: /root/.ivy2/cache
The jars for the packages stored in: /root/.ivy2/jars
org.xerial#sqlite-jdbc added as a dependency
:: resolving dependencies ::
org.apache.spark#spark-submit-parent-1b193f25-424a-4ca5-af6d-f4acd2882b79;1.0
confs: [default]
found org.xerial#sqlite-jdbc;3.53.1.0 in central
:: resolution report :: resolve 54ms :: artifacts dl 1ms
:: modules in use:
org.xerial#sqlite-jdbc;3.53.1.0 from central in [default]
---------------------------------------------------------------------
| | modules || artifacts |
| conf | number| search|dwnlded|evicted|| number|dwnlded|
---------------------------------------------------------------------
| default | 1 | 0 | 0 | 0 || 1 | 0 |
---------------------------------------------------------------------
:: retrieving ::
org.apache.spark#spark-submit-parent-1b193f25-424a-4ca5-af6d-f4acd2882b79
confs: [default]
0 artifacts copied, 1 already retrieved (0kB/2ms)
26/05/21 08:12:21 WARN NativeCodeLoader: Unable to load native-hadoop
library for your platform... using builtin-java classes where applicable
Setting default log level to "WARN".
To adjust logging level use sc.setLogLevel(newLevel). For SparkR, use
setLogLevel(newLevel).
26/05/21 08:12:21 WARN Utils: Service 'SparkUI' could not bind on port 4040.
Attempting port 4041.
Spark Web UI available at http://9c44279abf99:4041/
Spark master: local[*], Application Id: local-1779351141513
26/05/21 08:12:22 WARN JdbcCatalog: JDBC catalog is initialized without view
support. To auto-migrate the database's schema and enable view support, set
jdbc.schema-version=V1
spark-sql ()> CREATE DATABASE IF NOT EXISTS local.db;
Time taken: 0.438 seconds
spark-sql ()> CREATE TABLE local.db.sample (id int, name string);
Time taken: 0.205 seconds
spark-sql ()>
What's next:
Try Docker Debug for seamless, persistent debugging tools in any
container or image → docker debug spark-iceberg
Learn more at https://docs.docker.com/go/debug-cli/
$ tree warehouse
warehouse
├── db
│ └── sample
│ └── metadata
│ └── 00000-48503ebc-b476-47e0-8eea-f7a34777bb95.metadata.json
└── iceberg_catalog_db.sqlite
4 directories, 2 files
$
```
```
$ docker exec spark-iceberg curl -sL -o
/opt/spark/jars/sqlite-jdbc-3.53.1.0.jar \
https://repo1.maven.org/maven2/org/xerial/sqlite-jdbc/3.53.1.0/sqlite-jdbc-3.53.1.0.jar
$
$ docker exec spark-iceberg bash -c 'cat >>
/opt/spark/conf/spark-defaults.conf <<EOF
spark.sql.catalog.local
org.apache.iceberg.spark.SparkCatalog
spark.sql.catalog.local.type jdbc
spark.sql.catalog.local.uri
jdbc:sqlite:/home/iceberg/warehouse/iceberg_catalog_db.sqlite
spark.sql.catalog.local.warehouse /home/iceberg/warehouse
spark.sql.defaultCatalog local
EOF'
$
$ docker exec -it spark-iceberg spark-sql
Setting default log level to "WARN".
To adjust logging level use sc.setLogLevel(newLevel). For SparkR, use
setLogLevel(newLevel).
26/05/21 08:16:17 WARN NativeCodeLoader: Unable to load native-hadoop
library for your platform... using builtin-java classes where applicable
26/05/21 08:16:18 WARN Utils: Service 'SparkUI' could not bind on port 4040.
Attempting port 4041.
Spark Web UI available at http://3c972544b3e2:4041/
Spark master: local[*], Application Id: local-1779351378063
26/05/21 08:16:19 WARN JdbcCatalog: JDBC catalog is initialized without view
support. To auto-migrate the database's schema and enable view support, set
jdbc.schema-version=V1
spark-sql ()> CREATE DATABASE IF NOT EXISTS local.db;
Time taken: 0.421 seconds
spark-sql ()> CREATE TABLE local.db.sample (id int, name string);
Time taken: 0.21 seconds
spark-sql ()>
What's next:
Try Docker Debug for seamless, persistent debugging tools in any
container or image → docker debug spark-iceberg
Learn more at https://docs.docker.com/go/debug-cli/
$ tree warehouse
warehouse
├── db
│ └── sample
│ └── metadata
│ └── 00000-ad063400-08dc-42ff-a7d6-bb2f02886bab.metadata.json
└── iceberg_catalog_db.sqlite
4 directories, 2 files
$
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
