Re: [PR] Core: Add view support for JDBC catalog [iceberg]

2024-02-16 Thread via GitHub
jbonofre commented on PR #9487: URL: https://github.com/apache/iceberg/pull/9487#issuecomment-1948364274 @nastra yes, actually, I plan to improve `V0` / `V1` and more support in a separate PR introducing `JdbcAdapter` allowing us to adapt to different backend (like I did here https://githu

Re: [PR] Core: Add view support for JDBC catalog [iceberg]

2024-02-16 Thread via GitHub
nastra commented on code in PR #9487: URL: https://github.com/apache/iceberg/pull/9487#discussion_r1492427645 ## core/src/main/java/org/apache/iceberg/jdbc/JdbcUtil.java: ## @@ -81,85 +114,242 @@ final class JdbcUtil { + TABLE_NAME + ")" + ")"; -

Re: [PR] Core: Add view support for JDBC catalog [iceberg]

2024-02-16 Thread via GitHub
nastra commented on code in PR #9487: URL: https://github.com/apache/iceberg/pull/9487#discussion_r1492412706 ## core/src/main/java/org/apache/iceberg/jdbc/JdbcUtil.java: ## @@ -25,31 +25,62 @@ import java.util.Map; import java.util.Properties; import java.util.Set; +import o

Re: [PR] Core: Add view support for JDBC catalog [iceberg]

2024-02-16 Thread via GitHub
nastra commented on code in PR #9487: URL: https://github.com/apache/iceberg/pull/9487#discussion_r1492410874 ## core/src/test/java/org/apache/iceberg/jdbc/TestJdbcCatalog.java: ## @@ -158,6 +161,92 @@ public void testInitialize() { jdbcCatalog.initialize("test_jdbc_catalog

Re: [PR] Core: Add view support for JDBC catalog [iceberg]

2024-02-16 Thread via GitHub
nastra commented on code in PR #9487: URL: https://github.com/apache/iceberg/pull/9487#discussion_r1492409536 ## core/src/main/java/org/apache/iceberg/jdbc/JdbcTableOperations.java: ## @@ -182,18 +177,23 @@ private void createTable(String newMetadataLocation) throws SQLExceptio

Re: [PR] Core: Add view support for JDBC catalog [iceberg]

2024-02-16 Thread via GitHub
jbonofre commented on PR #9487: URL: https://github.com/apache/iceberg/pull/9487#issuecomment-1948053941 @nastra @ajantha-bhat @danielcweeks @rdblue I updated the PR by introducing `SchemaVersion`, but I keep the same `jdbc.add-view-support` property for users. Some notes: 1. I didn't

Re: [PR] Core: Add view support for JDBC catalog [iceberg]

2024-02-15 Thread via GitHub
jbonofre commented on code in PR #9487: URL: https://github.com/apache/iceberg/pull/9487#discussion_r1491386167 ## core/src/main/java/org/apache/iceberg/jdbc/JdbcCatalog.java: ## @@ -81,6 +87,7 @@ public class JdbcCatalog extends BaseMetastoreCatalog private final Function, J

Re: [PR] Core: Add view support for JDBC catalog [iceberg]

2024-02-15 Thread via GitHub
jbonofre commented on PR #9487: URL: https://github.com/apache/iceberg/pull/9487#issuecomment-1945703456 @danielcweeks Thanks ! Yeah, I assumed we won't have new schema update, but you are right, it's possible to have new updates in the future. So let me update the PR with schema versioning

Re: [PR] Core: Add view support for JDBC catalog [iceberg]

2024-02-14 Thread via GitHub
danielcweeks commented on code in PR #9487: URL: https://github.com/apache/iceberg/pull/9487#discussion_r1490219293 ## core/src/main/java/org/apache/iceberg/jdbc/JdbcCatalog.java: ## @@ -81,6 +87,7 @@ public class JdbcCatalog extends BaseMetastoreCatalog private final Functio

Re: [PR] Core: Add view support for JDBC catalog [iceberg]

2024-02-09 Thread via GitHub
jbonofre commented on code in PR #9487: URL: https://github.com/apache/iceberg/pull/9487#discussion_r1484377782 ## core/src/test/java/org/apache/iceberg/jdbc/TestJdbcCatalog.java: ## @@ -158,6 +161,92 @@ public void testInitialize() { jdbcCatalog.initialize("test_jdbc_catal

Re: [PR] Core: Add view support for JDBC catalog [iceberg]

2024-02-09 Thread via GitHub
nastra commented on code in PR #9487: URL: https://github.com/apache/iceberg/pull/9487#discussion_r1484373833 ## core/src/test/java/org/apache/iceberg/jdbc/TestJdbcCatalog.java: ## @@ -158,6 +161,92 @@ public void testInitialize() { jdbcCatalog.initialize("test_jdbc_catalog

Re: [PR] Core: Add view support for JDBC catalog [iceberg]

2024-02-09 Thread via GitHub
jbonofre commented on PR #9487: URL: https://github.com/apache/iceberg/pull/9487#issuecomment-1935969951 @nastra thanks ! I addressed your comments. The PR is ready for a new round 😄 Thanks again ! -- This is an automated message from the Apache Git Service. To respond to the message, ple

Re: [PR] Core: Add view support for JDBC catalog [iceberg]

2024-02-09 Thread via GitHub
nastra commented on code in PR #9487: URL: https://github.com/apache/iceberg/pull/9487#discussion_r1484320774 ## core/src/main/java/org/apache/iceberg/jdbc/JdbcUtil.java: ## @@ -81,85 +109,243 @@ final class JdbcUtil { + TABLE_NAME + ")" + ")"; -

Re: [PR] Core: Add view support for JDBC catalog [iceberg]

2024-02-09 Thread via GitHub
nastra commented on code in PR #9487: URL: https://github.com/apache/iceberg/pull/9487#discussion_r1484319960 ## core/src/main/java/org/apache/iceberg/jdbc/JdbcUtil.java: ## @@ -73,6 +99,8 @@ final class JdbcUtil { + " VARCHAR(1000)," + JdbcTableOperations.

Re: [PR] Core: Add view support for JDBC catalog [iceberg]

2024-02-09 Thread via GitHub
nastra commented on code in PR #9487: URL: https://github.com/apache/iceberg/pull/9487#discussion_r1484317280 ## core/src/main/java/org/apache/iceberg/jdbc/JdbcCatalog.java: ## @@ -60,16 +61,21 @@ import org.apache.iceberg.relocated.com.google.common.collect.Lists; import org.

Re: [PR] Core: Add view support for JDBC catalog [iceberg]

2024-02-09 Thread via GitHub
nastra commented on code in PR #9487: URL: https://github.com/apache/iceberg/pull/9487#discussion_r1484306822 ## core/src/main/java/org/apache/iceberg/jdbc/JdbcCatalog.java: ## @@ -185,14 +195,39 @@ private void initializeCatalogTables() throws InterruptedException, SQLExceptio

Re: [PR] Core: Add view support for JDBC catalog [iceberg]

2024-02-09 Thread via GitHub
nastra commented on code in PR #9487: URL: https://github.com/apache/iceberg/pull/9487#discussion_r1484299675 ## core/src/test/java/org/apache/iceberg/jdbc/TestJdbcCatalog.java: ## @@ -158,6 +162,66 @@ public void testInitialize() { jdbcCatalog.initialize("test_jdbc_catalog

Re: [PR] Core: Add view support for JDBC catalog [iceberg]

2024-02-09 Thread via GitHub
nastra commented on code in PR #9487: URL: https://github.com/apache/iceberg/pull/9487#discussion_r1484294481 ## core/src/test/java/org/apache/iceberg/jdbc/TestJdbcCatalog.java: ## @@ -158,6 +162,66 @@ public void testInitialize() { jdbcCatalog.initialize("test_jdbc_catalog

Re: [PR] Core: Add view support for JDBC catalog [iceberg]

2024-02-09 Thread via GitHub
nastra commented on code in PR #9487: URL: https://github.com/apache/iceberg/pull/9487#discussion_r1484294100 ## core/src/test/java/org/apache/iceberg/jdbc/TestJdbcCatalog.java: ## @@ -158,6 +162,66 @@ public void testInitialize() { jdbcCatalog.initialize("test_jdbc_catalog

Re: [PR] Core: Add view support for JDBC catalog [iceberg]

2024-02-09 Thread via GitHub
nastra commented on code in PR #9487: URL: https://github.com/apache/iceberg/pull/9487#discussion_r1484292812 ## core/src/test/java/org/apache/iceberg/jdbc/TestJdbcCatalog.java: ## @@ -854,4 +918,68 @@ public void report(MetricsReport report) { COUNTER.incrementAndGet();

Re: [PR] Core: Add view support for JDBC catalog [iceberg]

2024-02-09 Thread via GitHub
nastra commented on code in PR #9487: URL: https://github.com/apache/iceberg/pull/9487#discussion_r1484293364 ## core/src/test/java/org/apache/iceberg/jdbc/TestJdbcCatalog.java: ## @@ -158,6 +162,66 @@ public void testInitialize() { jdbcCatalog.initialize("test_jdbc_catalog

Re: [PR] Core: Add view support for JDBC catalog [iceberg]

2024-02-09 Thread via GitHub
jbonofre commented on code in PR #9487: URL: https://github.com/apache/iceberg/pull/9487#discussion_r1484131996 ## core/src/main/java/org/apache/iceberg/jdbc/JdbcCatalog.java: ## @@ -503,6 +560,104 @@ public boolean namespaceExists(Namespace namespace) { return JdbcUtil.nam

Re: [PR] Core: Add view support for JDBC catalog [iceberg]

2024-02-09 Thread via GitHub
jbonofre commented on code in PR #9487: URL: https://github.com/apache/iceberg/pull/9487#discussion_r1484131220 ## core/src/main/java/org/apache/iceberg/jdbc/JdbcCatalog.java: ## @@ -185,14 +195,41 @@ private void initializeCatalogTables() throws InterruptedException, SQLExcept

Re: [PR] Core: Add view support for JDBC catalog [iceberg]

2024-02-09 Thread via GitHub
nastra commented on code in PR #9487: URL: https://github.com/apache/iceberg/pull/9487#discussion_r1484017802 ## core/src/main/java/org/apache/iceberg/jdbc/JdbcCatalog.java: ## @@ -503,6 +560,104 @@ public boolean namespaceExists(Namespace namespace) { return JdbcUtil.names

Re: [PR] Core: Add view support for JDBC catalog [iceberg]

2024-02-09 Thread via GitHub
nastra commented on code in PR #9487: URL: https://github.com/apache/iceberg/pull/9487#discussion_r1484016984 ## core/src/main/java/org/apache/iceberg/jdbc/JdbcCatalog.java: ## @@ -185,14 +195,41 @@ private void initializeCatalogTables() throws InterruptedException, SQLExceptio

Re: [PR] Core: Add view support for JDBC catalog [iceberg]

2024-02-09 Thread via GitHub
nastra commented on code in PR #9487: URL: https://github.com/apache/iceberg/pull/9487#discussion_r1484015579 ## core/src/main/java/org/apache/iceberg/jdbc/JdbcCatalog.java: ## @@ -185,14 +195,41 @@ private void initializeCatalogTables() throws InterruptedException, SQLExceptio

Re: [PR] Core: Add view support for JDBC catalog [iceberg]

2024-02-09 Thread via GitHub
nastra commented on code in PR #9487: URL: https://github.com/apache/iceberg/pull/9487#discussion_r1484011553 ## core/src/main/java/org/apache/iceberg/jdbc/JdbcCatalog.java: ## @@ -60,16 +61,21 @@ import org.apache.iceberg.relocated.com.google.common.collect.Lists; import org.

Re: [PR] Core: Add view support for JDBC catalog [iceberg]

2024-02-08 Thread via GitHub
jbonofre commented on PR #9487: URL: https://github.com/apache/iceberg/pull/9487#issuecomment-1935456427 Oh by the way, I will update the PR to use single quote on the statement as double quote has a special meaning on some databases (like PostgreSQL). I'm on it. -- This is an automated

Re: [PR] Core: Add view support for JDBC catalog [iceberg]

2024-02-08 Thread via GitHub
jbonofre commented on PR #9487: URL: https://github.com/apache/iceberg/pull/9487#issuecomment-1935445632 @ajantha-bhat @nastra @rdblue @danielcweeks I updated the PR with: - support old SQL schema (not updated) - auto update (by configuration, disabled by default) - new tests for

Re: [PR] Core: Add view support for JDBC catalog [iceberg]

2024-02-08 Thread via GitHub
jbonofre commented on code in PR #9487: URL: https://github.com/apache/iceberg/pull/9487#discussion_r1483553753 ## core/src/test/java/org/apache/iceberg/jdbc/TestJdbcUtil.java: ## @@ -18,14 +18,116 @@ */ package org.apache.iceberg.jdbc; +import static org.assertj.core.api.A

Re: [PR] Core: Add view support for JDBC catalog [iceberg]

2024-02-07 Thread via GitHub
jbonofre commented on code in PR #9487: URL: https://github.com/apache/iceberg/pull/9487#discussion_r1481853805 ## core/src/test/java/org/apache/iceberg/jdbc/TestJdbcUtil.java: ## @@ -18,14 +18,116 @@ */ package org.apache.iceberg.jdbc; +import static org.assertj.core.api.A

Re: [PR] Core: Add view support for JDBC catalog [iceberg]

2024-02-07 Thread via GitHub
jbonofre commented on code in PR #9487: URL: https://github.com/apache/iceberg/pull/9487#discussion_r1481559062 ## core/src/main/java/org/apache/iceberg/jdbc/JdbcCatalog.java: ## @@ -80,19 +83,37 @@ public class JdbcCatalog extends BaseMetastoreCatalog private final Function,

Re: [PR] Core: Add view support for JDBC catalog [iceberg]

2024-02-07 Thread via GitHub
jbonofre commented on code in PR #9487: URL: https://github.com/apache/iceberg/pull/9487#discussion_r1481853805 ## core/src/test/java/org/apache/iceberg/jdbc/TestJdbcUtil.java: ## @@ -18,14 +18,116 @@ */ package org.apache.iceberg.jdbc; +import static org.assertj.core.api.A

Re: [PR] Core: Add view support for JDBC catalog [iceberg]

2024-02-07 Thread via GitHub
jbonofre commented on code in PR #9487: URL: https://github.com/apache/iceberg/pull/9487#discussion_r1481637607 ## core/src/test/java/org/apache/iceberg/jdbc/TestJdbcUtil.java: ## @@ -18,14 +18,116 @@ */ package org.apache.iceberg.jdbc; +import static org.assertj.core.api.A

Re: [PR] Core: Add view support for JDBC catalog [iceberg]

2024-02-07 Thread via GitHub
nastra commented on code in PR #9487: URL: https://github.com/apache/iceberg/pull/9487#discussion_r1481631049 ## core/src/test/java/org/apache/iceberg/jdbc/TestJdbcUtil.java: ## @@ -18,14 +18,116 @@ */ package org.apache.iceberg.jdbc; +import static org.assertj.core.api.Ass

Re: [PR] Core: Add view support for JDBC catalog [iceberg]

2024-02-07 Thread via GitHub
jbonofre commented on code in PR #9487: URL: https://github.com/apache/iceberg/pull/9487#discussion_r1481560466 ## core/src/test/java/org/apache/iceberg/jdbc/TestJdbcUtil.java: ## @@ -18,14 +18,116 @@ */ package org.apache.iceberg.jdbc; +import static org.assertj.core.api.A

Re: [PR] Core: Add view support for JDBC catalog [iceberg]

2024-02-07 Thread via GitHub
jbonofre commented on code in PR #9487: URL: https://github.com/apache/iceberg/pull/9487#discussion_r1481559062 ## core/src/main/java/org/apache/iceberg/jdbc/JdbcCatalog.java: ## @@ -80,19 +83,37 @@ public class JdbcCatalog extends BaseMetastoreCatalog private final Function,

Re: [PR] Core: Add view support for JDBC catalog [iceberg]

2024-02-06 Thread via GitHub
jbonofre commented on code in PR #9487: URL: https://github.com/apache/iceberg/pull/9487#discussion_r1480332429 ## core/src/main/java/org/apache/iceberg/jdbc/JdbcCatalog.java: ## @@ -245,13 +290,30 @@ public List listTables(Namespace namespace) { row -> Jd

Re: [PR] Core: Add view support for JDBC catalog [iceberg]

2024-02-06 Thread via GitHub
jbonofre commented on code in PR #9487: URL: https://github.com/apache/iceberg/pull/9487#discussion_r147994 ## core/src/main/java/org/apache/iceberg/jdbc/JdbcCatalog.java: ## @@ -80,19 +83,37 @@ public class JdbcCatalog extends BaseMetastoreCatalog private final Function,

Re: [PR] Core: Add view support for JDBC catalog [iceberg]

2024-02-06 Thread via GitHub
ajantha-bhat commented on code in PR #9487: URL: https://github.com/apache/iceberg/pull/9487#discussion_r1479508362 ## core/src/main/java/org/apache/iceberg/jdbc/JdbcCatalog.java: ## @@ -80,19 +83,37 @@ public class JdbcCatalog extends BaseMetastoreCatalog private final Funct

Re: [PR] Core: Add view support for JDBC catalog [iceberg]

2024-02-06 Thread via GitHub
ajantha-bhat commented on code in PR #9487: URL: https://github.com/apache/iceberg/pull/9487#discussion_r1479508362 ## core/src/main/java/org/apache/iceberg/jdbc/JdbcCatalog.java: ## @@ -80,19 +83,37 @@ public class JdbcCatalog extends BaseMetastoreCatalog private final Funct

Re: [PR] Core: Add view support for JDBC catalog [iceberg]

2024-02-06 Thread via GitHub
nastra commented on code in PR #9487: URL: https://github.com/apache/iceberg/pull/9487#discussion_r1479468964 ## core/src/main/java/org/apache/iceberg/jdbc/JdbcCatalog.java: ## @@ -80,19 +83,37 @@ public class JdbcCatalog extends BaseMetastoreCatalog private final Function, F

Re: [PR] Core: Add view support for JDBC catalog [iceberg]

2024-02-06 Thread via GitHub
nastra commented on code in PR #9487: URL: https://github.com/apache/iceberg/pull/9487#discussion_r1479429841 ## core/src/main/java/org/apache/iceberg/jdbc/JdbcCatalog.java: ## @@ -503,6 +566,92 @@ public boolean namespaceExists(Namespace namespace) { return JdbcUtil.namesp

Re: [PR] Core: Add view support for JDBC catalog [iceberg]

2024-02-06 Thread via GitHub
nastra commented on code in PR #9487: URL: https://github.com/apache/iceberg/pull/9487#discussion_r1479428086 ## core/src/main/java/org/apache/iceberg/jdbc/JdbcCatalog.java: ## @@ -245,13 +290,30 @@ public List listTables(Namespace namespace) { row -> Jdbc

Re: [PR] Core: Add view support for JDBC catalog [iceberg]

2024-02-05 Thread via GitHub
jbonofre commented on code in PR #9487: URL: https://github.com/apache/iceberg/pull/9487#discussion_r1478650850 ## core/src/main/java/org/apache/iceberg/jdbc/JdbcUtil.java: ## @@ -25,31 +25,36 @@ import java.util.Map; import java.util.Properties; import java.util.Set; +import

Re: [PR] Core: Add view support for JDBC catalog [iceberg]

2024-02-05 Thread via GitHub
jbonofre commented on code in PR #9487: URL: https://github.com/apache/iceberg/pull/9487#discussion_r1478542258 ## core/src/main/java/org/apache/iceberg/jdbc/JdbcUtil.java: ## @@ -81,85 +90,103 @@ final class JdbcUtil { + TABLE_NAME + ")" + ")";

Re: [PR] Core: Add view support for JDBC catalog [iceberg]

2024-02-05 Thread via GitHub
jbonofre commented on code in PR #9487: URL: https://github.com/apache/iceberg/pull/9487#discussion_r1478540570 ## core/src/test/java/org/apache/iceberg/jdbc/TestJdbcUtil.java: ## @@ -18,14 +18,116 @@ */ package org.apache.iceberg.jdbc; +import static org.assertj.core.api.A

Re: [PR] Core: Add view support for JDBC catalog [iceberg]

2024-02-05 Thread via GitHub
nastra commented on code in PR #9487: URL: https://github.com/apache/iceberg/pull/9487#discussion_r1478448616 ## core/src/main/java/org/apache/iceberg/jdbc/JdbcViewOperations.java: ## @@ -0,0 +1,204 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or mor

Re: [PR] Core: Add view support for JDBC catalog [iceberg]

2024-02-04 Thread via GitHub
jbonofre commented on code in PR #9487: URL: https://github.com/apache/iceberg/pull/9487#discussion_r1477725114 ## core/src/main/java/org/apache/iceberg/jdbc/JdbcCatalog.java: ## @@ -80,19 +83,37 @@ public class JdbcCatalog extends BaseMetastoreCatalog private final Function,

Re: [PR] Core: Add view support for JDBC catalog [iceberg]

2024-02-04 Thread via GitHub
ajantha-bhat commented on code in PR #9487: URL: https://github.com/apache/iceberg/pull/9487#discussion_r1477365303 ## core/src/main/java/org/apache/iceberg/jdbc/JdbcCatalog.java: ## @@ -80,19 +83,37 @@ public class JdbcCatalog extends BaseMetastoreCatalog private final Funct

Re: [PR] Core: Add view support for JDBC catalog [iceberg]

2024-02-02 Thread via GitHub
jbonofre commented on PR #9487: URL: https://github.com/apache/iceberg/pull/9487#issuecomment-1924405491 @nastra @danielcweeks @rdblue I updated the PR. You can already do a new round (I'm checking a couple of stuff but ready to review already). -- This is an automated message from the Ap

Re: [PR] Core: Add view support for JDBC catalog [iceberg]

2024-02-02 Thread via GitHub
jbonofre commented on code in PR #9487: URL: https://github.com/apache/iceberg/pull/9487#discussion_r1475826840 ## core/src/main/java/org/apache/iceberg/jdbc/JdbcTableOperations.java: ## @@ -182,18 +169,13 @@ private void createTable(String newMetadataLocation) throws SQLExcept

Re: [PR] Core: Add view support for JDBC catalog [iceberg]

2024-02-02 Thread via GitHub
jbonofre commented on code in PR #9487: URL: https://github.com/apache/iceberg/pull/9487#discussion_r1475821544 ## core/src/main/java/org/apache/iceberg/jdbc/JdbcCatalog.java: ## @@ -503,6 +550,84 @@ public boolean namespaceExists(Namespace namespace) { return JdbcUtil.name

Re: [PR] Core: Add view support for JDBC catalog [iceberg]

2024-02-02 Thread via GitHub
jbonofre commented on code in PR #9487: URL: https://github.com/apache/iceberg/pull/9487#discussion_r1476197844 ## core/src/main/java/org/apache/iceberg/jdbc/JdbcCatalog.java: ## @@ -503,6 +550,84 @@ public boolean namespaceExists(Namespace namespace) { return JdbcUtil.name

Re: [PR] Core: Add view support for JDBC catalog [iceberg]

2024-02-02 Thread via GitHub
nastra commented on code in PR #9487: URL: https://github.com/apache/iceberg/pull/9487#discussion_r1476145516 ## core/src/main/java/org/apache/iceberg/jdbc/JdbcCatalog.java: ## @@ -503,6 +550,84 @@ public boolean namespaceExists(Namespace namespace) { return JdbcUtil.namesp

Re: [PR] Core: Add view support for JDBC catalog [iceberg]

2024-02-02 Thread via GitHub
jbonofre commented on code in PR #9487: URL: https://github.com/apache/iceberg/pull/9487#discussion_r1475830202 ## core/src/main/java/org/apache/iceberg/jdbc/JdbcViewOperations.java: ## @@ -0,0 +1,202 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or m

Re: [PR] Core: Add view support for JDBC catalog [iceberg]

2024-02-02 Thread via GitHub
jbonofre commented on code in PR #9487: URL: https://github.com/apache/iceberg/pull/9487#discussion_r1475829647 ## core/src/main/java/org/apache/iceberg/jdbc/JdbcUtil.java: ## @@ -303,7 +330,152 @@ public static Properties filterAndRemovePrefix(Map properties, S return res

Re: [PR] Core: Add view support for JDBC catalog [iceberg]

2024-02-02 Thread via GitHub
jbonofre commented on code in PR #9487: URL: https://github.com/apache/iceberg/pull/9487#discussion_r1475829107 ## core/src/main/java/org/apache/iceberg/jdbc/JdbcTableOperations.java: ## @@ -71,7 +68,7 @@ public void doRefresh() { Map table; try { - table = getT

Re: [PR] Core: Add view support for JDBC catalog [iceberg]

2024-02-02 Thread via GitHub
jbonofre commented on code in PR #9487: URL: https://github.com/apache/iceberg/pull/9487#discussion_r1475826840 ## core/src/main/java/org/apache/iceberg/jdbc/JdbcTableOperations.java: ## @@ -182,18 +169,13 @@ private void createTable(String newMetadataLocation) throws SQLExcept

Re: [PR] Core: Add view support for JDBC catalog [iceberg]

2024-02-02 Thread via GitHub
jbonofre commented on code in PR #9487: URL: https://github.com/apache/iceberg/pull/9487#discussion_r1475821544 ## core/src/main/java/org/apache/iceberg/jdbc/JdbcCatalog.java: ## @@ -503,6 +550,84 @@ public boolean namespaceExists(Namespace namespace) { return JdbcUtil.name

Re: [PR] Core: Add view support for JDBC catalog [iceberg]

2024-02-02 Thread via GitHub
jbonofre commented on code in PR #9487: URL: https://github.com/apache/iceberg/pull/9487#discussion_r1475818719 ## core/src/main/java/org/apache/iceberg/jdbc/JdbcCatalog.java: ## @@ -245,13 +286,18 @@ public List listTables(Namespace namespace) { row -> Jd

Re: [PR] Core: Add view support for JDBC catalog [iceberg]

2024-02-02 Thread via GitHub
jbonofre commented on code in PR #9487: URL: https://github.com/apache/iceberg/pull/9487#discussion_r1475818070 ## core/src/main/java/org/apache/iceberg/jdbc/JdbcCatalog.java: ## @@ -80,10 +83,11 @@ public class JdbcCatalog extends BaseMetastoreCatalog private final Function,

Re: [PR] Core: Add view support for JDBC catalog [iceberg]

2024-02-01 Thread via GitHub
danielcweeks commented on code in PR #9487: URL: https://github.com/apache/iceberg/pull/9487#discussion_r1475391895 ## core/src/main/java/org/apache/iceberg/jdbc/JdbcCatalog.java: ## @@ -245,13 +286,18 @@ public List listTables(Namespace namespace) { row ->

Re: [PR] Core: Add view support for JDBC catalog [iceberg]

2024-02-01 Thread via GitHub
danielcweeks commented on code in PR #9487: URL: https://github.com/apache/iceberg/pull/9487#discussion_r1475388391 ## core/src/main/java/org/apache/iceberg/jdbc/JdbcViewOperations.java: ## @@ -0,0 +1,202 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + *

Re: [PR] Core: Add view support for JDBC catalog [iceberg]

2024-02-01 Thread via GitHub
danielcweeks commented on code in PR #9487: URL: https://github.com/apache/iceberg/pull/9487#discussion_r1475378129 ## core/src/main/java/org/apache/iceberg/jdbc/JdbcUtil.java: ## @@ -303,7 +330,152 @@ public static Properties filterAndRemovePrefix(Map properties, S return

Re: [PR] Core: Add view support for JDBC catalog [iceberg]

2024-02-01 Thread via GitHub
danielcweeks commented on code in PR #9487: URL: https://github.com/apache/iceberg/pull/9487#discussion_r1475376446 ## core/src/main/java/org/apache/iceberg/jdbc/JdbcTableOperations.java: ## @@ -71,7 +68,7 @@ public void doRefresh() { Map table; try { - table =

Re: [PR] Core: Add view support for JDBC catalog [iceberg]

2024-02-01 Thread via GitHub
danielcweeks commented on code in PR #9487: URL: https://github.com/apache/iceberg/pull/9487#discussion_r1475366557 ## core/src/main/java/org/apache/iceberg/jdbc/JdbcTableOperations.java: ## @@ -71,7 +68,7 @@ public void doRefresh() { Map table; try { - table =

Re: [PR] Core: Add view support for JDBC catalog [iceberg]

2024-02-01 Thread via GitHub
danielcweeks commented on code in PR #9487: URL: https://github.com/apache/iceberg/pull/9487#discussion_r1475366557 ## core/src/main/java/org/apache/iceberg/jdbc/JdbcTableOperations.java: ## @@ -71,7 +68,7 @@ public void doRefresh() { Map table; try { - table =

Re: [PR] Core: Add view support for JDBC catalog [iceberg]

2024-02-01 Thread via GitHub
danielcweeks commented on code in PR #9487: URL: https://github.com/apache/iceberg/pull/9487#discussion_r1475365294 ## core/src/main/java/org/apache/iceberg/jdbc/JdbcTableOperations.java: ## @@ -182,18 +169,13 @@ private void createTable(String newMetadataLocation) throws SQLEx

Re: [PR] Core: Add view support for JDBC catalog [iceberg]

2024-02-01 Thread via GitHub
danielcweeks commented on code in PR #9487: URL: https://github.com/apache/iceberg/pull/9487#discussion_r1475359184 ## core/src/main/java/org/apache/iceberg/jdbc/JdbcCatalog.java: ## @@ -503,6 +550,84 @@ public boolean namespaceExists(Namespace namespace) { return JdbcUtil.

Re: [PR] Core: Add view support for JDBC catalog [iceberg]

2024-02-01 Thread via GitHub
danielcweeks commented on code in PR #9487: URL: https://github.com/apache/iceberg/pull/9487#discussion_r1475337646 ## core/src/main/java/org/apache/iceberg/jdbc/JdbcCatalog.java: ## @@ -245,13 +286,18 @@ public List listTables(Namespace namespace) { row ->

Re: [PR] Core: Add view support for JDBC catalog [iceberg]

2024-02-01 Thread via GitHub
danielcweeks commented on code in PR #9487: URL: https://github.com/apache/iceberg/pull/9487#discussion_r1475329370 ## core/src/main/java/org/apache/iceberg/jdbc/JdbcCatalog.java: ## @@ -80,10 +83,11 @@ public class JdbcCatalog extends BaseMetastoreCatalog private final Funct

Re: [PR] Core: Add view support for JDBC catalog [iceberg]

2024-01-31 Thread via GitHub
jbonofre commented on PR #9487: URL: https://github.com/apache/iceberg/pull/9487#issuecomment-1918818634 I updated the PR with: - a property triggering SQL table update if needed - update of previous version SQL schema to the new model - update the SQL statement to get Iceberg table,

Re: [PR] Core: Add view support for JDBC catalog [iceberg]

2024-01-28 Thread via GitHub
jbonofre commented on PR #9487: URL: https://github.com/apache/iceberg/pull/9487#issuecomment-1913734869 @rdblue it's reasonable. Let me update the PR to add an option to migrate database schema and enable view. Thanks ! -- This is an automated message from the Apache Git Service. To resp

Re: [PR] Core: Add view support for JDBC catalog [iceberg]

2024-01-28 Thread via GitHub
rdblue commented on PR #9487: URL: https://github.com/apache/iceberg/pull/9487#issuecomment-1913713816 @jbonofre, I think @danielcweeks and I are both on the side of needing to have a single table. The less bad option is to have older clients fail to read metadata. There is no other solutio

Re: [PR] Core: Add view support for JDBC catalog [iceberg]

2024-01-25 Thread via GitHub
jbonofre commented on PR #9487: URL: https://github.com/apache/iceberg/pull/9487#issuecomment-1911560276 If we consider the atomic name between view and table is the most important to guarantee, then we should keep the current implementation with a single table. Using two tables would be po

Re: [PR] Core: Add view support for JDBC catalog [iceberg]

2024-01-25 Thread via GitHub
rdblue commented on PR #9487: URL: https://github.com/apache/iceberg/pull/9487#issuecomment-1911227076 > I don't know what the behavior would be when trying to load a table (which is actually a view) in an older client, but it would likely break in unexpected ways. The older client w

Re: [PR] Core: Add view support for JDBC catalog [iceberg]

2024-01-25 Thread via GitHub
danielcweeks commented on PR #9487: URL: https://github.com/apache/iceberg/pull/9487#issuecomment-1911129303 @jbonofre before we start changing the implementation back, I think we need to get consensus on what direction we're going and I don't think that's been established. ## Option

Re: [PR] Core: Add view support for JDBC catalog [iceberg]

2024-01-25 Thread via GitHub
jbonofre commented on PR #9487: URL: https://github.com/apache/iceberg/pull/9487#issuecomment-1910696472 @danielcweeks That's why initially I stored the views in a separate table: it doesn't break the existing clients. We can easily support 1.4.x client with 1.5.x JDBC catalog: I can upda

Re: [PR] Core: Add view support for JDBC catalog [iceberg]

2024-01-25 Thread via GitHub
danielcweeks commented on PR #9487: URL: https://github.com/apache/iceberg/pull/9487#issuecomment-1910665511 @jbonofre there are actually some really serious issues here that need to be addressed in terms of migration of the database. In addition to just the syntax that @nastra and @ajanth

Re: [PR] Core: Add view support for JDBC catalog [iceberg]

2024-01-25 Thread via GitHub
jbonofre commented on code in PR #9487: URL: https://github.com/apache/iceberg/pull/9487#discussion_r1466387462 ## core/src/main/java/org/apache/iceberg/jdbc/JdbcUtil.java: ## @@ -25,161 +25,177 @@ import java.util.Map; import java.util.Properties; import java.util.Set; +impo

Re: [PR] Core: Add view support for JDBC catalog [iceberg]

2024-01-25 Thread via GitHub
nastra commented on code in PR #9487: URL: https://github.com/apache/iceberg/pull/9487#discussion_r1466252837 ## core/src/main/java/org/apache/iceberg/jdbc/JdbcUtil.java: ## @@ -25,161 +25,177 @@ import java.util.Map; import java.util.Properties; import java.util.Set; +import

Re: [PR] Core: Add view support for JDBC catalog [iceberg]

2024-01-25 Thread via GitHub
jbonofre commented on PR #9487: URL: https://github.com/apache/iceberg/pull/9487#issuecomment-1910197483 @ajantha-bhat good point. Let me test and eventually update. I will also check if we can do better (that's not super good to override to statements imho 😄 ). -- This is an automated m

Re: [PR] Core: Add view support for JDBC catalog [iceberg]

2024-01-25 Thread via GitHub
ajantha-bhat commented on PR #9487: URL: https://github.com/apache/iceberg/pull/9487#issuecomment-1910127068 > do we know how this affects existing PyIceberg and Trino clients? I think once we've validated those still work we should be good to go. Trino client code: https://githu

Re: [PR] Core: Add view support for JDBC catalog [iceberg]

2024-01-25 Thread via GitHub
ajantha-bhat commented on code in PR #9487: URL: https://github.com/apache/iceberg/pull/9487#discussion_r1466259305 ## core/src/main/java/org/apache/iceberg/jdbc/JdbcUtil.java: ## @@ -25,161 +25,177 @@ import java.util.Map; import java.util.Properties; import java.util.Set; +

Re: [PR] Core: Add view support for JDBC catalog [iceberg]

2024-01-25 Thread via GitHub
ajantha-bhat commented on code in PR #9487: URL: https://github.com/apache/iceberg/pull/9487#discussion_r1466257875 ## core/src/main/java/org/apache/iceberg/jdbc/JdbcUtil.java: ## @@ -25,161 +25,177 @@ import java.util.Map; import java.util.Properties; import java.util.Set; +

Re: [PR] Core: Add view support for JDBC catalog [iceberg]

2024-01-25 Thread via GitHub
ajantha-bhat commented on code in PR #9487: URL: https://github.com/apache/iceberg/pull/9487#discussion_r1466256884 ## core/src/main/java/org/apache/iceberg/jdbc/JdbcUtil.java: ## @@ -25,161 +25,177 @@ import java.util.Map; import java.util.Properties; import java.util.Set; +

Re: [PR] Core: Add view support for JDBC catalog [iceberg]

2024-01-25 Thread via GitHub
jbonofre commented on code in PR #9487: URL: https://github.com/apache/iceberg/pull/9487#discussion_r1466256486 ## core/src/main/java/org/apache/iceberg/jdbc/JdbcUtil.java: ## @@ -25,161 +25,177 @@ import java.util.Map; import java.util.Properties; import java.util.Set; +impo

Re: [PR] Core: Add view support for JDBC catalog [iceberg]

2024-01-25 Thread via GitHub
nastra commented on code in PR #9487: URL: https://github.com/apache/iceberg/pull/9487#discussion_r1466252837 ## core/src/main/java/org/apache/iceberg/jdbc/JdbcUtil.java: ## @@ -25,161 +25,177 @@ import java.util.Map; import java.util.Properties; import java.util.Set; +import

Re: [PR] Core: Add view support for JDBC catalog [iceberg]

2024-01-25 Thread via GitHub
jbonofre commented on code in PR #9487: URL: https://github.com/apache/iceberg/pull/9487#discussion_r1466247335 ## core/src/main/java/org/apache/iceberg/jdbc/JdbcUtil.java: ## @@ -303,7 +319,152 @@ public static Properties filterAndRemovePrefix(Map properties, S return res

Re: [PR] Core: Add view support for JDBC catalog [iceberg]

2024-01-25 Thread via GitHub
jbonofre commented on code in PR #9487: URL: https://github.com/apache/iceberg/pull/9487#discussion_r1466246768 ## core/src/main/java/org/apache/iceberg/jdbc/JdbcUtil.java: ## @@ -25,161 +25,177 @@ import java.util.Map; import java.util.Properties; import java.util.Set; +impo

Re: [PR] Core: Add view support for JDBC catalog [iceberg]

2024-01-25 Thread via GitHub
nastra commented on code in PR #9487: URL: https://github.com/apache/iceberg/pull/9487#discussion_r1466245162 ## core/src/main/java/org/apache/iceberg/jdbc/JdbcUtil.java: ## @@ -25,161 +25,177 @@ import java.util.Map; import java.util.Properties; import java.util.Set; +import

Re: [PR] Core: Add view support for JDBC catalog [iceberg]

2024-01-25 Thread via GitHub
nastra commented on code in PR #9487: URL: https://github.com/apache/iceberg/pull/9487#discussion_r1466243152 ## core/src/main/java/org/apache/iceberg/jdbc/JdbcUtil.java: ## @@ -303,7 +319,152 @@ public static Properties filterAndRemovePrefix(Map properties, S return resul

Re: [PR] Core: Add view support for JDBC catalog [iceberg]

2024-01-25 Thread via GitHub
jbonofre commented on PR #9487: URL: https://github.com/apache/iceberg/pull/9487#issuecomment-1909753871 @rdblue Thanks ! Yes, @danielcweeks had a good point. Let me work on this, including a "db update test" 😄 -- This is an automated message from the Apache Git Service. To respond to th

Re: [PR] Core: Add view support for JDBC catalog [iceberg]

2024-01-25 Thread via GitHub
jbonofre commented on code in PR #9487: URL: https://github.com/apache/iceberg/pull/9487#discussion_r1466089102 ## core/src/main/java/org/apache/iceberg/jdbc/JdbcCatalog.java: ## @@ -217,10 +226,11 @@ public boolean dropTable(TableIdentifier identifier, boolean purge) {

Re: [PR] Core: Add view support for JDBC catalog [iceberg]

2024-01-24 Thread via GitHub
rdblue commented on PR #9487: URL: https://github.com/apache/iceberg/pull/9487#issuecomment-1909106993 @jbonofre, thanks for updating this. It's looking really good now! I think the remaining issue is what @danielcweeks brought up in the last sync: do we know how this affects existing

Re: [PR] Core: Add view support for JDBC catalog [iceberg]

2024-01-24 Thread via GitHub
rdblue commented on code in PR #9487: URL: https://github.com/apache/iceberg/pull/9487#discussion_r1465669874 ## core/src/main/java/org/apache/iceberg/jdbc/JdbcCatalog.java: ## @@ -217,10 +226,11 @@ public boolean dropTable(TableIdentifier identifier, boolean purge) { in

Re: [PR] Core: Add view support for JDBC catalog [iceberg]

2024-01-24 Thread via GitHub
jbonofre commented on PR #9487: URL: https://github.com/apache/iceberg/pull/9487#issuecomment-1908261665 @ajantha-bhat @nastra @rdblue I refactored the PR to use a single SQL table for both tables and views, using `type` column to distinguish. The pros for this new approach: 1. the code

Re: [PR] Core: Add view support for JDBC catalog [iceberg]

2024-01-23 Thread via GitHub
jbonofre commented on PR #9487: URL: https://github.com/apache/iceberg/pull/9487#issuecomment-1906568528 @ajantha-bhat correct, corner case but possible 😸 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above

Re: [PR] Core: Add view support for JDBC catalog [iceberg]

2024-01-23 Thread via GitHub
ajantha-bhat commented on PR #9487: URL: https://github.com/apache/iceberg/pull/9487#issuecomment-1906559757 > I'm reworking a bit the SQL statement to have views and tables in the same JDBC table, especially to address name atomically between table and view. When we create a view, we che

  1   2   >