Re: [PR] GCP: Add Iceberg Catalog for GCP BigQuery Metastore [iceberg]

2025-03-07 Thread via GitHub
hesham-medhat commented on PR #11039: URL: https://github.com/apache/iceberg/pull/11039#issuecomment-2707861898 Thanks for finding the time, Ryan. Unironically, now **I** am on a long medical leave, but Google is committed to this, and we will find the time to finalize this review soon. -

Re: [PR] GCP: Add Iceberg Catalog for GCP BigQuery Metastore [iceberg]

2025-03-07 Thread via GitHub
github-actions[bot] commented on PR #11039: URL: https://github.com/apache/iceberg/pull/11039#issuecomment-2707763058 This pull request has been marked as stale due to 30 days of inactivity. It will be closed in 1 week if no further activity occurs. If you think that’s incorrect or this pul

Re: [PR] GCP: Add Iceberg Catalog for GCP BigQuery Metastore [iceberg]

2025-02-05 Thread via GitHub
rdblue commented on code in PR #11039: URL: https://github.com/apache/iceberg/pull/11039#discussion_r1943623327 ## build.gradle: ## @@ -157,7 +157,7 @@ subprojects { rootTask.finalizedBy showDeprecationRulesOnRevApiFailure } } - + Review Comment: Unn

Re: [PR] GCP: Add Iceberg Catalog for GCP BigQuery Metastore [iceberg]

2025-02-05 Thread via GitHub
rdblue commented on code in PR #11039: URL: https://github.com/apache/iceberg/pull/11039#discussion_r1943622533 ## build.gradle: ## @@ -636,6 +636,45 @@ project(':iceberg-delta-lake') { } } +project(':iceberg-bigquery') { + test { +useJUnitPlatform() + } + + depende

Re: [PR] GCP: Add Iceberg Catalog for GCP BigQuery Metastore [iceberg]

2025-02-05 Thread via GitHub
rdblue commented on code in PR #11039: URL: https://github.com/apache/iceberg/pull/11039#discussion_r194362 ## build.gradle: ## @@ -636,6 +636,45 @@ project(':iceberg-delta-lake') { } } +project(':iceberg-bigquery') { + test { +useJUnitPlatform() + } + + depende

Re: [PR] GCP: Add Iceberg Catalog for GCP BigQuery Metastore [iceberg]

2025-02-05 Thread via GitHub
rdblue commented on code in PR #11039: URL: https://github.com/apache/iceberg/pull/11039#discussion_r1943620170 ## bigquery/src/main/java/org/apache/iceberg/gcp/bigquery/metastore/BigQueryMetastoreCatalog.java: ## @@ -0,0 +1,328 @@ +/* + * Licensed to the Apache Software Foundat

Re: [PR] GCP: Add Iceberg Catalog for GCP BigQuery Metastore [iceberg]

2025-02-05 Thread via GitHub
rdblue commented on code in PR #11039: URL: https://github.com/apache/iceberg/pull/11039#discussion_r1943619867 ## bigquery/src/main/java/org/apache/iceberg/gcp/bigquery/metastore/BigQueryMetastoreCatalog.java: ## @@ -0,0 +1,328 @@ +/* + * Licensed to the Apache Software Foundat

Re: [PR] GCP: Add Iceberg Catalog for GCP BigQuery Metastore [iceberg]

2025-02-05 Thread via GitHub
rdblue commented on code in PR #11039: URL: https://github.com/apache/iceberg/pull/11039#discussion_r1943617650 ## bigquery/src/main/java/org/apache/iceberg/gcp/bigquery/metastore/BigQueryMetastoreCatalog.java: ## @@ -0,0 +1,328 @@ +/* + * Licensed to the Apache Software Foundat

Re: [PR] GCP: Add Iceberg Catalog for GCP BigQuery Metastore [iceberg]

2025-02-05 Thread via GitHub
rdblue commented on code in PR #11039: URL: https://github.com/apache/iceberg/pull/11039#discussion_r1943617650 ## bigquery/src/main/java/org/apache/iceberg/gcp/bigquery/metastore/BigQueryMetastoreCatalog.java: ## @@ -0,0 +1,328 @@ +/* + * Licensed to the Apache Software Foundat

Re: [PR] GCP: Add Iceberg Catalog for GCP BigQuery Metastore [iceberg]

2025-02-05 Thread via GitHub
rdblue commented on code in PR #11039: URL: https://github.com/apache/iceberg/pull/11039#discussion_r1943616455 ## bigquery/src/main/java/org/apache/iceberg/gcp/bigquery/metastore/BigQueryMetastoreCatalog.java: ## @@ -0,0 +1,328 @@ +/* + * Licensed to the Apache Software Foundat

Re: [PR] GCP: Add Iceberg Catalog for GCP BigQuery Metastore [iceberg]

2025-02-05 Thread via GitHub
rdblue commented on code in PR #11039: URL: https://github.com/apache/iceberg/pull/11039#discussion_r1943615685 ## bigquery/src/main/java/org/apache/iceberg/gcp/bigquery/metastore/BigQueryMetastoreCatalog.java: ## @@ -0,0 +1,328 @@ +/* + * Licensed to the Apache Software Foundat

Re: [PR] GCP: Add Iceberg Catalog for GCP BigQuery Metastore [iceberg]

2025-02-05 Thread via GitHub
rdblue commented on code in PR #11039: URL: https://github.com/apache/iceberg/pull/11039#discussion_r1943611905 ## bigquery/src/main/java/org/apache/iceberg/gcp/bigquery/metastore/BigQueryMetastoreCatalog.java: ## @@ -0,0 +1,328 @@ +/* + * Licensed to the Apache Software Foundat

Re: [PR] GCP: Add Iceberg Catalog for GCP BigQuery Metastore [iceberg]

2025-02-05 Thread via GitHub
rdblue commented on code in PR #11039: URL: https://github.com/apache/iceberg/pull/11039#discussion_r1943611005 ## bigquery/src/main/java/org/apache/iceberg/gcp/bigquery/metastore/BigQueryMetastoreCatalog.java: ## @@ -0,0 +1,328 @@ +/* + * Licensed to the Apache Software Foundat

Re: [PR] GCP: Add Iceberg Catalog for GCP BigQuery Metastore [iceberg]

2025-02-05 Thread via GitHub
rdblue commented on code in PR #11039: URL: https://github.com/apache/iceberg/pull/11039#discussion_r1943610664 ## bigquery/src/main/java/org/apache/iceberg/gcp/bigquery/metastore/BigQueryMetastoreCatalog.java: ## @@ -0,0 +1,328 @@ +/* + * Licensed to the Apache Software Foundat

Re: [PR] GCP: Add Iceberg Catalog for GCP BigQuery Metastore [iceberg]

2025-02-05 Thread via GitHub
rdblue commented on code in PR #11039: URL: https://github.com/apache/iceberg/pull/11039#discussion_r1943610203 ## bigquery/src/main/java/org/apache/iceberg/gcp/bigquery/metastore/BigQueryMetastoreCatalog.java: ## @@ -0,0 +1,328 @@ +/* + * Licensed to the Apache Software Foundat

Re: [PR] GCP: Add Iceberg Catalog for GCP BigQuery Metastore [iceberg]

2025-02-05 Thread via GitHub
rdblue commented on code in PR #11039: URL: https://github.com/apache/iceberg/pull/11039#discussion_r1943609820 ## bigquery/src/main/java/org/apache/iceberg/gcp/bigquery/metastore/BigQueryMetastoreCatalog.java: ## @@ -0,0 +1,328 @@ +/* + * Licensed to the Apache Software Foundat

Re: [PR] GCP: Add Iceberg Catalog for GCP BigQuery Metastore [iceberg]

2025-02-05 Thread via GitHub
rdblue commented on code in PR #11039: URL: https://github.com/apache/iceberg/pull/11039#discussion_r1943608036 ## bigquery/src/main/java/org/apache/iceberg/gcp/bigquery/metastore/BigQueryMetastoreCatalog.java: ## @@ -0,0 +1,328 @@ +/* + * Licensed to the Apache Software Foundat

Re: [PR] GCP: Add Iceberg Catalog for GCP BigQuery Metastore [iceberg]

2025-02-05 Thread via GitHub
rdblue commented on code in PR #11039: URL: https://github.com/apache/iceberg/pull/11039#discussion_r1943605981 ## bigquery/src/main/java/org/apache/iceberg/gcp/bigquery/BigQueryClient.java: ## @@ -0,0 +1,129 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one

Re: [PR] GCP: Add Iceberg Catalog for GCP BigQuery Metastore [iceberg]

2025-02-05 Thread via GitHub
rdblue commented on code in PR #11039: URL: https://github.com/apache/iceberg/pull/11039#discussion_r1943579431 ## bigquery/src/main/java/org/apache/iceberg/gcp/bigquery/BigQueryClient.java: ## @@ -0,0 +1,129 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one

Re: [PR] GCP: Add Iceberg Catalog for GCP BigQuery Metastore [iceberg]

2025-02-05 Thread via GitHub
rdblue commented on code in PR #11039: URL: https://github.com/apache/iceberg/pull/11039#discussion_r1943575699 ## bigquery/src/main/java/org/apache/iceberg/gcp/bigquery/BigQueryClientImpl.java: ## @@ -0,0 +1,492 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under

Re: [PR] GCP: Add Iceberg Catalog for GCP BigQuery Metastore [iceberg]

2025-02-05 Thread via GitHub
rdblue commented on code in PR #11039: URL: https://github.com/apache/iceberg/pull/11039#discussion_r1943574434 ## bigquery/src/main/java/org/apache/iceberg/gcp/bigquery/BigQueryClientImpl.java: ## @@ -0,0 +1,492 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under

Re: [PR] GCP: Add Iceberg Catalog for GCP BigQuery Metastore [iceberg]

2025-02-05 Thread via GitHub
rdblue commented on code in PR #11039: URL: https://github.com/apache/iceberg/pull/11039#discussion_r1943572308 ## bigquery/src/main/java/org/apache/iceberg/gcp/bigquery/BigQueryClientImpl.java: ## @@ -0,0 +1,492 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under

Re: [PR] GCP: Add Iceberg Catalog for GCP BigQuery Metastore [iceberg]

2025-02-05 Thread via GitHub
rdblue commented on code in PR #11039: URL: https://github.com/apache/iceberg/pull/11039#discussion_r1943571247 ## bigquery/src/main/java/org/apache/iceberg/gcp/bigquery/BigQueryClientImpl.java: ## @@ -0,0 +1,492 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under

Re: [PR] GCP: Add Iceberg Catalog for GCP BigQuery Metastore [iceberg]

2025-02-05 Thread via GitHub
rdblue commented on code in PR #11039: URL: https://github.com/apache/iceberg/pull/11039#discussion_r1943569098 ## bigquery/src/main/java/org/apache/iceberg/gcp/bigquery/BigQueryClientImpl.java: ## @@ -0,0 +1,492 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under

Re: [PR] GCP: Add Iceberg Catalog for GCP BigQuery Metastore [iceberg]

2025-02-05 Thread via GitHub
rdblue commented on code in PR #11039: URL: https://github.com/apache/iceberg/pull/11039#discussion_r1943568625 ## bigquery/src/main/java/org/apache/iceberg/gcp/bigquery/BigQueryClientImpl.java: ## @@ -0,0 +1,492 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under

Re: [PR] GCP: Add Iceberg Catalog for GCP BigQuery Metastore [iceberg]

2025-02-05 Thread via GitHub
rdblue commented on code in PR #11039: URL: https://github.com/apache/iceberg/pull/11039#discussion_r1943567072 ## bigquery/src/main/java/org/apache/iceberg/gcp/bigquery/BigQueryClientImpl.java: ## @@ -0,0 +1,492 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under

Re: [PR] GCP: Add Iceberg Catalog for GCP BigQuery Metastore [iceberg]

2025-02-05 Thread via GitHub
rdblue commented on code in PR #11039: URL: https://github.com/apache/iceberg/pull/11039#discussion_r1943563130 ## bigquery/src/main/java/org/apache/iceberg/gcp/bigquery/metastore/BigQueryMetastoreCatalog.java: ## @@ -0,0 +1,328 @@ +/* + * Licensed to the Apache Software Foundat

Re: [PR] GCP: Add Iceberg Catalog for GCP BigQuery Metastore [iceberg]

2025-01-29 Thread via GitHub
emkornfield commented on PR #11039: URL: https://github.com/apache/iceberg/pull/11039#issuecomment-2623021449 > > @hesham-medhat @rdblue - could you pls give an update on this PR? It seems it would massively simplify the Iceberg table management in GCP for non-spark usecases. > > thanks!

Re: [PR] GCP: Add Iceberg Catalog for GCP BigQuery Metastore [iceberg]

2025-01-06 Thread via GitHub
vrishin-bolt commented on PR #11039: URL: https://github.com/apache/iceberg/pull/11039#issuecomment-2574498242 > @hesham-medhat @rdblue - could you pls give an update on this PR? It seems it would massively simplify the Iceberg table management in GCP for non-spark usecases. > > than

Re: [PR] GCP: Add Iceberg Catalog for GCP BigQuery Metastore [iceberg]

2024-12-21 Thread via GitHub
brunsgaard commented on PR #11039: URL: https://github.com/apache/iceberg/pull/11039#issuecomment-2558261178 This issue is increasingly concerning for the community. GCP intends to introduce Iceberg as a first-class citizen, but it is perplexing that this initiative is continually being pos

Re: [PR] GCP: Add Iceberg Catalog for GCP BigQuery Metastore [iceberg]

2024-12-16 Thread via GitHub
z-kovacs commented on PR #11039: URL: https://github.com/apache/iceberg/pull/11039#issuecomment-2546090559 Hi @hesham-medhat @rdblue - could you give us an update whether this expected to be merged or for some reason this angle has been abandoned by Iceberg / GCP teams? it is a gre

Re: [PR] GCP: Add Iceberg Catalog for GCP BigQuery Metastore [iceberg]

2024-12-15 Thread via GitHub
github-actions[bot] commented on PR #11039: URL: https://github.com/apache/iceberg/pull/11039#issuecomment-2544225522 This pull request has been marked as stale due to 30 days of inactivity. It will be closed in 1 week if no further activity occurs. If you think that’s incorrect or this pul

Re: [PR] GCP: Add Iceberg Catalog for GCP BigQuery Metastore [iceberg]

2024-11-15 Thread via GitHub
hesham-medhat commented on PR #11039: URL: https://github.com/apache/iceberg/pull/11039#issuecomment-2479805549 Thank you all for your enthusiasm! This is close, it's pending @rdblue's final pass/approval. A little while ago he told me he has been busy nevertheless will get to it as soon as

Re: [PR] GCP: Add Iceberg Catalog for GCP BigQuery Metastore [iceberg]

2024-11-15 Thread via GitHub
k-alkiek commented on PR #11039: URL: https://github.com/apache/iceberg/pull/11039#issuecomment-2479396505 This would be a great addition. Love to see the collaboration with GCP. @rdblue, can we get it across the finish line? -- This is an automated message from the Apache Git Service. To

Re: [PR] GCP: Add Iceberg Catalog for GCP BigQuery Metastore [iceberg]

2024-11-14 Thread via GitHub
brunsgaard commented on PR #11039: URL: https://github.com/apache/iceberg/pull/11039#issuecomment-2475686917 @hesham-medhat @rdblue, like @z-kovacs I would also really appreciate an update on this if possible<3 -- This is an automated message from the Apache Git Service. To respond to th

Re: [PR] GCP: Add Iceberg Catalog for GCP BigQuery Metastore [iceberg]

2024-11-13 Thread via GitHub
z-kovacs commented on PR #11039: URL: https://github.com/apache/iceberg/pull/11039#issuecomment-2473076906 @hesham-medhat @rdblue - could you pls give an update on this PR? It seems it would massively simplify the Iceberg table management in GCP for non-spark usecases. thanks! --

Re: [PR] GCP: Add Iceberg Catalog for GCP BigQuery Metastore [iceberg]

2024-11-11 Thread via GitHub
github-actions[bot] commented on PR #11039: URL: https://github.com/apache/iceberg/pull/11039#issuecomment-2469334657 This pull request has been marked as stale due to 30 days of inactivity. It will be closed in 1 week if no further activity occurs. If you think that’s incorrect or this pul

Re: [PR] GCP: Add Iceberg Catalog for GCP BigQuery Metastore [iceberg]

2024-09-11 Thread via GitHub
hesham-medhat commented on code in PR #11039: URL: https://github.com/apache/iceberg/pull/11039#discussion_r1755908398 ## build.gradle: ## @@ -647,20 +647,43 @@ project(':iceberg-gcp') { implementation project(':iceberg-common') implementation project(':iceberg-core')

Re: [PR] GCP: Add Iceberg Catalog for GCP BigQuery Metastore [iceberg]

2024-09-11 Thread via GitHub
hesham-medhat commented on code in PR #11039: URL: https://github.com/apache/iceberg/pull/11039#discussion_r1755903459 ## gcp/src/main/java/org/apache/iceberg/gcp/bigquery/BigQueryClientImpl.java: ## @@ -0,0 +1,481 @@ +/* + * Licensed to the Apache Software Foundation (ASF) unde

Re: [PR] GCP: Add Iceberg Catalog for GCP BigQuery Metastore [iceberg]

2024-09-11 Thread via GitHub
hesham-medhat commented on PR #11039: URL: https://github.com/apache/iceberg/pull/11039#issuecomment-2345020069 > Hi @hesham-medhat , I assume this PR will also enable read/write support for Flink pipelines that use BigLake external tables with BigQuery? Hi @brunsgaard! No. This one i

Re: [PR] GCP: Add Iceberg Catalog for GCP BigQuery Metastore [iceberg]

2024-09-11 Thread via GitHub
rdblue commented on code in PR #11039: URL: https://github.com/apache/iceberg/pull/11039#discussion_r1755571530 ## gcp/src/main/java/org/apache/iceberg/gcp/bigquery/BigQueryClientImpl.java: ## @@ -0,0 +1,481 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one +

Re: [PR] GCP: Add Iceberg Catalog for GCP BigQuery Metastore [iceberg]

2024-09-11 Thread via GitHub
rdblue commented on code in PR #11039: URL: https://github.com/apache/iceberg/pull/11039#discussion_r1755570976 ## gcp/src/main/java/org/apache/iceberg/gcp/bigquery/BigQueryClientImpl.java: ## @@ -0,0 +1,481 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one +

Re: [PR] GCP: Add Iceberg Catalog for GCP BigQuery Metastore [iceberg]

2024-09-11 Thread via GitHub
rdblue commented on code in PR #11039: URL: https://github.com/apache/iceberg/pull/11039#discussion_r1755564516 ## gcp/src/main/java/org/apache/iceberg/gcp/bigquery/BigQueryClientImpl.java: ## @@ -0,0 +1,481 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one +

Re: [PR] GCP: Add Iceberg Catalog for GCP BigQuery Metastore [iceberg]

2024-09-11 Thread via GitHub
rdblue commented on code in PR #11039: URL: https://github.com/apache/iceberg/pull/11039#discussion_r178263 ## build.gradle: ## @@ -647,20 +647,43 @@ project(':iceberg-gcp') { implementation project(':iceberg-common') implementation project(':iceberg-core') +

Re: [PR] GCP: Add Iceberg Catalog for GCP BigQuery Metastore [iceberg]

2024-09-11 Thread via GitHub
brunsgaard commented on PR #11039: URL: https://github.com/apache/iceberg/pull/11039#issuecomment-2344644652 Hi @hesham-medhat , I assume this PR will also enable read/write support for Flink pipelines that use BigLake external tables with BigQuery? -- This is an automated message from

Re: [PR] GCP: Add Iceberg Catalog for GCP BigQuery Metastore [iceberg]

2024-09-11 Thread via GitHub
hesham-medhat commented on code in PR #11039: URL: https://github.com/apache/iceberg/pull/11039#discussion_r1755320886 ## build.gradle: ## @@ -647,20 +647,43 @@ project(':iceberg-gcp') { implementation project(':iceberg-common') implementation project(':iceberg-core')

Re: [PR] GCP: Add Iceberg Catalog for GCP BigQuery Metastore [iceberg]

2024-09-11 Thread via GitHub
hesham-medhat commented on code in PR #11039: URL: https://github.com/apache/iceberg/pull/11039#discussion_r1755322285 ## .gitignore: ## @@ -66,3 +66,6 @@ metastore_db/ # Spark/metastore files spark-warehouse/ derby.log + +# BigQuery/metastore files +gcp/db_folder/ Review Co

Re: [PR] GCP: Add Iceberg Catalog for GCP BigQuery Metastore [iceberg]

2024-09-11 Thread via GitHub
hesham-medhat commented on code in PR #11039: URL: https://github.com/apache/iceberg/pull/11039#discussion_r1755319360 ## build.gradle: ## @@ -647,20 +647,43 @@ project(':iceberg-gcp') { implementation project(':iceberg-common') implementation project(':iceberg-core')

Re: [PR] GCP: Add Iceberg Catalog for GCP BigQuery Metastore [iceberg]

2024-09-10 Thread via GitHub
rdblue commented on code in PR #11039: URL: https://github.com/apache/iceberg/pull/11039#discussion_r1752779009 ## build.gradle: ## @@ -647,20 +647,43 @@ project(':iceberg-gcp') { implementation project(':iceberg-common') implementation project(':iceberg-core') +