Re: [PR] Deprecate DynamoDB Catalog to Reduce Catalog Scope [iceberg]

2024-02-27 Thread via GitHub
odysa commented on PR #9783: URL: https://github.com/apache/iceberg/pull/9783#issuecomment-1968212249 Some catalogs could be community-owned. While the Iceberg team may officially deprecate some catalogs, open-source communities can still maintain them, like @jackye1995 suggested -- T

Re: [PR] Deprecate DynamoDB Catalog to Reduce Catalog Scope [iceberg-python]

2024-02-27 Thread via GitHub
geruh commented on code in PR #475: URL: https://github.com/apache/iceberg-python/pull/475#discussion_r1505155942 ## pyiceberg/catalog/dynamodb.py: ## @@ -81,6 +82,10 @@ ITEM = "Item" +@deprecated( +deprecated_in="0.6.0", Review Comment: Hey @hussein-awala, yeah I a

Re: [PR] Deprecate DynamoDB Catalog to Reduce Catalog Scope [iceberg-python]

2024-02-27 Thread via GitHub
geruh commented on PR #475: URL: https://github.com/apache/iceberg-python/pull/475#issuecomment-1967930755 > Ah, just saw this comment: [apache/iceberg#9783 (comment)](https://github.com/apache/iceberg/pull/9783#issuecomment-1965781361). So maybe we can wait for a while to see how the discu

Re: [PR] Deprecate DynamoDB Catalog to Reduce Catalog Scope [iceberg-python]

2024-02-27 Thread via GitHub
geruh commented on code in PR #475: URL: https://github.com/apache/iceberg-python/pull/475#discussion_r1505155942 ## pyiceberg/catalog/dynamodb.py: ## @@ -81,6 +82,10 @@ ITEM = "Item" +@deprecated( +deprecated_in="0.6.0", Review Comment: Hey @hussein-awala, yeah we

Re: [PR] Deprecate DynamoDB Catalog to Reduce Catalog Scope [iceberg-python]

2024-02-27 Thread via GitHub
hussein-awala commented on code in PR #475: URL: https://github.com/apache/iceberg-python/pull/475#discussion_r1505068598 ## pyiceberg/catalog/dynamodb.py: ## @@ -81,6 +82,10 @@ ITEM = "Item" +@deprecated( +deprecated_in="0.6.0", Review Comment: `0.6.0` was already

Re: [PR] Deprecate DynamoDB Catalog to Reduce Catalog Scope [iceberg-python]

2024-02-27 Thread via GitHub
geruh commented on code in PR #475: URL: https://github.com/apache/iceberg-python/pull/475#discussion_r1504938114 ## pyiceberg/catalog/dynamodb.py: ## @@ -81,6 +82,10 @@ ITEM = "Item" +@deprecated( +deprecated_in="0.6.0", +removed_in="1.0.0", +) Review Comment:

Re: [PR] Deprecate DynamoDB Catalog to Reduce Catalog Scope [iceberg]

2024-02-27 Thread via GitHub
amogh-jahagirdar commented on PR #9783: URL: https://github.com/apache/iceberg/pull/9783#issuecomment-1967496545 Yeah I think it's important to emphasize that deprecating in this repo, and removing it in 2.0 does not mean DynamoDBCatalog cannot exist in some form in some other repo.

Re: [PR] Deprecate DynamoDB Catalog to Reduce Catalog Scope [iceberg]

2024-02-27 Thread via GitHub
jackye1995 commented on PR #9783: URL: https://github.com/apache/iceberg/pull/9783#issuecomment-1967478372 @SreeramGarlapati @namrathamyske we discussed specifically about Dynamo in that community sync that we want to deprecate it directly. Let us talk more on the devlist about this.

Re: [PR] Deprecate DynamoDB Catalog to Reduce Catalog Scope [iceberg]

2024-02-27 Thread via GitHub
ajantha-bhat commented on PR #9783: URL: https://github.com/apache/iceberg/pull/9783#issuecomment-1966383303 For time being I have reverted the DynamoDb catalog deprecation from the code and prepared RC4. Let us discuss more on mailing list and conclude whether it needs to be deprecated or

Re: [PR] Deprecate DynamoDB Catalog to Reduce Catalog Scope [iceberg]

2024-02-26 Thread via GitHub
ajantha-bhat commented on PR #9783: URL: https://github.com/apache/iceberg/pull/9783#issuecomment-1965932224 @SreeramGarlapati , @namrathamyske: Can you also comment that salesforce is using it and mention the problems with Glue catalog in details as a reply to the mailing list discussion?

Re: [PR] Deprecate DynamoDB Catalog to Reduce Catalog Scope [iceberg]

2024-02-26 Thread via GitHub
namrathamyske commented on PR #9783: URL: https://github.com/apache/iceberg/pull/9783#issuecomment-1965884032 @geruh it was decided in sync to move all catalogs to another repo if i am not wrong, can we do that instead of removing the catalog altogether -- This is an automated message fro

Re: [PR] Deprecate DynamoDB Catalog to Reduce Catalog Scope [iceberg-python]

2024-02-26 Thread via GitHub
HonahX commented on PR #475: URL: https://github.com/apache/iceberg-python/pull/475#issuecomment-1965871433 Ah, just saw this comment: https://github.com/apache/iceberg/pull/9783#issuecomment-1965781361. So maybe we can wait for a while to see how the discussion goes, -- This is an autom

Re: [PR] Deprecate DynamoDB Catalog to Reduce Catalog Scope [iceberg-python]

2024-02-26 Thread via GitHub
HonahX commented on code in PR #475: URL: https://github.com/apache/iceberg-python/pull/475#discussion_r1503700463 ## pyiceberg/catalog/dynamodb.py: ## @@ -81,6 +82,10 @@ ITEM = "Item" +@deprecated( +deprecated_in="0.6.0", +removed_in="1.0.0", +) Review Comment:

Re: [PR] Deprecate DynamoDB Catalog to Reduce Catalog Scope [iceberg]

2024-02-26 Thread via GitHub
SreeramGarlapati commented on PR #9783: URL: https://github.com/apache/iceberg/pull/9783#issuecomment-1965781361 @geruh - we (Salesforce DataCloud team) use DynamoDB catalog. AWS is the biggest cloud out there & dynamoDb is the most reliable aws's no-sql db as mgd. svc. we started with

[PR] Deprecate DynamoDB Catalog to Reduce Catalog Scope [iceberg-python]

2024-02-26 Thread via GitHub
geruh opened a new pull request, #475: URL: https://github.com/apache/iceberg-python/pull/475 The deprecation of the dynamo catalog in python is stemming from the discussion in the [community sync](https://youtu.be/uAQVGd5zV4I?si=cj0xpfprgvJIGYIm&t=1323), we want to reduce the scope of sup

Re: [PR] Deprecate DynamoDB Catalog to Reduce Catalog Scope [iceberg]

2024-02-23 Thread via GitHub
jackye1995 commented on PR #9783: URL: https://github.com/apache/iceberg/pull/9783#issuecomment-1961768557 I see, let's do that quickly @geruh -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

Re: [PR] Deprecate DynamoDB Catalog to Reduce Catalog Scope [iceberg]

2024-02-23 Thread via GitHub
nastra commented on PR #9783: URL: https://github.com/apache/iceberg/pull/9783#issuecomment-1961712021 That might work too. Also do we want to update `aws.md` since it has some examples with that catalog? -- This is an automated message from the Apache Git Service. To respond to the messa

Re: [PR] Deprecate DynamoDB Catalog to Reduce Catalog Scope [iceberg]

2024-02-23 Thread via GitHub
jackye1995 commented on PR #9783: URL: https://github.com/apache/iceberg/pull/9783#issuecomment-1961703475 Oh sorry I totally missed that comment. I think we can put another PR to change the comment to say deprecated since 1.5.0, what do you think? -- This is an automated message from the

Re: [PR] Deprecate DynamoDB Catalog to Reduce Catalog Scope [iceberg]

2024-02-23 Thread via GitHub
nastra commented on PR #9783: URL: https://github.com/apache/iceberg/pull/9783#issuecomment-1961679888 @jackye1995 as I mentioned we should be holding off the merge until 1.5.0 is out. If we do another RC, then this diff will be included and I'm not sure this is what we wanted -- This is

Re: [PR] Deprecate DynamoDB Catalog to Reduce Catalog Scope [iceberg]

2024-02-23 Thread via GitHub
jackye1995 merged PR #9783: URL: https://github.com/apache/iceberg/pull/9783 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.

Re: [PR] Deprecate DynamoDB Catalog to Reduce Catalog Scope [iceberg]

2024-02-23 Thread via GitHub
jackye1995 commented on PR #9783: URL: https://github.com/apache/iceberg/pull/9783#issuecomment-1961650652 Thanks for the help, merging! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specif

Re: [PR] Deprecate DynamoDB Catalog to Reduce Catalog Scope [iceberg]

2024-02-22 Thread via GitHub
jackye1995 commented on PR #9783: URL: https://github.com/apache/iceberg/pull/9783#issuecomment-1960741059 Side note, we should also deprecate the catalog in python, probably in a separated PR. -- This is an automated message from the Apache Git Service. To respond to the message, please

Re: [PR] Deprecate DynamoDB Catalog to Reduce Catalog Scope [iceberg]

2024-02-22 Thread via GitHub
ajantha-bhat commented on code in PR #9783: URL: https://github.com/apache/iceberg/pull/9783#discussion_r1500113028 ## aws/src/main/java/org/apache/iceberg/aws/dynamodb/DynamoDbTableOperations.java: ## @@ -44,6 +44,8 @@ import software.amazon.awssdk.services.dynamodb.model.PutI

Re: [PR] Deprecate DynamoDB Catalog to Reduce Catalog Scope [iceberg]

2024-02-22 Thread via GitHub
ajantha-bhat commented on code in PR #9783: URL: https://github.com/apache/iceberg/pull/9783#discussion_r1500113608 ## aws/src/main/java/org/apache/iceberg/aws/dynamodb/DynamoDbCatalog.java: ## @@ -84,7 +84,12 @@ import software.amazon.awssdk.services.dynamodb.model.TransactWr

Re: [PR] Deprecate DynamoDB Catalog to Reduce Catalog Scope [iceberg]

2024-02-22 Thread via GitHub
ajantha-bhat commented on code in PR #9783: URL: https://github.com/apache/iceberg/pull/9783#discussion_r1500112881 ## aws/src/main/java/org/apache/iceberg/aws/dynamodb/DynamoDbCatalog.java: ## @@ -84,7 +84,12 @@ import software.amazon.awssdk.services.dynamodb.model.TransactWr

Re: [PR] Deprecate DynamoDB Catalog to Reduce Catalog Scope [iceberg]

2024-02-22 Thread via GitHub
ajantha-bhat commented on code in PR #9783: URL: https://github.com/apache/iceberg/pull/9783#discussion_r1500113028 ## aws/src/main/java/org/apache/iceberg/aws/dynamodb/DynamoDbTableOperations.java: ## @@ -44,6 +44,8 @@ import software.amazon.awssdk.services.dynamodb.model.PutI

Re: [PR] Deprecate DynamoDB Catalog to Reduce Catalog Scope [iceberg]

2024-02-22 Thread via GitHub
ajantha-bhat commented on code in PR #9783: URL: https://github.com/apache/iceberg/pull/9783#discussion_r1500112881 ## aws/src/main/java/org/apache/iceberg/aws/dynamodb/DynamoDbCatalog.java: ## @@ -84,7 +84,12 @@ import software.amazon.awssdk.services.dynamodb.model.TransactWr

Re: [PR] Deprecate DynamoDB Catalog to Reduce Catalog Scope [iceberg]

2024-02-22 Thread via GitHub
amogh-jahagirdar commented on code in PR #9783: URL: https://github.com/apache/iceberg/pull/9783#discussion_r1500025586 ## aws/src/main/java/org/apache/iceberg/aws/dynamodb/DynamoDbCatalog.java: ## @@ -84,7 +84,12 @@ import software.amazon.awssdk.services.dynamodb.model.Transa

Re: [PR] Deprecate DynamoDB Catalog to Reduce Catalog Scope [iceberg]

2024-02-22 Thread via GitHub
geruh commented on code in PR #9783: URL: https://github.com/apache/iceberg/pull/9783#discussion_r1499969849 ## aws/src/main/java/org/apache/iceberg/aws/dynamodb/DynamoDbCatalog.java: ## @@ -84,7 +84,12 @@ import software.amazon.awssdk.services.dynamodb.model.TransactWriteItem

[PR] Deprecate DynamoDB Catalog to Reduce Catalog Scope [iceberg]

2024-02-22 Thread via GitHub
geruh opened a new pull request, #9783: URL: https://github.com/apache/iceberg/pull/9783 As discussed in the [community sync](https://youtu.be/uAQVGd5zV4I?si=cj0xpfprgvJIGYIm&t=1323), we want to reduce the scope of supported catalogs in Iceberg. There are currently many options, which crea