Re: [PR] Disable Spark Catalog caching for integration tests [iceberg-python]

2024-03-06 Thread via GitHub
amogh-jahagirdar merged PR #501: URL: https://github.com/apache/iceberg-python/pull/501 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr.

Re: [PR] Disable Spark Catalog caching for integration tests [iceberg-python]

2024-03-06 Thread via GitHub
amogh-jahagirdar commented on PR #501: URL: https://github.com/apache/iceberg-python/pull/501#issuecomment-1982123757 Sweet, thanks @kevinjqliu! I'm going to go ahead and merge this now. -- This is an automated message from the Apache Git Service. To respond to the message, please log on t

Re: [PR] Disable Spark Catalog caching for integration tests [iceberg-python]

2024-03-06 Thread via GitHub
syun64 commented on PR #501: URL: https://github.com/apache/iceberg-python/pull/501#issuecomment-1982096927 Great idea @kevinjqliu ! Thanks for adding this -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above

Re: [PR] Disable Spark Catalog caching for integration tests [iceberg-python]

2024-03-06 Thread via GitHub
kevinjqliu commented on code in PR #501: URL: https://github.com/apache/iceberg-python/pull/501#discussion_r1515301588 ## tests/integration/test_writes.py: ## @@ -355,6 +355,26 @@ def test_data_files(spark: SparkSession, session_catalog: Catalog, arrow_table_w assert [row.

Re: [PR] Disable Spark Catalog caching for integration tests [iceberg-python]

2024-03-06 Thread via GitHub
amogh-jahagirdar commented on code in PR #501: URL: https://github.com/apache/iceberg-python/pull/501#discussion_r1515300598 ## tests/integration/test_writes.py: ## @@ -355,6 +355,26 @@ def test_data_files(spark: SparkSession, session_catalog: Catalog, arrow_table_w assert

Re: [PR] Disable Spark Catalog caching for integration tests [iceberg-python]

2024-03-06 Thread via GitHub
amogh-jahagirdar commented on PR #501: URL: https://github.com/apache/iceberg-python/pull/501#issuecomment-1982063186 Thanks @kevinjqliu I think this change makes sense. I don't think there's ever a reason on the Python side where we want to have the spark caching enabled. On the Iceberg Ja