Re: [PR] feat: add `RollingManifestWriter` [iceberg-python]

2024-07-10 Thread via GitHub
felixscherz commented on code in PR #650: URL: https://github.com/apache/iceberg-python/pull/650#discussion_r1671747269 ## pyiceberg/manifest.py: ## @@ -770,6 +818,81 @@ def add_entry(self, entry: ManifestEntry) -> ManifestWriter: self._writer.write_block([self.prepare

Re: [PR] feat: add `RollingManifestWriter` [iceberg-python]

2024-07-10 Thread via GitHub
felixscherz commented on code in PR #650: URL: https://github.com/apache/iceberg-python/pull/650#discussion_r1671754175 ## pyiceberg/manifest.py: ## @@ -770,6 +818,81 @@ def add_entry(self, entry: ManifestEntry) -> ManifestWriter: self._writer.write_block([self.prepare

Re: [PR] feat: add `RollingManifestWriter` [iceberg-python]

2024-07-08 Thread via GitHub
felixscherz commented on PR #650: URL: https://github.com/apache/iceberg-python/pull/650#issuecomment-2216648246 @Fokko Thanks for taking a look! Sorry about the formatting, should be fixed now:) -- This is an automated message from the Apache Git Service. To respond to the message, pleas

Re: [PR] feat: add `RollingManifestWriter` [iceberg-python]

2024-07-08 Thread via GitHub
Fokko commented on code in PR #650: URL: https://github.com/apache/iceberg-python/pull/650#discussion_r1669196117 ## pyiceberg/manifest.py: ## @@ -770,6 +818,81 @@ def add_entry(self, entry: ManifestEntry) -> ManifestWriter: self._writer.write_block([self.prepare_entry

Re: [PR] feat: add `RollingManifestWriter` [iceberg-python]

2024-07-08 Thread via GitHub
Fokko commented on code in PR #650: URL: https://github.com/apache/iceberg-python/pull/650#discussion_r1669195441 ## pyiceberg/manifest.py: ## @@ -770,6 +818,81 @@ def add_entry(self, entry: ManifestEntry) -> ManifestWriter: self._writer.write_block([self.prepare_entry

Re: [PR] feat: add `RollingManifestWriter` [iceberg-python]

2024-05-04 Thread via GitHub
felixscherz commented on PR #650: URL: https://github.com/apache/iceberg-python/pull/650#issuecomment-2094148537 Hi, I finally had some time to continue working on this. Based on your suggestions @geruh I added a `tell` method to the `OutputStream` protocol that returns the number of

Re: [PR] feat: add `RollingManifestWriter` [iceberg-python]

2024-04-23 Thread via GitHub
felixscherz commented on PR #650: URL: https://github.com/apache/iceberg-python/pull/650#issuecomment-2074101193 Sounds good, I will have a look at the implementation and make a suggestion. Thank you! -- This is an automated message from the Apache Git Service. To respond to the message,

Re: [PR] feat: add `RollingManifestWriter` [iceberg-python]

2024-04-23 Thread via GitHub
geruh commented on PR #650: URL: https://github.com/apache/iceberg-python/pull/650#issuecomment-2073414087 I believe in the Java implementation we have a concept of a [PositionOutputStream](https://github.com/apache/iceberg/blob/866021d7d34f274349ce7de1f29d113395e7f28c/api/src/main/java/org/

[PR] feat: add `RollingManifestWriter` [iceberg-python]

2024-04-23 Thread via GitHub
felixscherz opened a new pull request, #650: URL: https://github.com/apache/iceberg-python/pull/650 Hi, this is in regards to #596 and still WIP. The `RollingManifestWriter` implementation closely follows the java [implementation](https://github.com/apache/iceberg/blob/main/core/src/