Re: [PR] Construct a writer tree [iceberg-python]

Fokko merged PR #40: URL: https://github.com/apache/iceberg-python/pull/40 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.ap

Re: [PR] Construct a writer tree [iceberg-python]

Fokko commented on code in PR #40: URL: https://github.com/apache/iceberg-python/pull/40#discussion_r1354399467 ## pyiceberg/avro/resolver.py: ## @@ -233,7 +256,95 @@ def skip(self, decoder: BinaryDecoder) -> None: pass -class SchemaResolver(PrimitiveWithPartnerVisi

Re: [PR] Construct a writer tree [iceberg-python]

Fokko commented on PR #40: URL: https://github.com/apache/iceberg-python/pull/40#issuecomment-1757087652 > We should fix this, but I'm going to approve this to unblock it since everything else looks correct. Definitely! Thanks for reviewing, and excited to get this in 🚀 -- This is

Re: [PR] Construct a writer tree [iceberg-python]

rdblue commented on PR #40: URL: https://github.com/apache/iceberg-python/pull/40#issuecomment-1756223971 I think overall this is ready. There's one bug where a missing record field produces a double `OptionWriter`, but it wouldn't actually result in bad data because the outer option will a

Re: [PR] Construct a writer tree [iceberg-python]

rdblue commented on code in PR #40: URL: https://github.com/apache/iceberg-python/pull/40#discussion_r1353347354 ## tests/avro/test_resolver.py: ## @@ -293,11 +306,103 @@ def test_resolver_initial_value() -> None: schema_id=2, ) -assert resolve(write_schema,

Re: [PR] Construct a writer tree [iceberg-python]

rdblue commented on code in PR #40: URL: https://github.com/apache/iceberg-python/pull/40#discussion_r1353340562 ## pyiceberg/avro/resolver.py: ## @@ -192,7 +194,28 @@ def visit_binary(self, binary_type: BinaryType) -> Writer: return BinaryWriter() -def resolve( +CO

Re: [PR] Construct a writer tree [iceberg-python]

rdblue commented on code in PR #40: URL: https://github.com/apache/iceberg-python/pull/40#discussion_r135477 ## pyiceberg/avro/resolver.py: ## @@ -233,7 +255,107 @@ def skip(self, decoder: BinaryDecoder) -> None: pass -class SchemaResolver(PrimitiveWithPartnerVi

Re: [PR] Construct a writer tree [iceberg-python]

rdblue commented on code in PR #40: URL: https://github.com/apache/iceberg-python/pull/40#discussion_r1353328176 ## pyiceberg/avro/resolver.py: ## @@ -233,7 +256,95 @@ def skip(self, decoder: BinaryDecoder) -> None: pass -class SchemaResolver(PrimitiveWithPartnerVis

Re: [PR] Construct a writer tree [iceberg-python]

Fokko commented on PR #40: URL: https://github.com/apache/iceberg-python/pull/40#issuecomment-1755767706 Forgot to push, just pushed the latest changes -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to g

Re: [PR] Construct a writer tree [iceberg-python]

Fokko commented on code in PR #40: URL: https://github.com/apache/iceberg-python/pull/40#discussion_r1352346498 ## pyiceberg/avro/resolver.py: ## @@ -233,7 +256,93 @@ def skip(self, decoder: BinaryDecoder) -> None: pass -class SchemaResolver(PrimitiveWithPartnerVisi

Re: [PR] Construct a writer tree [iceberg-python]

Fokko commented on code in PR #40: URL: https://github.com/apache/iceberg-python/pull/40#discussion_r1352317115 ## pyiceberg/avro/resolver.py: ## @@ -233,7 +256,93 @@ def skip(self, decoder: BinaryDecoder) -> None: pass -class SchemaResolver(PrimitiveWithPartnerVisi

Re: [PR] Construct a writer tree [iceberg-python]

Fokko commented on code in PR #40: URL: https://github.com/apache/iceberg-python/pull/40#discussion_r1352317115 ## pyiceberg/avro/resolver.py: ## @@ -233,7 +256,93 @@ def skip(self, decoder: BinaryDecoder) -> None: pass -class SchemaResolver(PrimitiveWithPartnerVisi

Re: [PR] Construct a writer tree [iceberg-python]

Fokko commented on code in PR #40: URL: https://github.com/apache/iceberg-python/pull/40#discussion_r1352318212 ## pyiceberg/avro/resolver.py: ## @@ -233,7 +256,93 @@ def skip(self, decoder: BinaryDecoder) -> None: pass -class SchemaResolver(PrimitiveWithPartnerVisi

Re: [PR] Construct a writer tree [iceberg-python]

Fokko commented on code in PR #40: URL: https://github.com/apache/iceberg-python/pull/40#discussion_r1352299345 ## pyiceberg/avro/resolver.py: ## @@ -192,7 +195,26 @@ def visit_binary(self, binary_type: BinaryType) -> Writer: return BinaryWriter() -def resolve( +CON

Re: [PR] Construct a writer tree [iceberg-python]

Fokko commented on code in PR #40: URL: https://github.com/apache/iceberg-python/pull/40#discussion_r1352300965 ## pyiceberg/avro/resolver.py: ## @@ -192,7 +194,28 @@ def visit_binary(self, binary_type: BinaryType) -> Writer: return BinaryWriter() -def resolve( +CON

Re: [PR] Construct a writer tree [iceberg-python]

Fokko commented on code in PR #40: URL: https://github.com/apache/iceberg-python/pull/40#discussion_r1352299345 ## pyiceberg/avro/resolver.py: ## @@ -192,7 +195,26 @@ def visit_binary(self, binary_type: BinaryType) -> Writer: return BinaryWriter() -def resolve( +CON

Re: [PR] Construct a writer tree [iceberg-python]

Fokko commented on code in PR #40: URL: https://github.com/apache/iceberg-python/pull/40#discussion_r1352295604 ## pyiceberg/avro/resolver.py: ## @@ -233,7 +255,107 @@ def skip(self, decoder: BinaryDecoder) -> None: pass -class SchemaResolver(PrimitiveWithPartnerVis

Re: [PR] Construct a writer tree [iceberg-python]

Fokko commented on code in PR #40: URL: https://github.com/apache/iceberg-python/pull/40#discussion_r1351918311 ## pyiceberg/avro/resolver.py: ## @@ -233,7 +255,107 @@ def skip(self, decoder: BinaryDecoder) -> None: pass -class SchemaResolver(PrimitiveWithPartnerVis

Re: [PR] Construct a writer tree [iceberg-python]

rdblue commented on code in PR #40: URL: https://github.com/apache/iceberg-python/pull/40#discussion_r1351007903 ## pyiceberg/avro/resolver.py: ## @@ -233,7 +256,93 @@ def skip(self, decoder: BinaryDecoder) -> None: pass -class SchemaResolver(PrimitiveWithPartnerVis

Re: [PR] Construct a writer tree [iceberg-python]

rdblue commented on code in PR #40: URL: https://github.com/apache/iceberg-python/pull/40#discussion_r1351008344 ## pyiceberg/avro/resolver.py: ## @@ -233,7 +256,93 @@ def skip(self, decoder: BinaryDecoder) -> None: pass -class SchemaResolver(PrimitiveWithPartnerVis

Re: [PR] Construct a writer tree [iceberg-python]

rdblue commented on code in PR #40: URL: https://github.com/apache/iceberg-python/pull/40#discussion_r1351007903 ## pyiceberg/avro/resolver.py: ## @@ -233,7 +256,93 @@ def skip(self, decoder: BinaryDecoder) -> None: pass -class SchemaResolver(PrimitiveWithPartnerVis

Re: [PR] Construct a writer tree [iceberg-python]

rdblue commented on code in PR #40: URL: https://github.com/apache/iceberg-python/pull/40#discussion_r1351010254 ## pyiceberg/avro/resolver.py: ## @@ -233,7 +256,93 @@ def skip(self, decoder: BinaryDecoder) -> None: pass -class SchemaResolver(PrimitiveWithPartnerVis

Re: [PR] Construct a writer tree [iceberg-python]

rdblue commented on code in PR #40: URL: https://github.com/apache/iceberg-python/pull/40#discussion_r1351008344 ## pyiceberg/avro/resolver.py: ## @@ -233,7 +256,93 @@ def skip(self, decoder: BinaryDecoder) -> None: pass -class SchemaResolver(PrimitiveWithPartnerVis

Re: [PR] Construct a writer tree [iceberg-python]

rdblue commented on code in PR #40: URL: https://github.com/apache/iceberg-python/pull/40#discussion_r1351007903 ## pyiceberg/avro/resolver.py: ## @@ -233,7 +256,93 @@ def skip(self, decoder: BinaryDecoder) -> None: pass -class SchemaResolver(PrimitiveWithPartnerVis

Re: [PR] Construct a writer tree [iceberg-python]

rdblue commented on code in PR #40: URL: https://github.com/apache/iceberg-python/pull/40#discussion_r1351006286 ## pyiceberg/avro/resolver.py: ## @@ -192,7 +194,28 @@ def visit_binary(self, binary_type: BinaryType) -> Writer: return BinaryWriter() -def resolve( +CO

Re: [PR] Construct a writer tree [iceberg-python]

rdblue commented on code in PR #40: URL: https://github.com/apache/iceberg-python/pull/40#discussion_r1351005850 ## pyiceberg/avro/resolver.py: ## @@ -192,7 +195,26 @@ def visit_binary(self, binary_type: BinaryType) -> Writer: return BinaryWriter() -def resolve( +CO

Re: [PR] Construct a writer tree [iceberg-python]

rdblue commented on code in PR #40: URL: https://github.com/apache/iceberg-python/pull/40#discussion_r1351005623 ## pyiceberg/avro/resolver.py: ## @@ -192,7 +195,26 @@ def visit_binary(self, binary_type: BinaryType) -> Writer: return BinaryWriter() -def resolve( +CO

Re: [PR] Construct a writer tree [iceberg-python]

rdblue commented on code in PR #40: URL: https://github.com/apache/iceberg-python/pull/40#discussion_r1351003830 ## pyiceberg/avro/resolver.py: ## @@ -233,7 +255,107 @@ def skip(self, decoder: BinaryDecoder) -> None: pass -class SchemaResolver(PrimitiveWithPartnerVi

Re: [PR] Construct a writer tree [iceberg-python]

Fokko commented on code in PR #40: URL: https://github.com/apache/iceberg-python/pull/40#discussion_r1350771798 ## pyiceberg/avro/resolver.py: ## @@ -233,7 +255,107 @@ def skip(self, decoder: BinaryDecoder) -> None: pass -class SchemaResolver(PrimitiveWithPartnerVis

Re: [PR] Construct a writer tree [iceberg-python]

Fokko commented on code in PR #40: URL: https://github.com/apache/iceberg-python/pull/40#discussion_r1350771798 ## pyiceberg/avro/resolver.py: ## @@ -233,7 +255,107 @@ def skip(self, decoder: BinaryDecoder) -> None: pass -class SchemaResolver(PrimitiveWithPartnerVis

Re: [PR] Construct a writer tree [iceberg-python]

Fokko commented on code in PR #40: URL: https://github.com/apache/iceberg-python/pull/40#discussion_r1350771798 ## pyiceberg/avro/resolver.py: ## @@ -233,7 +255,107 @@ def skip(self, decoder: BinaryDecoder) -> None: pass -class SchemaResolver(PrimitiveWithPartnerVis

Re: [PR] Construct a writer tree [iceberg-python]

Fokko commented on code in PR #40: URL: https://github.com/apache/iceberg-python/pull/40#discussion_r1349777648 ## pyiceberg/manifest.py: ## @@ -603,10 +691,26 @@ def __exit__( def content(self) -> ManifestContent: ... +@property @abstractmethod -def

Re: [PR] Construct a writer tree [iceberg-python]

Fokko commented on code in PR #40: URL: https://github.com/apache/iceberg-python/pull/40#discussion_r1349777193 ## pyiceberg/manifest.py: ## @@ -99,101 +100,185 @@ def __repr__(self) -> str: return f"FileFormat.{self.name}" -DATA_FILE_TYPE_V1 = StructType( -Nest

Re: [PR] Construct a writer tree [iceberg-python]

Fokko commented on code in PR #40: URL: https://github.com/apache/iceberg-python/pull/40#discussion_r1349777115 ## pyiceberg/avro/writer.py: ## @@ -49,58 +51,69 @@ def __repr__(self) -> str: return f"{self.__class__.__name__}()" +@dataclass(frozen=True) class NoneW

Re: [PR] Construct a writer tree [iceberg-python]

Fokko commented on code in PR #40: URL: https://github.com/apache/iceberg-python/pull/40#discussion_r1349777029 ## pyiceberg/avro/resolver.py: ## @@ -233,7 +255,107 @@ def skip(self, decoder: BinaryDecoder) -> None: pass -class SchemaResolver(PrimitiveWithPartnerVis

Re: [PR] Construct a writer tree [iceberg-python]

Fokko commented on code in PR #40: URL: https://github.com/apache/iceberg-python/pull/40#discussion_r1349776857 ## pyiceberg/avro/resolver.py: ## @@ -233,7 +255,107 @@ def skip(self, decoder: BinaryDecoder) -> None: pass -class SchemaResolver(PrimitiveWithPartnerVis

Re: [PR] Construct a writer tree [iceberg-python]

Fokko commented on code in PR #40: URL: https://github.com/apache/iceberg-python/pull/40#discussion_r1349776729 ## pyiceberg/avro/resolver.py: ## @@ -233,7 +255,107 @@ def skip(self, decoder: BinaryDecoder) -> None: pass -class SchemaResolver(PrimitiveWithPartnerVis

Re: [PR] Construct a writer tree [iceberg-python]

Fokko commented on code in PR #40: URL: https://github.com/apache/iceberg-python/pull/40#discussion_r1349776586 ## pyiceberg/avro/resolver.py: ## @@ -233,7 +255,107 @@ def skip(self, decoder: BinaryDecoder) -> None: pass -class SchemaResolver(PrimitiveWithPartnerVis

Re: [PR] Construct a writer tree [iceberg-python]

Fokko commented on code in PR #40: URL: https://github.com/apache/iceberg-python/pull/40#discussion_r1349776456 ## pyiceberg/avro/resolver.py: ## @@ -233,7 +255,107 @@ def skip(self, decoder: BinaryDecoder) -> None: pass -class SchemaResolver(PrimitiveWithPartnerVis

Re: [PR] Construct a writer tree [iceberg-python]

Fokko commented on code in PR #40: URL: https://github.com/apache/iceberg-python/pull/40#discussion_r1349776424 ## pyiceberg/avro/resolver.py: ## @@ -233,7 +255,107 @@ def skip(self, decoder: BinaryDecoder) -> None: pass -class SchemaResolver(PrimitiveWithPartnerVis

Re: [PR] Construct a writer tree [iceberg-python]

rdblue commented on code in PR #40: URL: https://github.com/apache/iceberg-python/pull/40#discussion_r1349767535 ## pyiceberg/avro/resolver.py: ## @@ -233,7 +255,107 @@ def skip(self, decoder: BinaryDecoder) -> None: pass -class SchemaResolver(PrimitiveWithPartnerVi

Re: [PR] Construct a writer tree [iceberg-python]

rdblue commented on code in PR #40: URL: https://github.com/apache/iceberg-python/pull/40#discussion_r1349766913 ## pyiceberg/manifest.py: ## @@ -603,10 +691,26 @@ def __exit__( def content(self) -> ManifestContent: ... +@property @abstractmethod -de

Re: [PR] Construct a writer tree [iceberg-python]

rdblue commented on code in PR #40: URL: https://github.com/apache/iceberg-python/pull/40#discussion_r1349766166 ## pyiceberg/avro/resolver.py: ## @@ -192,7 +195,26 @@ def visit_binary(self, binary_type: BinaryType) -> Writer: return BinaryWriter() -def resolve( +CO

Re: [PR] Construct a writer tree [iceberg-python]

rdblue commented on code in PR #40: URL: https://github.com/apache/iceberg-python/pull/40#discussion_r1349766027 ## pyiceberg/avro/resolver.py: ## @@ -233,7 +255,107 @@ def skip(self, decoder: BinaryDecoder) -> None: pass -class SchemaResolver(PrimitiveWithPartnerVi

Re: [PR] Construct a writer tree [iceberg-python]

rdblue commented on code in PR #40: URL: https://github.com/apache/iceberg-python/pull/40#discussion_r1349765776 ## pyiceberg/avro/resolver.py: ## @@ -233,7 +255,107 @@ def skip(self, decoder: BinaryDecoder) -> None: pass -class SchemaResolver(PrimitiveWithPartnerVi

Re: [PR] Construct a writer tree [iceberg-python]

rdblue commented on code in PR #40: URL: https://github.com/apache/iceberg-python/pull/40#discussion_r1349764876 ## pyiceberg/manifest.py: ## @@ -99,101 +100,185 @@ def __repr__(self) -> str: return f"FileFormat.{self.name}" -DATA_FILE_TYPE_V1 = StructType( -Nes

Re: [PR] Construct a writer tree [iceberg-python]

rdblue commented on code in PR #40: URL: https://github.com/apache/iceberg-python/pull/40#discussion_r1349764721 ## pyiceberg/avro/resolver.py: ## @@ -233,7 +255,107 @@ def skip(self, decoder: BinaryDecoder) -> None: pass -class SchemaResolver(PrimitiveWithPartnerVi

Re: [PR] Construct a writer tree [iceberg-python]

rdblue commented on code in PR #40: URL: https://github.com/apache/iceberg-python/pull/40#discussion_r1349764258 ## pyiceberg/avro/resolver.py: ## @@ -233,7 +255,107 @@ def skip(self, decoder: BinaryDecoder) -> None: pass -class SchemaResolver(PrimitiveWithPartnerVi

Re: [PR] Construct a writer tree [iceberg-python]

rdblue commented on code in PR #40: URL: https://github.com/apache/iceberg-python/pull/40#discussion_r1349764107 ## pyiceberg/avro/resolver.py: ## @@ -233,7 +255,107 @@ def skip(self, decoder: BinaryDecoder) -> None: pass -class SchemaResolver(PrimitiveWithPartnerVi

Re: [PR] Construct a writer tree [iceberg-python]

rdblue commented on code in PR #40: URL: https://github.com/apache/iceberg-python/pull/40#discussion_r1349763567 ## pyiceberg/avro/writer.py: ## @@ -49,58 +51,69 @@ def __repr__(self) -> str: return f"{self.__class__.__name__}()" +@dataclass(frozen=True) class None

Re: [PR] Construct a writer tree [iceberg-python]

rdblue commented on code in PR #40: URL: https://github.com/apache/iceberg-python/pull/40#discussion_r1349762706 ## pyiceberg/avro/resolver.py: ## @@ -233,7 +255,107 @@ def skip(self, decoder: BinaryDecoder) -> None: pass -class SchemaResolver(PrimitiveWithPartnerVi

Re: [PR] Construct a writer tree [iceberg-python]

rdblue commented on code in PR #40: URL: https://github.com/apache/iceberg-python/pull/40#discussion_r1349762706 ## pyiceberg/avro/resolver.py: ## @@ -233,7 +255,107 @@ def skip(self, decoder: BinaryDecoder) -> None: pass -class SchemaResolver(PrimitiveWithPartnerVi

Re: [PR] Construct a writer tree [iceberg-python]

rdblue commented on code in PR #40: URL: https://github.com/apache/iceberg-python/pull/40#discussion_r1349762539 ## pyiceberg/avro/resolver.py: ## @@ -233,7 +255,107 @@ def skip(self, decoder: BinaryDecoder) -> None: pass -class SchemaResolver(PrimitiveWithPartnerVi

Re: [PR] Construct a writer tree [iceberg-python]

rdblue commented on code in PR #40: URL: https://github.com/apache/iceberg-python/pull/40#discussion_r1349762135 ## pyiceberg/avro/resolver.py: ## @@ -233,7 +255,107 @@ def skip(self, decoder: BinaryDecoder) -> None: pass -class SchemaResolver(PrimitiveWithPartnerVi

Re: [PR] Construct a writer tree [iceberg-python]

rdblue commented on code in PR #40: URL: https://github.com/apache/iceberg-python/pull/40#discussion_r1349761999 ## pyiceberg/avro/resolver.py: ## @@ -233,7 +255,107 @@ def skip(self, decoder: BinaryDecoder) -> None: pass -class SchemaResolver(PrimitiveWithPartnerVi

Re: [PR] Construct a writer tree [iceberg-python]

Fokko commented on code in PR #40: URL: https://github.com/apache/iceberg-python/pull/40#discussion_r1345991299 ## pyiceberg/manifest.py: ## @@ -262,15 +346,13 @@ class DataFile(Record): "split_offsets", "equality_ids", "sort_order_id", -"spec_

Re: [PR] Construct a writer tree [iceberg-python]

Fokko commented on code in PR #40: URL: https://github.com/apache/iceberg-python/pull/40#discussion_r1345991299 ## pyiceberg/manifest.py: ## @@ -262,15 +346,13 @@ class DataFile(Record): "split_offsets", "equality_ids", "sort_order_id", -"spec_