Re: [I] Support native Pydantic schemas [iceberg-python]

2025-05-09 Thread via GitHub
jim-ngoo commented on issue #1934: URL: https://github.com/apache/iceberg-python/issues/1934#issuecomment-2865539528 > [@jim-ngoo](https://github.com/jim-ngoo) so from Dx perspective we should just pass in a pydantic schema as e.g. `catalog.create_table('mytable', schema=MyPydanticModel)`?

Re: [I] Support native Pydantic schemas [iceberg-python]

2025-05-05 Thread via GitHub
potatochipcoconut commented on issue #1934: URL: https://github.com/apache/iceberg-python/issues/1934#issuecomment-2851356331 @jim-ngoo so from Dx perspective we should just pass in a pydantic schema as e.g. `catalog.create_table('mytable', schema=MyPydanticModel)`? That would be nice --

Re: [I] Support native Pydantic schemas [iceberg-python]

2025-05-02 Thread via GitHub
jim-ngoo commented on issue #1934: URL: https://github.com/apache/iceberg-python/issues/1934#issuecomment-2846711694 > Type mapping is notoriously difficult. Since we already support bidirectional interchange between pyarrow schema and iceberg schema, i think it would be easier if we can m

Re: [I] Support native Pydantic schemas [iceberg-python]

2025-04-25 Thread via GitHub
potatochipcoconut commented on issue #1934: URL: https://github.com/apache/iceberg-python/issues/1934#issuecomment-2831447185 @choucavalier started playing with your lib and seems like generally what I was looking for, ran into issues around UUID (opened issues there and duckdb). Loc

Re: [I] Support native Pydantic schemas [iceberg-python]

2025-04-20 Thread via GitHub
choucavalier commented on issue #1934: URL: https://github.com/apache/iceberg-python/issues/1934#issuecomment-2817036701 Pydantic, TypedDict, dataclasses, JSON schema, etc. There are many formats that could potentially generate iceberg schemas. It's not entirely obvious to me that pyiceber

Re: [I] Support native Pydantic schemas [iceberg-python]

2025-04-19 Thread via GitHub
kevinjqliu commented on issue #1934: URL: https://github.com/apache/iceberg-python/issues/1934#issuecomment-2816805023 Type mapping is notoriously difficult. Since we already support bidirectional interchange between pyarrow schema and iceberg schema, i think it would be easier if we can m

Re: [I] Support native Pydantic schemas [iceberg-python]

2025-04-19 Thread via GitHub
potatochipcoconut commented on issue #1934: URL: https://github.com/apache/iceberg-python/issues/1934#issuecomment-2816728751 @choucavalier agree it seems like a natural enhancement, possibly should be available directly through pyiceberg? -- This is an automated message from the Apache

Re: [I] Support native Pydantic schemas [iceberg-python]

2025-04-19 Thread via GitHub
choucavalier commented on issue #1934: URL: https://github.com/apache/iceberg-python/issues/1934#issuecomment-2816600466 You can look at https://github.com/simw/pydantic-to-pyarrow for inspiration I think this is the future. being able to manipulate / validate objects individually, t

Re: [I] Support native Pydantic schemas [iceberg-python]

2025-04-18 Thread via GitHub
potatochipcoconut commented on issue #1934: URL: https://github.com/apache/iceberg-python/issues/1934#issuecomment-2816103494 Came up with naive attempt, open to feedback. Not sure how it would handle e.g. int vs long, float vs double, etc ``` import builtins import datetime imp

[I] Support native Pydantic schemas [iceberg-python]

2025-04-18 Thread via GitHub
potatochipcoconut opened a new issue, #1934: URL: https://github.com/apache/iceberg-python/issues/1934 ### Feature Request / Improvement How difficult or feasible would it be to enable support for native Pydantic schemas that automatically map to the underlying pyiceberg types etc?