kevinjqliu commented on code in PR #305:
URL: https://github.com/apache/iceberg-python/pull/305#discussion_r1468547838
##########
tests/io/test_pyarrow_visitor.py:
##########
@@ -572,3 +477,15 @@ def
test_pyarrow_schema_to_schema_missing_ids_using_name_mapping_nested_missing_
with pytest.raises(ValueError) as exc_info:
_ = pyarrow_to_schema(schema, name_mapping)
assert "Could not find field with name: quux.value.key" in
str(exc_info.value)
+
+
+def test_pyarrow_schema_to_schema_fresh_ids_simple_schema(
+ pyarrow_schema_simple_without_ids: pa.Schema, iceberg_schema_simple: Schema
+) -> None:
+ assert pre_order_visit_pyarrow(pyarrow_schema_simple_without_ids,
_ConvertToIcebergWithFreshIds()) == iceberg_schema_simple
Review Comment:
is there a need here to check for `schema_id` equality? `Schema.__eq__` does
not check for `schema_id`. `schema_id` should be `0` here
##########
tests/catalog/test_base.py:
##########
@@ -255,6 +258,10 @@ def catalog() -> InMemoryCatalog:
return InMemoryCatalog("test.in.memory.catalog", **{"test.key":
"test.value"})
+TEST_PYARROW_SCHEMA = pa.schema([
Review Comment:
nit: i think this is no longer used, tests are using
`pyarrow_schema_simple_without_ids` from conftest instead
##########
mkdocs/docs/api.md:
##########
@@ -146,6 +146,26 @@ catalog.create_table(
)
```
+One can also create an Iceberg table using a pyarrow schema:
+
+```python
+import pyarrow as pa
+
+pa.schema(
+ [
+ pa.field("foo", pa.string(), nullable=True),
+ pa.field("bar", pa.int32(), nullable=False),
+ pa.field("baz", pa.bool_(), nullable=True),
+ ]
+)
+
+catalog.create_table(
+ identifier="docs_example.bids",
+ schema=schema,
+ location="s3://pyiceberg",
+)
Review Comment:
```suggestion
schema = pa.schema(
[
pa.field("foo", pa.string(), nullable=True),
pa.field("bar", pa.int32(), nullable=False),
pa.field("baz", pa.bool_(), nullable=True),
]
)
catalog.create_table(
identifier="docs_example.bids",
schema=schema,
)
```
##########
pyiceberg/catalog/__init__.py:
##########
@@ -512,6 +516,22 @@ def _check_for_overlap(removals: Optional[Set[str]],
updates: Properties) -> Non
if overlap:
raise ValueError(f"Updates and deletes have an overlap:
{overlap}")
+ @staticmethod
+ def _convert_schema_if_needed(schema: Union[Schema, "pa.Schema"]) ->
Schema:
+ try:
Review Comment:
nit: it might be easier to first type check `Schema`, then `pa.Schema`, else
ValueError
##########
mkdocs/docs/api.md:
##########
@@ -146,6 +146,26 @@ catalog.create_table(
)
```
+One can also create an Iceberg table using a pyarrow schema:
Review Comment:
nit:
```suggestion
To create a table using a pyarrow schema:
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
