frankliee opened a new issue, #8487:
URL: https://github.com/apache/iceberg/issues/8487
### Apache Iceberg version
None
### Query engine
None
### Please describe the bug 🐞
I have created a simple iceberg table by using spark with iceberg.
```
scala> sql("create table ice.default.ice_t1( a string, b string, c int)
using iceberg")
scala> sql("insert into ice.default.ice_t1 values('a','a',1),('b','b',2)")
scala> sql("select * from ice.default.ice_t1").show(false)
+---+---+---+
|a |b |c |
+---+---+---+
|a |a |1 |
|b |b |2 |
+---+---+---+
```
Then I use pyiceberg to query this table by the following code.
```
from pyiceberg.catalog import load_catalog
catalog = load_catalog("default")
catalog.list_namespaces()
table = catalog.load_table("default.ice_t1")
from pyiceberg.expressions import GreaterThanOrEqual
scan = table.scan(
row_filter=GreaterThanOrEqual("c", 1),
selected_fields=("a", "b", "c"),
limit=100,
)
[task.file.file_path for task in scan.plan_files()]
```
However, it seems that pyiceberg cannot resolve the avro file.
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/usr/local/lib/python3.8/dist-packages/pyiceberg/table/__init__.py",
line 827, in plan_files
for manifest_entry in chain(
File "/usr/lib/python3.8/concurrent/futures/_base.py", line 619, in
result_iterator
yield fs.pop().result()
File "/usr/lib/python3.8/concurrent/futures/_base.py", line 444, in result
return self.__get_result()
File "/usr/lib/python3.8/concurrent/futures/_base.py", line 389, in
__get_result
raise self._exception
File "/usr/lib/python3.8/concurrent/futures/thread.py", line 57, in run
result = self.fn(*self.args, **self.kwargs)
File "/usr/local/lib/python3.8/dist-packages/pyiceberg/table/__init__.py",
line 829, in <lambda>
lambda args: _open_manifest(*args),
File "/usr/local/lib/python3.8/dist-packages/pyiceberg/table/__init__.py",
line 698, in _open_manifest
for manifest_entry in manifest.fetch_manifest_entry(io,
discard_deleted=True)
File "/usr/local/lib/python3.8/dist-packages/pyiceberg/manifest.py", line
348, in fetch_manifest_entry
with AvroFile[ManifestEntry](
File "/usr/local/lib/python3.8/dist-packages/pyiceberg/avro/file.py", line
175, in __enter__
self.reader = resolve(self.schema, self.read_schema, self.read_types,
self.read_enums)
File "/usr/local/lib/python3.8/dist-packages/pyiceberg/avro/resolver.py",
line 212, in resolve
return visit_with_partner(
File "/usr/lib/python3.8/functools.py", line 875, in wrapper
return dispatch(args[0].__class__)(*args, **kw)
File "/usr/local/lib/python3.8/dist-packages/pyiceberg/schema.py", line
591, in _
return visitor.schema(schema, partner,
visit_with_partner(schema.as_struct(), struct_partner, visitor, accessor)) #
type: ignore
File "/usr/lib/python3.8/functools.py", line 875, in wrapper
return dispatch(args[0].__class__)(*args, **kw)
File "/usr/local/lib/python3.8/dist-packages/pyiceberg/schema.py", line
601, in _
field_result = visit_with_partner(field.field_type, field_partner,
visitor, accessor) # type: ignore
File "/usr/lib/python3.8/functools.py", line 875, in wrapper
return dispatch(args[0].__class__)(*args, **kw)
File "/usr/local/lib/python3.8/dist-packages/pyiceberg/schema.py", line
601, in _
field_result = visit_with_partner(field.field_type, field_partner,
visitor, accessor) # type: ignore
File "/usr/lib/python3.8/functools.py", line 875, in wrapper
return dispatch(args[0].__class__)(*args, **kw)
File "/usr/local/lib/python3.8/dist-packages/pyiceberg/schema.py", line
623, in _
key_partner = accessor.map_key_partner(partner)
File "/usr/local/lib/python3.8/dist-packages/pyiceberg/avro/resolver.py",
line 391, in map_key_partner
raise ResolveError(f"File/read schema are not aligned for map, got
{partner_map}")
pyiceberg.exceptions.ResolveError: File/read schema are not aligned for map,
got None
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org
For additional commands, e-mail: issues-h...@iceberg.apache.org
