vincenzon opened a new issue, #1357:
URL: https://github.com/apache/iceberg-python/issues/1357
### Apache Iceberg version
0.8.0 (latest release)
### Please describe the bug 🐞
A row_filter passed table overwrite throws a parse exception if the column
begins with a underscore. The example below demonstrates the issue. I tried
quoting the column name but that didn't help.
```import pathlib
import pyarrow as pa
import pandas as pd
from pyiceberg.schema import Schema
from pyiceberg.types import StringType, NestedField
from pyiceberg.exceptions import NamespaceAlreadyExistsError
from pyiceberg.catalog.sql import SqlCatalog
catalog_path = "/tmp/iceberg_data"
pathlib.Path(catalog_path).mkdir(exist_ok=True)
catalog = SqlCatalog(
"default",
**{
"uri": f"sqlite:///{catalog_path}/pyiceberg_catalog.db",
"warehouse": f"file://{catalog_path}",
},
)
try:
catalog.create_namespace("default")
except NamespaceAlreadyExistsError:
pass
df = pa.Table.from_pandas(pd.DataFrame(
{
"_X": ['a', 'b', 'c'],
"YY": ['A','A','A'],
}))
schema = Schema(
NestedField(field_id=1, name='_X', field_type=StringType(),
required=False),
NestedField(field_id=2, name='YY', field_type=StringType(),
required=False),
schema_id=0,
identifier_field_ids=[])
table = "default.foo"
try:
catalog.drop_table(table)
except:
pass
tbl = catalog.create_table(
identifier=table,
schema=schema,
)
tbl.append(df)
# These two examples raise ParseException:
tbl.overwrite(df, """ _X == 'a' """)
#tbl.overwrite(df, """ "_X" == 'a' """)
# These two examples are fine:
#tbl.overwrite(df, """ YY == 'A' """)
#tbl.overwrite(df, """ "YY" == 'A' """)
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
