Re: [I] ValueError: Unrecognized filesystem type in URI: abfss [iceberg-python]

2024-03-17 Thread via GitHub
djouallah closed issue #527: ValueError: Unrecognized filesystem type in URI: abfss URL: https://github.com/apache/iceberg-python/issues/527 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the speci

Re: [I] ValueError: Unrecognized filesystem type in URI: abfss [iceberg-python]

2024-03-17 Thread via GitHub
djouallah commented on issue #527: URL: https://github.com/apache/iceberg-python/issues/527#issuecomment-2002696867 Thanks, it works now -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specif

Re: [I] ValueError: Unrecognized filesystem type in URI: abfss [iceberg-python]

2024-03-17 Thread via GitHub
kevinjqliu commented on issue #527: URL: https://github.com/apache/iceberg-python/issues/527#issuecomment-2002552232 oh interesting, this is because of a dependency issue. The actual error shows up when you try to import that class ``` from pyiceberg.io.fsspec import FsspecFileI

Re: [I] ValueError: Unrecognized filesystem type in URI: abfss [iceberg-python]

2024-03-16 Thread via GitHub
djouallah commented on issue #527: URL: https://github.com/apache/iceberg-python/issues/527#issuecomment-2002280228 getting this error now ``` 319 return file_io 320 else: --> 321 raise ValueError(f"Could not initialize FileIO: {io_impl}")

Re: [I] ValueError: Unrecognized filesystem type in URI: abfss [iceberg-python]

2024-03-16 Thread via GitHub
kevinjqliu commented on issue #527: URL: https://github.com/apache/iceberg-python/issues/527#issuecomment-2002034161 In the meantime, I think you might be able to workaround this by explicitly using fsspec. You'd have to set this in the catalog properties setting ``` catalog = SqlC

Re: [I] ValueError: Unrecognized filesystem type in URI: abfss [iceberg-python]

2024-03-16 Thread via GitHub
kevinjqliu commented on issue #527: URL: https://github.com/apache/iceberg-python/issues/527#issuecomment-2002032924 Looks like pyarrow can support "fsspec-compatible filesystems" like Azure Blob Storage (`abfs`/`abfss`) https://arrow.apache.org/docs/python/filesystems.html#using-fsspec-

Re: [I] ValueError: Unrecognized filesystem type in URI: abfss [iceberg-python]

2024-03-16 Thread via GitHub
kevinjqliu commented on issue #527: URL: https://github.com/apache/iceberg-python/issues/527#issuecomment-2002031396 Pulling the example out of the notebook: ``` table = catalog.create_table( "default.taxi_dataset", schema=df.schema,location="abfss://onelakene.dfs.cor