kevinjqliu commented on issue #1041:
URL:
https://github.com/apache/iceberg-python/issues/1041#issuecomment-2575575091
@jiakai-li we can close this issue!
> At the meantime, I'm keen to work on the write.data.path and
write.metadata.path if that's something we want to enable and no
kevinjqliu closed issue #1041: [Bug] Cannot use PyIceberg with multiple FS
URL: https://github.com/apache/iceberg-python/issues/1041
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comme
jiakai-li commented on issue #1041:
URL:
https://github.com/apache/iceberg-python/issues/1041#issuecomment-2574308740
@kevinjqliu I guess we can close this issue now? At the meantime, I'm keen
to work on the `write.data.path` and `write.metadata.path` if that's something
we want to enable
kevinjqliu commented on issue #1041:
URL:
https://github.com/apache/iceberg-python/issues/1041#issuecomment-2557842068
assigned to you @jiakai-li
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to
jiakai-li commented on issue #1041:
URL:
https://github.com/apache/iceberg-python/issues/1041#issuecomment-2557828231
Hey guys, I can pick this up together with #1279 if no one is currently
working on this.
--
This is an automated message from the Apache Git Service.
To respond to the me
Sairam90 commented on issue #1041:
URL:
https://github.com/apache/iceberg-python/issues/1041#issuecomment-2557215917
Any updates on this issue , I face a similar issue when creating a table on
S3 as well
--
This is an automated message from the Apache Git Service.
To respond to the messa
kevinjqliu commented on issue #1041:
URL:
https://github.com/apache/iceberg-python/issues/1041#issuecomment-2324865049
Its configurable via the write properties. See this comment
https://github.com/apache/iceberg-python/issues/1041#issuecomment-2323380629
--
This is an automated mess
TiansuYu commented on issue #1041:
URL:
https://github.com/apache/iceberg-python/issues/1041#issuecomment-2324857080
Reading on table spec, I just realised that there is a field `location` in
https://iceberg.apache.org/spec/#table-metadata-fields that specifies a base
location of the table
kevinjqliu commented on issue #1041:
URL:
https://github.com/apache/iceberg-python/issues/1041#issuecomment-2324831712
yep! There's definitely opportunities to consolidate the two. I opened #310
with some details.
--
This is an automated message from the Apache Git Service.
To respo
TiansuYu commented on issue #1041:
URL:
https://github.com/apache/iceberg-python/issues/1041#issuecomment-2324824803
Also reading on here:
https://arrow.apache.org/docs/python/filesystems.html#using-arrow-filesystems-with-fsspec
There might be some opportunity that we simplify the
TiansuYu commented on issue #1041:
URL:
https://github.com/apache/iceberg-python/issues/1041#issuecomment-2324794360
@kevinjqliu I think resolving fs at file level should make the API cleaner.
I would say one benefit one might choose fs on table level is to reuse that fs
instance for perfo
kevinjqliu commented on issue #1041:
URL:
https://github.com/apache/iceberg-python/issues/1041#issuecomment-2324762218
Generally, this problem should go away if we re-evaluate `fs` and `io` each
time a file is read and written. Or other words, we should stop passing the
`io` parameter arou
kevinjqliu commented on issue #1041:
URL:
https://github.com/apache/iceberg-python/issues/1041#issuecomment-2324758344
> I dont think fixing SqlCatalog alone is the proper answer to this bug. The
io layer seems to me ill written and has to be fixed somewhere in the uppper
level (e.g. Fsspe
kevinjqliu commented on issue #1041:
URL:
https://github.com/apache/iceberg-python/issues/1041#issuecomment-2324750207
Thanks for taking a look at this @TiansuYu
> why we are implementing a custom
I think custom scheme parsing avoids picking one library over another
(`fsspec`
TiansuYu commented on issue #1041:
URL:
https://github.com/apache/iceberg-python/issues/1041#issuecomment-2324566016
Read my comment
[here](https://gist.github.com/kevinjqliu/647808faba256855639e91dd58243082?permalink_comment_id=5175413#gistcomment-5175413)
for the cause of the issue.
TiansuYu commented on issue #1041:
URL:
https://github.com/apache/iceberg-python/issues/1041#issuecomment-2324181879
I will have a look this issue.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go t
kevinjqliu commented on issue #1041:
URL:
https://github.com/apache/iceberg-python/issues/1041#issuecomment-2323380629
Oh interesting, thanks!
Here's the config definition for `write.data.path` and `write.metadata.path`
https://iceberg.apache.org/docs/latest/configuration/#write-prope
Fokko commented on issue #1041:
URL:
https://github.com/apache/iceberg-python/issues/1041#issuecomment-2298248947
This is a good point, I've heard that folks store their metadata on HDFS,
and the data itself on S3.
I don't think the example with the add-files is the best, it would be
18 matches
Mail list logo
