Re: [PR] feat: add file_io and local fs impl [iceberg-cpp]

2025-01-16 Thread via GitHub
zhjwpku commented on PR #30: URL: https://github.com/apache/iceberg-cpp/pull/30#issuecomment-2597349794 > > Can we provide both? > > I think we can. It is pretty straight-forward to do something similar to `arrow::RandomnAccessFile`: https://github.com/apache/arrow/blob/main/cpp/src/

Re: [PR] feat: add file_io and local fs impl [iceberg-cpp]

2025-01-16 Thread via GitHub
wgtmac commented on PR #30: URL: https://github.com/apache/iceberg-cpp/pull/30#issuecomment-2597327334 > Can we provide both? I think we can. It is pretty straight-forward to do something similar to `arrow::RandomnAccessFile`: https://github.com/apache/arrow/blob/main/cpp/src/arrow/i

Re: [PR] feat: add file_io and local fs impl [iceberg-cpp]

2025-01-16 Thread via GitHub
zhjwpku commented on code in PR #30: URL: https://github.com/apache/iceberg-cpp/pull/30#discussion_r1918650611 ## src/iceberg/io/fs_file_io.h: ## @@ -0,0 +1,55 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See

Re: [PR] feat: add file_io and local fs impl [iceberg-cpp]

2025-01-15 Thread via GitHub
zhjwpku commented on PR #30: URL: https://github.com/apache/iceberg-cpp/pull/30#issuecomment-2594455927 > Is my understanding correct that this FileIo pertains to locations that are local, on HDFS, or on S3? Yeah, I hope this FileIO to be extended to other storages. -- This is an a

Re: [PR] feat: add file_io and local fs impl [iceberg-cpp]

2025-01-15 Thread via GitHub
zhjwpku commented on PR #30: URL: https://github.com/apache/iceberg-cpp/pull/30#issuecomment-2594498304 > For instance, async vs sync: [#2 (comment)](https://github.com/apache/iceberg-cpp/issues/2#issuecomment-2522607394) Can we provide both? > > Or, whether the core librar

Re: [PR] feat: add file_io and local fs impl [iceberg-cpp]

2025-01-15 Thread via GitHub
MisterRaindrop commented on PR #30: URL: https://github.com/apache/iceberg-cpp/pull/30#issuecomment-2594372750 Is my understanding correct that this FileIo pertains to locations that are local, on HDFS, or on S3? -- This is an automated message from the Apache Git Service. To respond to t

Re: [PR] feat: add file_io and local fs impl [iceberg-cpp]

2025-01-15 Thread via GitHub
lidavidm commented on PR #30: URL: https://github.com/apache/iceberg-cpp/pull/30#issuecomment-2594348439 For instance, async vs sync: https://github.com/apache/iceberg-cpp/issues/2#issuecomment-2522607394 Or, whether the core library should do any IO at all: https://github.com/apache

Re: [PR] feat: add file_io and local fs impl [iceberg-cpp]

2025-01-15 Thread via GitHub
zhjwpku commented on PR #30: URL: https://github.com/apache/iceberg-cpp/pull/30#issuecomment-2594324922 > Has iceberg-cpp decided on an IO strategy already? > > It might be more productive to start writing the IO-less components, such as parsing the various metadata files, etc.

Re: [PR] feat: add file_io and local fs impl [iceberg-cpp]

2025-01-15 Thread via GitHub
lidavidm commented on code in PR #30: URL: https://github.com/apache/iceberg-cpp/pull/30#discussion_r1917497458 ## src/iceberg/io/file_io.h: ## @@ -0,0 +1,78 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See th

Re: [PR] feat: add file_io and local fs impl [iceberg-cpp]

2025-01-15 Thread via GitHub
pitrou commented on PR #30: URL: https://github.com/apache/iceberg-cpp/pull/30#issuecomment-2593303684 Has iceberg-cpp decided on an IO strategy already? It might be more productive to start writing the IO-less components, such as parsing the various metadata files, etc. -- This is

Re: [PR] feat: add file_io and local fs impl [iceberg-cpp]

2025-01-15 Thread via GitHub
wgtmac commented on PR #30: URL: https://github.com/apache/iceberg-cpp/pull/30#issuecomment-2593244484 cc @lidavidm @MisterRaindrop -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific co

Re: [PR] feat: add file_io and local fs impl [iceberg-cpp]

2025-01-15 Thread via GitHub
zhjwpku commented on PR #30: URL: https://github.com/apache/iceberg-cpp/pull/30#issuecomment-2592893837 @wgtmac @Fokko @pitrou @Xuanwo @raulcd Please take a look at this early implementation of FileIO, I plan to add more interfaces, before that, I hope to hear some advices, thanks. -- T