Re: [I] EPIC: Rust Based Compaction [iceberg-rust]

2024-10-17 Thread via GitHub
liurenjie1024 commented on issue #624: URL: https://github.com/apache/iceberg-rust/issues/624#issuecomment-2419144850 Hi, @camuel >Does anyone has any insights on how computation heavy is the compaction workload really? Like on a beefy machine what compaction rate will be possible?

Re: [I] EPIC: Rust Based Compaction [iceberg-rust]

2024-10-14 Thread via GitHub
camuel commented on issue #624: URL: https://github.com/apache/iceberg-rust/issues/624#issuecomment-2412544556 Does anyone has any insights on how computation heavy is the compaction workload really? Like on a beefy machine what compaction rate will be possible? Like 1GB/sec? 10GB/sec? A ba

Re: [I] EPIC: Rust Based Compaction [iceberg-rust]

2024-09-25 Thread via GitHub
liurenjie1024 commented on issue #624: URL: https://github.com/apache/iceberg-rust/issues/624#issuecomment-2373702420 Thanks @Xuanwo for raisings this. I think we have a long way to go, for example, support for deletion files, transaction api. Also, I'm not sure if we actually don't need a

Re: [I] EPIC: Rust Based Compaction [iceberg-rust]

2024-09-11 Thread via GitHub
sdd commented on issue #624: URL: https://github.com/apache/iceberg-rust/issues/624#issuecomment-2342825385 I will be working on adding support for positional and identity deletes to the table scans next. Hopefully this will help towards this goal. -- This is an automated message from the

Re: [I] EPIC: Rust Based Compaction [iceberg-rust]

2024-09-09 Thread via GitHub
kevinjqliu commented on issue #624: URL: https://github.com/apache/iceberg-rust/issues/624#issuecomment-2339045843 Thanks for starting this! Linking the relevant issue from the pyiceberg side, [iceberg-python/#1092](https://github.com/apache/iceberg-python/issues/1092) Would be great to