liurenjie1024 commented on issue #495: URL: https://github.com/apache/iceberg-rust/issues/495#issuecomment-2258512349
> > Me and @ZENOTME did integration tests in icelake before, and I have to say that without sql engine support, it's painful to maintain those tests. > > I agree that we need a SQL engine to make testing easier. > > However, maintaining basic unit tests based on `fs` or `memory` should be straightforward, right? We don't need separate test modules; just implement them as unit tests in the REST catalog. For example, it could be as simple as... > > ```rust > // catalog / file io setup, balbalba > let table = balabala(); > > let scan = table.scan().select_all().build().unwrap(); > let batch_stream = scan.to_arrow().await.unwrap(); > > dbg!(scan); > > let batches: Vec<_> = batch_stream.try_collect().await.unwrap(); > ``` Correctly writing data into iceberg is not supported yet, so we need external systems such as spark to ingest data. Putting pre generated parquet files maybe an approach, but that requires maintaining binaries in repo. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org For additional commands, e-mail: issues-h...@iceberg.apache.org
