liurenjie1024 commented on code in PR #373:
URL: https://github.com/apache/iceberg-rust/pull/373#discussion_r1701569163
##########
crates/iceberg/src/scan.rs:
##########
@@ -111,6 +135,44 @@ impl<'a> TableScanBuilder<'a> {
self
}
+ /// Provides a TableScanConfig to use as the config for this TableScan
+ pub fn with_config(mut self, table_scan_config: TableScanConfig) -> Self {
+ self.table_scan_config = table_scan_config;
+ self
+ }
+
+ /// Sets the concurrency limit for both manifest files and manifest
+ /// entries for this scan
+ pub fn with_concurrency_limit(mut self, limit: usize) -> Self {
+ self.table_scan_config = TableScanConfig {
+ concurrency_limit_manifest_files: limit,
+ concurrency_limit_manifest_entries: limit,
+ };
+ self
+ }
+
+ /// sets the manifest file concurrency limit for this scan
+ pub fn with_manifest_file_concurrency_limit(mut self, limit: usize) ->
Self {
+ self.table_scan_config = TableScanConfig {
+ concurrency_limit_manifest_files: limit,
+ concurrency_limit_manifest_entries: self
+ .table_scan_config
+ .concurrency_limit_manifest_entries,
+ };
+ self
+ }
+
+ /// sets the manifest entry concurrency limit for this scan
+ pub fn with_manifest_entru_concurrency_limit(mut self, limit: usize) ->
Self {
Review Comment:
```suggestion
pub fn with_manifest_entry_concurrency_limit(mut self, limit: usize) ->
Self {
```
##########
crates/iceberg/src/scan.rs:
##########
@@ -111,6 +135,44 @@ impl<'a> TableScanBuilder<'a> {
self
}
+ /// Provides a TableScanConfig to use as the config for this TableScan
+ pub fn with_config(mut self, table_scan_config: TableScanConfig) -> Self {
+ self.table_scan_config = table_scan_config;
+ self
+ }
+
+ /// Sets the concurrency limit for both manifest files and manifest
+ /// entries for this scan
+ pub fn with_concurrency_limit(mut self, limit: usize) -> Self {
+ self.table_scan_config = TableScanConfig {
+ concurrency_limit_manifest_files: limit,
+ concurrency_limit_manifest_entries: limit,
+ };
+ self
+ }
+
+ /// sets the manifest file concurrency limit for this scan
Review Comment:
```suggestion
/// Sets the manifest file concurrency limit for this scan
```
##########
crates/iceberg/src/scan.rs:
##########
@@ -111,6 +135,44 @@ impl<'a> TableScanBuilder<'a> {
self
}
+ /// Provides a TableScanConfig to use as the config for this TableScan
+ pub fn with_config(mut self, table_scan_config: TableScanConfig) -> Self {
+ self.table_scan_config = table_scan_config;
+ self
+ }
+
+ /// Sets the concurrency limit for both manifest files and manifest
+ /// entries for this scan
+ pub fn with_concurrency_limit(mut self, limit: usize) -> Self {
+ self.table_scan_config = TableScanConfig {
+ concurrency_limit_manifest_files: limit,
+ concurrency_limit_manifest_entries: limit,
+ };
+ self
+ }
+
+ /// sets the manifest file concurrency limit for this scan
+ pub fn with_manifest_file_concurrency_limit(mut self, limit: usize) ->
Self {
+ self.table_scan_config = TableScanConfig {
+ concurrency_limit_manifest_files: limit,
+ concurrency_limit_manifest_entries: self
+ .table_scan_config
+ .concurrency_limit_manifest_entries,
+ };
+ self
+ }
+
+ /// sets the manifest entry concurrency limit for this scan
Review Comment:
```suggestion
/// Sets the manifest entry concurrency limit for this scan
```
##########
crates/iceberg/src/scan.rs:
##########
@@ -46,6 +46,28 @@ pub type FileScanTaskStream = BoxStream<'static,
Result<FileScanTask>>;
/// A stream of arrow [`RecordBatch`]es.
pub type ArrowRecordBatchStream = BoxStream<'static, Result<RecordBatch>>;
+/// Configuration settings for a TableScan
+#[derive(Debug)]
+pub struct TableScanConfig {
Review Comment:
How about just putting these fields in `TableScanBuilder`? I think they are
similar to other parts such as `batch_size`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org
For additional commands, e-mail: issues-h...@iceberg.apache.org
