jackye1995 commented on code in PR #9695:
URL: https://github.com/apache/iceberg/pull/9695#discussion_r1511879160
##########
open-api/rest-catalog-open-api.yaml:
##########
@@ -2068,6 +2162,145 @@ components:
items:
$ref: '#/components/schemas/PartitionStatisticsFile'
+ PlanTask:
+ description:
+ A JSON object that contains information provided by the server,
+ to be utilized by clients for distributed planning, should be supplied
+ as is for input in PlanTable operation.
+ type: object
+
+ FileScanTask:
+ type: object
+ required:
+ - schema
+ - spec
+ - start
+ - length
+ - data-file
+ properties:
+ data-file:
+ $ref: '#/components/schemas/ContentFile'
+ partition:
+ type: object
+ additionalProperties:
+ type: string
+ size-bytes:
+ type: number
+ start:
+ type: number
+ length:
+ type: number
+ estimated-rows-count:
+ type: number
+ delete-files:
+ type: array
+ items:
+ $ref: '#/components/schemas/ContentFile'
+ schema:
+ $ref: '#/components/schemas/Schema'
Review Comment:
Looks like this is the schema of the partition spec of the manifest file,
which means this is the default schema of the table that is bounded to the spec
when initializing the table?
At least this seems to be what these code snippets suggest:
```
ManifestGroup manifestGroup =
new ManifestGroup(io, dataManifests, deleteManifests)
.caseSensitive(isCaseSensitive())
.select(scanColumns())
.filterData(filter())
.specsById(table().specs())
.scanMetrics(scanMetrics())
.ignoreDeleted();
```
```
LoadingCache<Integer, TaskContext> taskContextCache =
Caffeine.newBuilder()
.build(
specId -> {
PartitionSpec spec = specsById.get(specId);
ResidualEvaluator residuals = residualCache.get(specId);
return new TaskContext(spec, deleteFiles, residuals,
dropStats, scanMetrics);
});
```
```
TaskContext(
PartitionSpec spec,
DeleteFileIndex deletes,
ResidualEvaluator residuals,
boolean dropStats,
ScanMetrics scanMetrics) {
this.schemaAsString = SchemaParser.toJson(spec.schema());
this.specAsString = PartitionSpecParser.toJson(spec);
this.deletes = deletes;
this.residuals = residuals;
this.dropStats = dropStats;
this.scanMetrics = scanMetrics;
}
```
In that case it seems like it can be omitted.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org
For additional commands, e-mail: issues-h...@iceberg.apache.org
