haizhou-zhao commented on issue #6798: URL: https://github.com/apache/iceberg/issues/6798#issuecomment-1430593707
@Fokko thanks for the explanation and your patience. I might be asking many rudimental questions as I'm new to this section of the code base. I see what you meant by the existing OpenAPI spec is just not complex enough to handle all the scenarios. I apologize if I talk too much from the perspective of java, as I haven't played around too much with Iceberg's python or other language APIs. Perhaps taking a step back, my initial question was out of the assumption that currently there're **two sets of models co-existing**: 1. the set of models that were written in pure Java inside the iceberg-core library - e.g. [Schema](https://github.com/apache/iceberg/blob/master/api/src/main/java/org/apache/iceberg/Schema.java) 2. the set of models that could be generated from OpenAPI definition - e.g. [Schema](https://github.com/apache/iceberg/blob/master/open-api/rest-catalog-open-api.yaml#L1075) I believe the first set of models are used by [Iceberg Catalog](https://github.com/apache/iceberg/blob/master/api/src/main/java/org/apache/iceberg/catalog/Catalog.java), and the second set of models are used by the "Rest Catalog Services" (who implements the open API spec). Let me know if my assumption is correct so far, cause my later statements/questions are based on them. Now, I summarize my questions on rest service models and spec into these **2 questions**: 1. Is there any existing (model) converter/adapter to convert the Rest Service's model, for example, "`LoadTableResponse`" (of generated model, set 2 ^) to "Table" (of the existing pure java code based model, set 1 ^), in a ["getTable"](https://github.com/apache/iceberg/blob/master/core/src/main/java/org/apache/iceberg/rest/RESTCatalog.java#L93) call initiated by RestCatalog? 2. Is there any future plan to converge the two sets of models? -------------------------- Some investigation and thoughts I had on the questions above: On Q1: I found this part of [RESTSessionCatalog](https://github.com/apache/iceberg/blob/master/core/src/main/java/org/apache/iceberg/rest/RESTSessionCatalog.java#L269) might be related to "converting/adapting" RestService's LoadTableResponse. To my reading, it looks like the code here is assuming that the rest service will return an object of [`LoadTableResponse.class`](https://github.com/apache/iceberg/blob/master/core/src/main/java/org/apache/iceberg/rest/responses/LoadTableResponse.java)? But shouldn't the rest service return an object named [`LoadTableResponse`](https://github.com/apache/iceberg/blob/master/open-api/rest-catalog-open-api.yaml#L2310) as defined by OpenAPI? To me, it doesn't look like the two different `LoadTableResponse`s have the same layout, and can be directly de/serialized into one another. On Q2: I understand your point that there needs to be two sets of model so that we could maintain backward compatibility for the existing Tables (metadata, snapshots, etc.)? Conceptually, if we technically have a way to merge these two sets of models (which are conceptually for the same purpose), then would it make things easier by eliminating the need of an "converter/adapter" between the 2? Again, thx for your patience reading through my insanely long post. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org For additional commands, e-mail: issues-h...@iceberg.apache.org
