dimas-b commented on code in PR #10908: URL: https://github.com/apache/iceberg/pull/10908#discussion_r1731580319
########## open-api/README.md: ########## @@ -40,3 +40,66 @@ make generate ``` The generated code is not being used in the project, but helps to see what the changes in the open-API definition are in the generated code. + +# REST Compatibility Kit (RCK) + +The REST Compatibility Kit (RCK) is a Technology Compatibility Kit (TCK) implementation for the +Iceberg REST Specification. This includes a series of tests based on the Java reference +implementation of the REST Catalog that can be executed against any REST server that implements the +spec. Review Comment: maybe nit-picky, but I do believe RCK (TCK) ought to be based on the _spec_, not on the reference implementation. The RCK should _pass_ on the reference impl.. but the foundation for it is the spec. Technically, the reference impl. may be incorrect with the respect to the spec (as any piece of software), so the test ought to validate that as well as catalog implementations from other projects. ########## open-api/README.md: ########## @@ -40,3 +40,66 @@ make generate ``` The generated code is not being used in the project, but helps to see what the changes in the open-API definition are in the generated code. + +# REST Compatibility Kit (RCK) + +The REST Compatibility Kit (RCK) is a Technology Compatibility Kit (TCK) implementation for the +Iceberg REST Specification. This includes a series of tests based on the Java reference +implementation of the REST Catalog that can be executed against any REST server that implements the +spec. + +## Test Configuration + +The RCK can be configured using either environment variables or java system properties and allows +for configuring both the tests and the REST client. Environment variables prefixed by `CATALOG_` +are passed through the catalog configuring with the following mutations: + +1. The `CATALOG_` prefix is stripped from the key name +2. Single underscore (`_`) is replaced with a dot (`.`) +3. Double underscore (`__`) is replaced with a dash (`-`) +4. The key names are converted to lowercase + +A basic environment configuration would look like the following: + +```shell +CATALOG_URI=https://my_rest_server.io/ ## -> uri=https://my_rest_server.io/ +CATALOG_WAREHOUSE=test_warehouse ## -> warehouse=test_warehouse +CATALOG_IO__IMPL=org.apache.iceberg.aws.s3.S3FileIO ## -> io-impl=org.apache.iceberg.aws.s3.S3FileIO +CATALOG_CREDENTIAL=<oauth_key>:<oauth_secret> ## -> credential=<oauth_key>:<oauth_secret> +``` + +Java properties passed to the test must be prefixed with `rck.`, which can be used to configure some +test configurations described below and any catalog client properties. + +An example of the same configuration using java system properties would look like the following: +```shell +rck.uri=https://my_rest_server.io/ ## -> uri=https://my_rest_server.io/ +rck.warehouse=test_warehouse ## -> warehouse=test_warehouse +rck.io-impl=org.apache.iceberg.aws.s3.S3FileIO ## -> io-impl=org.apache.iceberg.aws.s3.S3FileIO +rck.credential=<oauth_key>:<oauth_secret> ## -> credential=<oauth_key>:<oauth_secret> +``` + +Some test behaviors are configurable depending on the catalog implementations. Not all behaviors +are strictly defined by the REST Specification. The following are currently configurable: + +| config | default | +|-------------------------------|---------| +| rck.requires-namespace-create | true | +| rck.supports-serverside-retry | true | + + +## Running Compatibility Tests + +The compatibility tests can be invoked via gradle with the following: + +Note: The default behavior is to run a local http server with a jdbc backend for testing purposes, +so `-Drck.local=false` must be set to point to an external REST server. + +```shell +./gradlew :iceberg-open-api:test --tests RESTCompatibilityKitSuite \ Review Comment: Could you add an example or instructions for running the RCK in a downstream Catalog project? ########## build.gradle: ########## @@ -952,6 +952,36 @@ project(':iceberg-snowflake') { } project(':iceberg-open-api') { + apply plugin: 'java-test-fixtures' Review Comment: Thx. I guess the best path forward for now is to merge, release and try it out in downstream projects. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org For additional commands, e-mail: issues-h...@iceberg.apache.org
