This is an automated email from the ASF dual-hosted git repository.
sungwy pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/iceberg-verification.git
The following commit(s) were added to refs/heads/main by this push:
new 8765e4e Adding Fixtures and Assertions for Schemas (#3)
8765e4e is described below
commit 8765e4e091b7daa833ebab6ef623f51545bebc72
Author: Sung Yun <[email protected]>
AuthorDate: Wed Sep 30 18:53:25 2026 -0400
Adding Fixtures and Assertions for Schemas (#3)
* add schema test fixtures
* parameterized types
* split geo
* adopt feedback - thanks Andrei and moomindani
---
table-spec/schema/README.md | 142 ++++++
table-spec/schema/core/cases.json | 836 +++++++++++++++++++++++++++++++++++
table-spec/schema/unknown/cases.json | 56 +++
3 files changed, 1034 insertions(+)
diff --git a/table-spec/schema/README.md b/table-spec/schema/README.md
new file mode 100644
index 0000000..f18908e
--- /dev/null
+++ b/table-spec/schema/README.md
@@ -0,0 +1,142 @@
+<!--
+ ~ Licensed to the Apache Software Foundation (ASF) under one
+ ~ or more contributor license agreements. See the NOTICE file
+ ~ distributed with this work for additional information
+ ~ regarding copyright ownership. The ASF licenses this file
+ ~ to you under the Apache License, Version 2.0 (the
+ ~ "License"); you may not use this file except in compliance
+ ~ with the License. You may obtain a copy of the License at
+ ~
+ ~ http://www.apache.org/licenses/LICENSE-2.0
+ ~
+ ~ Unless required by applicable law or agreed to in writing,
+ ~ software distributed under the License is distributed on an
+ ~ "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ ~ KIND, either express or implied. See the License for the
+ ~ specific language governing permissions and limitations
+ ~ under the License.
+ -->
+
+# Schema decoding
+
+Reading a schema JSON should produce the same schema in every implementation.
+This surface covers the schema document: the field tree and its id spaces,
+column order, required flags, `doc`, `schema-id` and `identifier-field-ids`.
+Type-string grammar, including the parsing and rejection of parameterized types
+such as `decimal(P, S)` and `fixed[L]`, is covered by the types surface under
+`table-spec/types/`.
+
+## Assertion
+
+```
+project(parse(input)) == decoded
+```
+
+Bytes are not compared. The spec fixes no key order in a schema JSON, and does
+not say whether an absent optional field is written out, so two byte-different
+schema JSONs can be the same schema. `decoded` is the comparable form.
+
+## Scope
+
+This surface reads the schema JSON object and nothing around it. Writing a
schema
+back out is out of scope for this surface. Whether a type is legal at a given
+format version cannot be decided here, because a schema JSON carries no
version.
+Type to format-version conformance is out of scope and is best verified at a
+higher level.
+
+## Inputs
+
+`core/` holds the shape cases: nesting, every id space, column order, `doc`,
+`schema-id`, `identifier-field-ids`, and every v1 and v2 primitive type. Every
+input in `core/` is valid writer output for format versions 1, 2 and 3.
+
+`unknown/` holds the cases for the `unknown` type, which must be optional.
+
+Consumers subscribe by subdirectory. An implementation with no `unknown` type
+runs `core/` only and names `unknown/` as excluded in its own configuration.
+
+## Case format
+
+Each file holds a `cases` array. An example:
+
+```json
+{
+ "id": "map-of-primitive",
+ "valid": true,
+ "input": {
+ "type": "struct", "schema-id": 0,
+ "fields": [
+ { "id": 1, "name": "id", "required": true, "type": "long" },
+ { "id": 2, "name": "props", "required": true, "type": {
+ "type": "map", "key-id": 3, "key": "string",
+ "value-id": 4, "value-required": false, "value": "int" } }
+ ]
+ },
+ "decoded": {
+ "schema-id": 0,
+ "identifier-field-ids": [],
+ "fields": [
+ { "id": 1, "name": "id", "parent": null, "required": true, "type":
"long", "doc": null },
+ { "id": 2, "name": "props", "parent": null, "required": true, "type":
"map", "doc": null },
+ { "id": 3, "name": "key", "parent": 2, "required": true, "type":
"string", "doc": null },
+ { "id": 4, "name": "value", "parent": 2, "required": false, "type":
"int", "doc": null }
+ ]
+ }
+}
+```
+
+- `id` is a stable name for the case, used to label failures.
+- `valid` is whether parsing should succeed. A `valid: false` case has no
+ `decoded`.
+- `input` is a schema JSON.
+- `decoded` is the expected result of parsing it: `schema-id`,
+ `identifier-field-ids`, and `fields`, one row per field id in document order.
+
+Each row of `decoded.fields` is:
+
+- `id`, the field id. List elements, map keys and map values carry
`element-id`,
+ `key-id` and `value-id`.
+- `name`, the field name. Elements, keys and values are named `element`, `key`
+ and `value`.
+- `parent`, the enclosing field's id, or `null` at the top level.
+- `required`, the field's required flag. Map keys are always `true`.
+- `type`. A nested type is `struct`, `list` or `map`, and its children are
their
+ own rows. A parameterized type is an object holding the type name and its
+ decoded parameters: `decimal` carries `precision` and `scale`, and `fixed`
+ carries `length`.
+ Every other type is the type name from the Appendix C types table.
+- `doc`, the field's doc string, or `null`.
+
+## Procedure
+
+The cases in each subscribed subdirectory are worked through one at a time.
+
+The schema in `input` is read through the same path the implementation uses to
+read a schema out of table metadata.
+
+A case marked `"valid": false` holds a malformed schema, and the read is
expected
+to fail. The case passes if it does. If the read succeeds instead, the case
+fails, and the parser is more lenient than the spec.
+
+Otherwise the read is expected to succeed. The resulting schema is turned into
+the row list described above, with each value read through the implementation's
+public accessors. Serializing the schema and reading values back out of the
JSON
+checks the writer rather than the reader, so it does not satisfy the case. The
+row list is then compared against `decoded`.
+
+The comparison is exact. A value differing only in whitespace or letter case is
+still a difference, and the case fails. Row order is preserved, because that
+order is the column order the schema asserts, and sorting rows by id discards
the
+assertion. `identifier-field-ids` is the one exception, and is compared as a
set,
+which is what the spec calls it.
+
+Every difference in a case is reported, not only the first, so that one run
shows
+everything that diverged. A report names the subdirectory, the case and the
+value, reading like `core/list-of-struct fields[2].parent`.
+
+## Notes
+
+Expected values are JSON. Field ids, `schema-id` and `parent` are written as
+numbers, because the spec types field ids as `int` and caps them below
+`Integer.MAX_VALUE - 200`. `parent` and `doc` are written as `null` where they
+have no value, and `null` there is a decoded value, not a missing assertion.
diff --git a/table-spec/schema/core/cases.json
b/table-spec/schema/core/cases.json
new file mode 100644
index 0000000..ad7d4eb
--- /dev/null
+++ b/table-spec/schema/core/cases.json
@@ -0,0 +1,836 @@
+{
+ "cases": [
+ {
+ "id": "flat-required-and-optional",
+ "valid": true,
+ "input": {
+ "type": "struct",
+ "schema-id": 0,
+ "fields": [
+ {
+ "id": 1,
+ "name": "id",
+ "required": true,
+ "type": "long"
+ },
+ {
+ "id": 2,
+ "name": "data",
+ "required": false,
+ "type": "string"
+ }
+ ]
+ },
+ "decoded": {
+ "schema-id": 0,
+ "identifier-field-ids": [],
+ "fields": [
+ { "id": 1, "name": "id", "parent": null, "required": true, "type":
"long", "doc": null },
+ { "id": 2, "name": "data", "parent": null, "required": false,
"type": "string", "doc": null }
+ ]
+ }
+ },
+ {
+ "id": "fields-not-in-id-order",
+ "valid": true,
+ "input": {
+ "type": "struct",
+ "schema-id": 0,
+ "fields": [
+ {
+ "id": 3,
+ "name": "c",
+ "required": true,
+ "type": "int"
+ },
+ {
+ "id": 1,
+ "name": "a",
+ "required": true,
+ "type": "int"
+ },
+ {
+ "id": 2,
+ "name": "b",
+ "required": false,
+ "type": "int"
+ }
+ ]
+ },
+ "decoded": {
+ "schema-id": 0,
+ "identifier-field-ids": [],
+ "fields": [
+ { "id": 3, "name": "c", "parent": null, "required": true, "type":
"int", "doc": null },
+ { "id": 1, "name": "a", "parent": null, "required": true, "type":
"int", "doc": null },
+ { "id": 2, "name": "b", "parent": null, "required": false, "type":
"int", "doc": null }
+ ]
+ }
+ },
+ {
+ "id": "nested-struct",
+ "valid": true,
+ "input": {
+ "type": "struct",
+ "schema-id": 0,
+ "fields": [
+ {
+ "id": 1,
+ "name": "id",
+ "required": true,
+ "type": "long"
+ },
+ {
+ "id": 2,
+ "name": "location",
+ "required": false,
+ "type": {
+ "type": "struct",
+ "fields": [
+ {
+ "id": 3,
+ "name": "lat",
+ "required": true,
+ "type": "double"
+ },
+ {
+ "id": 4,
+ "name": "lon",
+ "required": true,
+ "type": "double"
+ }
+ ]
+ }
+ }
+ ]
+ },
+ "decoded": {
+ "schema-id": 0,
+ "identifier-field-ids": [],
+ "fields": [
+ { "id": 1, "name": "id", "parent": null, "required": true, "type":
"long", "doc": null },
+ { "id": 2, "name": "location", "parent": null, "required": false,
"type": "struct", "doc": null },
+ { "id": 3, "name": "lat", "parent": 2, "required": true, "type":
"double", "doc": null },
+ { "id": 4, "name": "lon", "parent": 2, "required": true, "type":
"double", "doc": null }
+ ]
+ }
+ },
+ {
+ "id": "list-of-primitive",
+ "valid": true,
+ "input": {
+ "type": "struct",
+ "schema-id": 0,
+ "fields": [
+ {
+ "id": 1,
+ "name": "id",
+ "required": true,
+ "type": "long"
+ },
+ {
+ "id": 2,
+ "name": "tags",
+ "required": true,
+ "type": {
+ "type": "list",
+ "element-id": 3,
+ "element-required": false,
+ "element": "string"
+ }
+ }
+ ]
+ },
+ "decoded": {
+ "schema-id": 0,
+ "identifier-field-ids": [],
+ "fields": [
+ { "id": 1, "name": "id", "parent": null, "required": true, "type":
"long", "doc": null },
+ { "id": 2, "name": "tags", "parent": null, "required": true, "type":
"list", "doc": null },
+ { "id": 3, "name": "element", "parent": 2, "required": false,
"type": "string", "doc": null }
+ ]
+ }
+ },
+ {
+ "id": "map-of-primitive",
+ "valid": true,
+ "input": {
+ "type": "struct",
+ "schema-id": 0,
+ "fields": [
+ {
+ "id": 1,
+ "name": "id",
+ "required": true,
+ "type": "long"
+ },
+ {
+ "id": 2,
+ "name": "props",
+ "required": true,
+ "type": {
+ "type": "map",
+ "key-id": 3,
+ "key": "string",
+ "value-id": 4,
+ "value-required": false,
+ "value": "int"
+ }
+ }
+ ]
+ },
+ "decoded": {
+ "schema-id": 0,
+ "identifier-field-ids": [],
+ "fields": [
+ { "id": 1, "name": "id", "parent": null, "required": true, "type":
"long", "doc": null },
+ { "id": 2, "name": "props", "parent": null, "required": true,
"type": "map", "doc": null },
+ { "id": 3, "name": "key", "parent": 2, "required": true, "type":
"string", "doc": null },
+ { "id": 4, "name": "value", "parent": 2, "required": false, "type":
"int", "doc": null }
+ ]
+ }
+ },
+ {
+ "id": "list-of-struct",
+ "valid": true,
+ "input": {
+ "type": "struct",
+ "schema-id": 0,
+ "fields": [
+ {
+ "id": 1,
+ "name": "id",
+ "required": true,
+ "type": "long"
+ },
+ {
+ "id": 2,
+ "name": "points",
+ "required": true,
+ "type": {
+ "type": "list",
+ "element-id": 3,
+ "element-required": true,
+ "element": {
+ "type": "struct",
+ "fields": [
+ {
+ "id": 4,
+ "name": "x",
+ "required": true,
+ "type": "int"
+ },
+ {
+ "id": 5,
+ "name": "y",
+ "required": false,
+ "type": "int"
+ }
+ ]
+ }
+ }
+ }
+ ]
+ },
+ "decoded": {
+ "schema-id": 0,
+ "identifier-field-ids": [],
+ "fields": [
+ { "id": 1, "name": "id", "parent": null, "required": true, "type":
"long", "doc": null },
+ { "id": 2, "name": "points", "parent": null, "required": true,
"type": "list", "doc": null },
+ { "id": 3, "name": "element", "parent": 2, "required": true, "type":
"struct", "doc": null },
+ { "id": 4, "name": "x", "parent": 3, "required": true, "type":
"int", "doc": null },
+ { "id": 5, "name": "y", "parent": 3, "required": false, "type":
"int", "doc": null }
+ ]
+ }
+ },
+ {
+ "id": "map-with-struct-value",
+ "valid": true,
+ "input": {
+ "type": "struct",
+ "schema-id": 0,
+ "fields": [
+ {
+ "id": 1,
+ "name": "id",
+ "required": true,
+ "type": "long"
+ },
+ {
+ "id": 2,
+ "name": "by_name",
+ "required": false,
+ "type": {
+ "type": "map",
+ "key-id": 3,
+ "key": "string",
+ "value-id": 4,
+ "value-required": false,
+ "value": {
+ "type": "struct",
+ "fields": [
+ {
+ "id": 5,
+ "name": "count",
+ "required": true,
+ "type": "long"
+ }
+ ]
+ }
+ }
+ }
+ ]
+ },
+ "decoded": {
+ "schema-id": 0,
+ "identifier-field-ids": [],
+ "fields": [
+ { "id": 1, "name": "id", "parent": null, "required": true, "type":
"long", "doc": null },
+ { "id": 2, "name": "by_name", "parent": null, "required": false,
"type": "map", "doc": null },
+ { "id": 3, "name": "key", "parent": 2, "required": true, "type":
"string", "doc": null },
+ { "id": 4, "name": "value", "parent": 2, "required": false, "type":
"struct", "doc": null },
+ { "id": 5, "name": "count", "parent": 4, "required": true, "type":
"long", "doc": null }
+ ]
+ }
+ },
+ {
+ "id": "schema-id-and-identifier-fields",
+ "valid": true,
+ "input": {
+ "type": "struct",
+ "schema-id": 7,
+ "identifier-field-ids": [
+ 1,
+ 2
+ ],
+ "fields": [
+ {
+ "id": 1,
+ "name": "tenant",
+ "required": true,
+ "type": "string"
+ },
+ {
+ "id": 2,
+ "name": "key",
+ "required": true,
+ "type": "long"
+ },
+ {
+ "id": 3,
+ "name": "payload",
+ "required": false,
+ "type": "string"
+ }
+ ]
+ },
+ "decoded": {
+ "schema-id": 7,
+ "identifier-field-ids": [1, 2],
+ "fields": [
+ { "id": 1, "name": "tenant", "parent": null, "required": true,
"type": "string", "doc": null },
+ { "id": 2, "name": "key", "parent": null, "required": true, "type":
"long", "doc": null },
+ { "id": 3, "name": "payload", "parent": null, "required": false,
"type": "string", "doc": null }
+ ]
+ }
+ },
+ {
+ "id": "field-doc",
+ "valid": true,
+ "input": {
+ "type": "struct",
+ "schema-id": 0,
+ "fields": [
+ {
+ "id": 1,
+ "name": "id",
+ "required": true,
+ "type": "long",
+ "doc": "primary key"
+ },
+ {
+ "id": 2,
+ "name": "data",
+ "required": false,
+ "type": "string"
+ }
+ ]
+ },
+ "decoded": {
+ "schema-id": 0,
+ "identifier-field-ids": [],
+ "fields": [
+ { "id": 1, "name": "id", "parent": null, "required": true, "type":
"long", "doc": "primary key" },
+ { "id": 2, "name": "data", "parent": null, "required": false,
"type": "string", "doc": null }
+ ]
+ }
+ },
+ {
+ "id": "all-primitive-types",
+ "valid": true,
+ "input": {
+ "type": "struct",
+ "schema-id": 0,
+ "fields": [
+ {
+ "id": 1,
+ "name": "f_boolean",
+ "required": false,
+ "type": "boolean"
+ },
+ {
+ "id": 2,
+ "name": "f_int",
+ "required": false,
+ "type": "int"
+ },
+ {
+ "id": 3,
+ "name": "f_long",
+ "required": false,
+ "type": "long"
+ },
+ {
+ "id": 4,
+ "name": "f_float",
+ "required": false,
+ "type": "float"
+ },
+ {
+ "id": 5,
+ "name": "f_double",
+ "required": false,
+ "type": "double"
+ },
+ {
+ "id": 6,
+ "name": "f_date",
+ "required": false,
+ "type": "date"
+ },
+ {
+ "id": 7,
+ "name": "f_time",
+ "required": false,
+ "type": "time"
+ },
+ {
+ "id": 8,
+ "name": "f_timestamp",
+ "required": false,
+ "type": "timestamp"
+ },
+ {
+ "id": 9,
+ "name": "f_timestamptz",
+ "required": false,
+ "type": "timestamptz"
+ },
+ {
+ "id": 10,
+ "name": "f_string",
+ "required": false,
+ "type": "string"
+ },
+ {
+ "id": 11,
+ "name": "f_uuid",
+ "required": false,
+ "type": "uuid"
+ },
+ {
+ "id": 12,
+ "name": "f_fixed",
+ "required": false,
+ "type": "fixed[16]"
+ },
+ {
+ "id": 13,
+ "name": "f_binary",
+ "required": false,
+ "type": "binary"
+ },
+ {
+ "id": 14,
+ "name": "f_decimal",
+ "required": false,
+ "type": "decimal(9, 2)"
+ }
+ ]
+ },
+ "decoded": {
+ "schema-id": 0,
+ "identifier-field-ids": [],
+ "fields": [
+ { "id": 1, "name": "f_boolean", "parent": null, "required": false,
"type": "boolean", "doc": null },
+ { "id": 2, "name": "f_int", "parent": null, "required": false,
"type": "int", "doc": null },
+ { "id": 3, "name": "f_long", "parent": null, "required": false,
"type": "long", "doc": null },
+ { "id": 4, "name": "f_float", "parent": null, "required": false,
"type": "float", "doc": null },
+ { "id": 5, "name": "f_double", "parent": null, "required": false,
"type": "double", "doc": null },
+ { "id": 6, "name": "f_date", "parent": null, "required": false,
"type": "date", "doc": null },
+ { "id": 7, "name": "f_time", "parent": null, "required": false,
"type": "time", "doc": null },
+ { "id": 8, "name": "f_timestamp", "parent": null, "required": false,
"type": "timestamp", "doc": null },
+ { "id": 9, "name": "f_timestamptz", "parent": null, "required":
false, "type": "timestamptz", "doc": null },
+ { "id": 10, "name": "f_string", "parent": null, "required": false,
"type": "string", "doc": null },
+ { "id": 11, "name": "f_uuid", "parent": null, "required": false,
"type": "uuid", "doc": null },
+ { "id": 12, "name": "f_fixed", "parent": null, "required": false,
"type": { "type": "fixed", "length": 16 }, "doc": null },
+ { "id": 13, "name": "f_binary", "parent": null, "required": false,
"type": "binary", "doc": null },
+ { "id": 14, "name": "f_decimal", "parent": null, "required": false,
"type": { "type": "decimal", "precision": 9, "scale": 2 }, "doc": null }
+ ]
+ }
+ },
+ {
+ "id": "decimal-max-precision",
+ "valid": true,
+ "input": {
+ "type": "struct",
+ "schema-id": 0,
+ "fields": [
+ {
+ "id": 1,
+ "name": "id",
+ "required": true,
+ "type": "long"
+ },
+ {
+ "id": 2,
+ "name": "amount",
+ "required": false,
+ "type": "decimal(38, 10)"
+ }
+ ]
+ },
+ "decoded": {
+ "schema-id": 0,
+ "identifier-field-ids": [],
+ "fields": [
+ { "id": 1, "name": "id", "parent": null, "required": true, "type":
"long", "doc": null },
+ { "id": 2, "name": "amount", "parent": null, "required": false,
"type": { "type": "decimal", "precision": 38, "scale": 10 }, "doc": null }
+ ]
+ }
+ },
+ {
+ "id": "reject-identifier-field-does-not-exist",
+ "valid": false,
+ "input": {
+ "type": "struct",
+ "schema-id": 0,
+ "identifier-field-ids": [
+ 999
+ ],
+ "fields": [
+ {
+ "id": 1,
+ "name": "id",
+ "required": true,
+ "type": "long"
+ }
+ ]
+ }
+ },
+ {
+ "id": "reject-identifier-optional-field",
+ "valid": false,
+ "input": {
+ "type": "struct",
+ "schema-id": 0,
+ "identifier-field-ids": [
+ 2
+ ],
+ "fields": [
+ {
+ "id": 1,
+ "name": "id",
+ "required": true,
+ "type": "long"
+ },
+ {
+ "id": 2,
+ "name": "note",
+ "required": false,
+ "type": "string"
+ }
+ ]
+ }
+ },
+ {
+ "id": "reject-identifier-double-field",
+ "valid": false,
+ "input": {
+ "type": "struct",
+ "schema-id": 0,
+ "identifier-field-ids": [
+ 2
+ ],
+ "fields": [
+ {
+ "id": 1,
+ "name": "id",
+ "required": true,
+ "type": "long"
+ },
+ {
+ "id": 2,
+ "name": "score",
+ "required": true,
+ "type": "double"
+ }
+ ]
+ }
+ },
+ {
+ "id": "reject-identifier-float-field",
+ "valid": false,
+ "input": {
+ "type": "struct",
+ "schema-id": 0,
+ "identifier-field-ids": [
+ 2
+ ],
+ "fields": [
+ {
+ "id": 1,
+ "name": "id",
+ "required": true,
+ "type": "long"
+ },
+ {
+ "id": 2,
+ "name": "score",
+ "required": true,
+ "type": "float"
+ }
+ ]
+ }
+ },
+ {
+ "id": "reject-identifier-non-primitive-field",
+ "valid": false,
+ "input": {
+ "type": "struct",
+ "schema-id": 0,
+ "identifier-field-ids": [
+ 2
+ ],
+ "fields": [
+ {
+ "id": 1,
+ "name": "id",
+ "required": true,
+ "type": "long"
+ },
+ {
+ "id": 2,
+ "name": "loc",
+ "required": true,
+ "type": {
+ "type": "struct",
+ "fields": [
+ {
+ "id": 3,
+ "name": "x",
+ "required": true,
+ "type": "long"
+ }
+ ]
+ }
+ }
+ ]
+ }
+ },
+ {
+ "id": "reject-identifier-nested-in-list",
+ "valid": false,
+ "input": {
+ "type": "struct",
+ "schema-id": 0,
+ "identifier-field-ids": [
+ 4
+ ],
+ "fields": [
+ {
+ "id": 1,
+ "name": "id",
+ "required": true,
+ "type": "long"
+ },
+ {
+ "id": 2,
+ "name": "rows",
+ "required": true,
+ "type": {
+ "type": "list",
+ "element-id": 3,
+ "element-required": true,
+ "element": {
+ "type": "struct",
+ "fields": [
+ {
+ "id": 4,
+ "name": "k",
+ "required": true,
+ "type": "string"
+ }
+ ]
+ }
+ }
+ }
+ ]
+ }
+ },
+ {
+ "id": "reject-identifier-nested-in-map-value",
+ "valid": false,
+ "input": {
+ "type": "struct",
+ "schema-id": 0,
+ "identifier-field-ids": [
+ 5
+ ],
+ "fields": [
+ {
+ "id": 1,
+ "name": "id",
+ "required": true,
+ "type": "long"
+ },
+ {
+ "id": 2,
+ "name": "rows",
+ "required": true,
+ "type": {
+ "type": "map",
+ "key-id": 3,
+ "key": "string",
+ "value-id": 4,
+ "value-required": true,
+ "value": {
+ "type": "struct",
+ "fields": [
+ {
+ "id": 5,
+ "name": "k",
+ "required": true,
+ "type": "string"
+ }
+ ]
+ }
+ }
+ }
+ ]
+ }
+ },
+ {
+ "id": "reject-identifier-map-key",
+ "valid": false,
+ "input": {
+ "type": "struct",
+ "schema-id": 0,
+ "identifier-field-ids": [
+ 3
+ ],
+ "fields": [
+ {
+ "id": 1,
+ "name": "id",
+ "required": true,
+ "type": "long"
+ },
+ {
+ "id": 2,
+ "name": "props",
+ "required": true,
+ "type": {
+ "type": "map",
+ "key-id": 3,
+ "key": "string",
+ "value-id": 4,
+ "value-required": true,
+ "value": "string"
+ }
+ }
+ ]
+ }
+ },
+ {
+ "id": "identifier-nested-in-required-struct",
+ "valid": true,
+ "input": {
+ "type": "struct",
+ "schema-id": 0,
+ "identifier-field-ids": [
+ 3
+ ],
+ "fields": [
+ {
+ "id": 1,
+ "name": "id",
+ "required": true,
+ "type": "long"
+ },
+ {
+ "id": 2,
+ "name": "key",
+ "required": true,
+ "type": {
+ "type": "struct",
+ "fields": [
+ {
+ "id": 3,
+ "name": "part",
+ "required": true,
+ "type": "string"
+ }
+ ]
+ }
+ }
+ ]
+ },
+ "decoded": {
+ "schema-id": 0,
+ "identifier-field-ids": [3],
+ "fields": [
+ { "id": 1, "name": "id", "parent": null, "required": true, "type":
"long", "doc": null },
+ { "id": 2, "name": "key", "parent": null, "required": true, "type":
"struct", "doc": null },
+ { "id": 3, "name": "part", "parent": 2, "required": true, "type":
"string", "doc": null }
+ ]
+ }
+ },
+ {
+ "id": "reject-identifier-nested-in-optional-struct",
+ "valid": false,
+ "input": {
+ "type": "struct",
+ "schema-id": 0,
+ "identifier-field-ids": [
+ 3
+ ],
+ "fields": [
+ {
+ "id": 1,
+ "name": "id",
+ "required": true,
+ "type": "long"
+ },
+ {
+ "id": 2,
+ "name": "loc",
+ "required": false,
+ "type": {
+ "type": "struct",
+ "fields": [
+ {
+ "id": 3,
+ "name": "x",
+ "required": true,
+ "type": "long"
+ }
+ ]
+ }
+ }
+ ]
+ }
+ }
+ ]
+}
diff --git a/table-spec/schema/unknown/cases.json
b/table-spec/schema/unknown/cases.json
new file mode 100644
index 0000000..31c4d87
--- /dev/null
+++ b/table-spec/schema/unknown/cases.json
@@ -0,0 +1,56 @@
+{
+ "cases": [
+ {
+ "id": "unknown-optional",
+ "valid": true,
+ "input": {
+ "type": "struct",
+ "schema-id": 0,
+ "fields": [
+ {
+ "id": 1,
+ "name": "id",
+ "required": true,
+ "type": "long"
+ },
+ {
+ "id": 2,
+ "name": "future",
+ "required": false,
+ "type": "unknown"
+ }
+ ]
+ },
+ "decoded": {
+ "schema-id": 0,
+ "identifier-field-ids": [],
+ "fields": [
+ { "id": 1, "name": "id", "parent": null, "required": true, "type":
"long", "doc": null },
+ { "id": 2, "name": "future", "parent": null, "required": false,
"type": "unknown", "doc": null }
+ ]
+ }
+ },
+ {
+ "id": "reject-unknown-required",
+ "valid": false,
+ "input": {
+ "type": "struct",
+ "schema-id": 0,
+ "fields": [
+ {
+ "id": 1,
+ "name": "id",
+ "required": true,
+ "type": "long"
+ },
+ {
+ "id": 2,
+ "name": "future",
+ "required": true,
+ "type": "unknown"
+ }
+ ]
+ }
+ }
+ ]
+}
