The GitHub Actions job "Docs 404 check (beta)" on 
flink.git/FLINK-40910-reject-equality has failed.
Run started by GitHub user raminqaf (triggered by raminqaf).

Head commit for run:
37ddb7ac1539020eeeac7e0ce457c640ec0fa249 / Ramin Gharib <[email protected]>
[FLINK-40910][table] Reject VARIANT where a query needs equality or ordering

Flink compares VARIANT values by their binary encoding, but one value has many 
encodings. Object keys can be stored in any order, a field read from an object 
keeps the key dictionary of its parent, and a number keeps its storage width. 
So GROUP BY, DISTINCT, joins, set operations, PARTITION BY and = treated equal 
values as different and silently returned wrong results in streaming. Batch 
rejected most of these late, during code generation. A batch nested loop join 
still compared VARIANT keys by their bytes and returned wrong rows.

LogicalTypeChecks.isComparableKeyType checks that a type contains no VARIANT, 
which every grouping, join, partition and sort key and every operand of a 
comparison operator needs. This includes a VARIANT nested in a ROW, ARRAY, MAP 
or structured type. A new KeyTypeValidator checks the logical plan before 
optimization, so it covers SQL and the Table API. It rejects such types in 
grouping keys, DISTINCT aggregates, set operations other than UNION ALL, ORDER 
BY, the PARTITION BY and ORDER BY of OVER windows, MATCH_RECOGNIZE and table 
arguments, comparison operators and IN subqueries. This includes DISTINCT 
aggregates in OVER windows and comparisons in watermark expressions. Joins are 
covered by the comparisons in their conditions.

A job restored from a compiled plan skips KeyTypeValidator and keeps running.

The VARIANT section of the data types page lists where a query cannot compare 
VARIANT values and shows the cast to use instead.

Report URL: https://github.com/apache/flink/actions/runs/37644760145

With regards,
GitHub Actions via GitBox

Reply via email to