The GitHub Actions job "Docs 404 check (beta)" on flink.git/FLINK-40936-variant-function-equality has succeeded. Run started by GitHub user raminqaf (triggered by raminqaf).
Head commit for run: 2491ce91f48743a51d33c8816d0f46cba57d5ce2 / Ramin Gharib <[email protected]> [FLINK-40936][table] Reject VARIANT elements and keys in functions and operators that compare them Flink compares VARIANT values by their binary encoding, but one value has many encodings. So functions and operators that compare array elements, map keys or collected values treated equal VARIANT values as different and silently returned wrong results. ComparableTypeValidator now checks the array elements and map keys of ARRAY_CONTAINS, ARRAY_DISTINCT, ARRAY_POSITION, ARRAY_REMOVE, ARRAY_UNION, ARRAY_EXCEPT, ARRAY_INTERSECT, MAP_CONTAINS_KEY and MAP_UNION, the map key of an item access m[k], and the argument of COLLECT, also in an OVER window. An index into an array and a field access on a VARIANT compare nothing and stay allowed. The validator names what it checks in sets: the functions, the operators that compare elements, and the comparison operators that need equality or order. The comparison operator sets hold the same kinds that the validator matched before. The check lives in the validator and not in the input type strategy of each function. That would need a new argument strategy and a flag in three collection strategies, for a check that a future VARIANT equality may remove. LogicalTypeChecks.areComparable rejects every type that is not comparable, see LogicalTypeChecks.isComparableType. This also covers a structured type with a VARIANT attribute. GREATEST, LEAST, MAP_FROM_ENTRIES and Table API comparisons reject VARIANT through it during type inference. Ten functions build their equality check again during code generation: ARRAY_CONTAINS, ARRAY_DISTINCT, ARRAY_POSITION, ARRAY_REMOVE, ARRAY_UNION, ARRAY_EXCEPT, ARRAY_INTERSECT, MAP_CONTAINS_KEY, MAP_UNION and MAP_FROM_ENTRIES. That check goes through LogicalTypeChecks.areComparable, not through the validator, which only runs when a query is planned. So a job restored from a compiled plan that calls one of them on VARIANT elements or keys now fails when it is executed, before the job is submitted. Such a plan returned wrong results before. A user SpecializedFunction that builds an = evaluator on VARIANT values now fails too, in new and in restored plans. The VARIANT section of the data types page lists these functions and operators. Report URL: https://github.com/apache/flink/actions/runs/40227581907 With regards, GitHub Actions via GitBox
