morningman opened a new pull request, #68938:
URL: https://github.com/apache/doris/pull/68938
### What problem does this PR solve?
Issue Number: None
Related PR: #65561, #67627, #68935
Problem Summary:
**In short.** A Variant holds a UUID either as a typed value or as an
encoded Parquet Variant value, and `CAST(... AS STRING)` gives different text
for the two: the typed one becomes `00112233-4455-6677-8899-aabbccddeeff`, the
encoded one becomes the JSON string `"00112233-4455-6677-8899-aabbccddeeff"`,
quotes included. This PR makes the encoded form convert like the typed one. It
is not a branch-4.x port (branch-4.x has no UUID type); it came up while
testing #68935.
**Background.** `ColumnVariantV2` keeps a column either typed (one Doris
scalar type) or encoded (Parquet Variant bytes), and the two states are meant
to give the same results; #67627 already checks that a typed and an encoded
UUID hash the same. For a STRING target, `cast_variant_to_string.cpp` sends
each encoded scalar either through the concrete cast of its Doris type or to
JSON text. #65561 wrote that split before Doris had a UUID type, so
`VariantPrimitiveId::UUID` went to the JSON side together with BINARY and
TIME_NTZ, which still have no Doris type. #67627 then added the UUID type and
the encoded-UUID group in `cast_variant_to_scalar.cpp` (so `CAST(... AS UUID)`
of an encoded value works), but left the STRING split unchanged.
**The problem, and what it cost.** An encoded UUID shows up as soon as the
value goes through an encoded Variant: an element of a Variant array, or a
Paimon VARIANT column read back.
| Expression | master | with this PR |
|---|---|---|
| `CAST(CAST(CAST(u AS UUID) AS VARIANT) AS STRING)` (typed) | `00112233-…`
| `00112233-…` |
| `CAST(CAST(ARRAY(CAST(u AS UUID)) AS VARIANT)[1] AS STRING)` (encoded) |
`"00112233-…"` | `00112233-…` |
| `CAST(CAST(ARRAY(CAST(u AS UUID)) AS VARIANT) AS ARRAY<STRING>)` |
`[""00112233-…""]`: the element itself carries the quotes | `["00112233-…"]` |
| `CAST(v AS STRING)` on a Paimon VARIANT column holding a UUID |
`"6ba7b810-…"` | `6ba7b810-…` |
| `CAST(CAST(ARRAY(CAST(u AS UUID)) AS VARIANT)[1] AS JSON)` |
`"00112233-…"` | unchanged (a JSON string) |
So the same value printed differently depending on how it was stored, and a
comparison such as `CAST(v AS STRING) = '00112233-…'` failed for the encoded
form. DATE, DECIMAL and TIMESTAMP values were not affected: they already used
their concrete cast.
**How this PR fixes it.** `uses_concrete_string_cast` now sends
`VariantPrimitiveId::UUID` to the concrete cast, which decodes the value into
the UUID group and casts it with the UUID-to-STRING cast, the same path a typed
UUID takes. BINARY and TIME_NTZ stay JSON text. `CAST(... AS ARRAY<STRING>)`
uses the same rule for its elements, so it changes too; JSON output is a
separate path and keeps the quoted string.
**Results.** The encoded and typed forms now give the same STRING, in a
whole-column cast, in a column that mixes UUIDs with JSON-only values, and as
`ARRAY<STRING>` elements.
### Release note
CAST of a Variant value holding a UUID to STRING, VARCHAR, CHAR or
ARRAY<STRING> now returns the UUID text without JSON quotes, whether the
Variant value is typed or encoded (for example an element of a Variant array or
a value read from a Paimon VARIANT column).
### Check List (For Author)
- Test: Regression test / Unit Test
- BE UT: `CastVariantV2FromTest.EncodedUuidStringMatchesTypedUuidString`
(typed UUID, an all-UUID encoded column, a mixed encoded column, and an
`ARRAY<STRING>` target). `EOnlyScalarStringRulesStayJsonQuoted` keeps BINARY
and TIME_NTZ, the scalars that still have no Doris type, and drops UUID. The
new case fails without the fix.
- Regression: `datatype_p0/uuid/test_uuid_variant_identity` adds
`uuid_variant_to_string` (the typed and encoded STRING casts, `ARRAY<STRING>`,
the JSON cast, and the whole array as STRING). On master the encoded columns
come back quoted; the `.out` was generated by the script with this change.
- Behavior changed: Yes. CAST of an encoded Variant UUID to a string type
returns the UUID without JSON quotes.
- Does this need documentation: No
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
