andygrove opened a new issue, #5796:
URL: https://github.com/apache/datafusion-comet/issues/5796
Comet's JSON **writer** is fine — `to_json` is the fastest of the three
engines I measured. The two JSON **readers** are ~9x more expensive than
Velox's, and at 1.06x / 1.08x they deliver essentially no gain over Spark's JVM
implementation despite running natively.
### Measurements
2,000,000 rows, `device struct<os: string, version: string, screen:
struct<width: int, height: int>>`, 10% null. Median of 5 iterations after 2
warmups, `noop` sink. `to_json` is used to feed the readers because it gives
both engines the identical input string and makes the writer subtractable.
| query | Spark | Comet | Gluten/Velox |
|---|---|---|---|
| `SELECT to_json(device) FROM t` | 166.7 ms | 139.4 ms — **1.20x** | 166.4
ms — 1.00x |
| `SELECT get_json_object(to_json(device), '$.os') FROM t` | 914.0 ms |
858.7 ms — 1.06x | 247.2 ms — 3.70x |
| `SELECT from_json(to_json(device), 'os string, version string').os FROM t`
| 966.5 ms | 898.9 ms — 1.08x | 257.4 ms — 3.75x |
Subtracting the `to_json` row, the isolated cost of each reader is:
| kernel | Comet | Velox | ratio |
|---|---|---|---|
| `get_json_object` | **719 ms** | 81 ms | 8.9x |
| `from_json` | **760 ms** | 91 ms | 8.3x |
Plans are fully native in both engines — no fallback.
### Notes on cause
`from_json` is already on the #4942 backlog ("allocates a full
`serde_json::Value` tree per row just to read named fields"), and the source
still matches that description: `json_funcs/from_json.rs` does a full
`serde_json::from_str::<serde_json::Value>` per row and then a `get()` per
projected field, so the whole document is parsed and heap-allocated even when
two fields are projected. This is a measurement to attach to that item — it is
worth more than "medium impact" suggests when a query touches JSON at all.
The `get_json_object` number is the more surprising one. It is listed under
#4942's "already optimized" note, and the implementation in
`string_funcs/get_json_object.rs` is genuinely better than `from_json`'s — it
parses the path once for a scalar path argument and uses a
`DeserializeSeed`/`Visitor` that skips non-matching subtrees with `IgnoredAny`
rather than materialising the whole tree. Even so it lands within 6% of Spark
and 8.9x behind Velox, so whatever the remaining cost is, the lazy-descent
optimisation did not close the gap. Both readers still consume the entire
document per row (`de.end()`), and both allocate the result `String` per row.
There is already a bench at `spark-expr/benches/get_json_object.rs` to work
against.
I measured 1.0.0, not current main, and I have not profiled — treat the
cause notes as a starting point rather than a diagnosis.
### Environment
- Comet 1.0.0 (`comet-spark-spark3.5_2.12-1.0.0.jar` from Maven Central),
Spark 3.5.3, Scala 2.12
- OpenJDK 17.0.20, Ubuntu 22.04 (kernel 6.8), AMD Ryzen 9 7950X3D, 124 GB RAM
- `spark.master=local[4]`, `spark.driver.memory=8g`,
`spark.memory.offHeap.size=8g`,
`spark.sql.shuffle.partitions=8`, AQE on, session timezone UTC
- Comet confs: `spark.comet.enabled`, `spark.comet.exec.enabled`,
`spark.comet.exec.shuffle.enabled` all true, `CometShuffleManager`
### Method
Each query is written to a `noop` sink, 2 warmup iterations then 5 measured,
median
reported. Source data is a 2,000,000-row synthetic Parquet dataset (snappy,
8 files).
The comparison numbers come from running the identical SQL, on the identical
files, in
the identical JVM configuration, with the Gluten 1.6.0 Velox bundle swapped
in for
Comet. This is ad-hoc measurement, not a rigorous benchmark harness — the
ratios are
large and stable enough to be worth reporting, but please treat the absolute
milliseconds as indicative.
Every plan below was confirmed fully native from `explain` output — no
fallback to
Spark, so these are native-vs-native numbers.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
