github  

Messages by Thread

• • Re: [PR] fix: make columnar-to-row benchmarks exercise Comet [datafusion-comet] via GitHub
  • Re: [PR] fix: make columnar-to-row benchmarks exercise Comet [datafusion-comet] via GitHub
• [I] revertToSpark erases CometIcebergWriteExec / CometNativeWriteExec because originalPlan is the node's own child [datafusion-comet] via GitHub
• [I] Re-enable adaptive skip-partial aggregation for supported high-cardinality aggregates [datafusion-comet] via GitHub
  • Re: [I] Re-enable adaptive skip-partial aggregation for supported high-cardinality aggregates [datafusion-comet] via GitHub
• [I] String-to-timestamp cast rejects explicit positive years accepted by Spark [datafusion-comet] via GitHub
  • Re: [I] String-to-timestamp cast rejects explicit positive years accepted by Spark [datafusion-comet] via GitHub
• [PR] feat: normalize marked Variant arrays at the native Parquet boundary [datafusion-comet] via GitHub
  • Re: [PR] feat: normalize marked Variant arrays at the native Parquet boundary [datafusion-comet] via GitHub
  • Re: [PR] feat: normalize marked Variant arrays at the native Parquet boundary [datafusion-comet] via GitHub
• [PR] feat: fuse the typed Dataset map sandwich into a Comet projection [datafusion-comet] via GitHub
  • Re: [PR] feat: fuse the typed Dataset map sandwich into a Comet projection [datafusion-comet] via GitHub
• [PR] refactor: make Spark-to-Arrow batch row counts explicit [datafusion-comet] via GitHub
  • Re: [PR] refactor: make Spark-to-Arrow batch row counts explicit [datafusion-comet] via GitHub
  • Re: [PR] test: cover Spark-to-Arrow batch conversion edge cases [datafusion-comet] via GitHub
  • Re: [PR] test: cover Spark-to-Arrow batch conversion edge cases [datafusion-comet] via GitHub
  • Re: [PR] test: cover Spark-to-Arrow batch conversion edge cases [datafusion-comet] via GitHub
• [I] Follow up on post-merge review of the native sequence kernel: perf crossover, unbounded per-batch allocation, and missing ceiling coverage [datafusion-comet] via GitHub
• Re: [PR] perf: project cached batches by buffer selection, prune on collated strings [datafusion-comet] via GitHub
• [I] `make format` output fails scalastyle IfBraceChecker (scalafmt wraps long brace-less if/else without adding braces) [datafusion-comet] via GitHub
  • Re: [I] `make format` output fails scalastyle IfBraceChecker (scalafmt wraps long brace-less if/else without adding braces) [datafusion-comet] via GitHub
  • Re: [I] `make format` output fails scalastyle IfBraceChecker (scalafmt wraps long brace-less if/else without adding braces) [datafusion-comet] via GitHub
  • Re: [I] `make format` output fails scalastyle IfBraceChecker (scalafmt wraps long brace-less if/else without adding braces) [datafusion-comet] via GitHub
  • Re: [I] `make format` output fails scalastyle IfBraceChecker (scalafmt wraps long brace-less if/else without adding braces) [datafusion-comet] via GitHub
• Re: [I] Implement sequence natively for integral types instead of JVM codegen dispatch [datafusion-comet] via GitHub
• Re: [I] Native Celeborn shuffle: default maxFrameBytes of 64 MiB is unreachable, and a large row fails the whole job [datafusion-comet] via GitHub
• [I] Fuse the SerializeFromObject / MapElements / DeserializeToObject sandwich into a Comet projection instead of falling back [datafusion-comet] via GitHub
• [I] Slice arrays before normalizing to avoid allocating over the whole backing buffer [datafusion-comet] via GitHub
  • Re: [I] Slice arrays before normalizing to avoid allocating over the whole backing buffer [datafusion-comet] via GitHub
• Re: [I] Support to read S3-compliant object stores [datafusion-comet] via GitHub
• [I] Re-enable the two ignored CometAggregateSuite metric tests after the DataFusion 55 peak_mem_used change [datafusion-comet] via GitHub
  • Re: [I] Re-enable the two ignored CometAggregateSuite metric tests after the DataFusion 55 peak_mem_used change [datafusion-comet] via GitHub
• [I] [EPIC] DataFusion 55 upgrade follow-ups [datafusion-comet] via GitHub
  • Re: [I] [EPIC] DataFusion 55 upgrade follow-ups [datafusion-comet] via GitHub
• [I] Establish whether the DataFusion 55 TPC-H off-heap bump is real, and changelog it if so [datafusion-comet] via GitHub
  • Re: [I] Establish whether the DataFusion 55 TPC-H off-heap bump is real, and changelog it if so [datafusion-comet] via GitHub
• [I] Nested Parquet cast retention accepts an ambiguous case-insensitive field match [datafusion-comet] via GitHub
  • Re: [I] Nested Parquet cast retention accepts an ambiguous case-insensitive field match [datafusion-comet] via GitHub
  • Re: [I] Nested Parquet cast retention accepts an ambiguous case-insensitive field match [datafusion-comet] via GitHub
• [I] SpillWriter::path() conflates "nothing spilled" with "backend has no local path" [datafusion-comet] via GitHub
  • Re: [I] SpillWriter::path() conflates "nothing spilled" with "backend has no local path" [datafusion-comet] via GitHub
  • Re: [I] SpillWriter::path() conflates "nothing spilled" with "backend has no local path" [datafusion-comet] via GitHub
• [I] copy_array panics on offset overflow instead of returning an error [datafusion-comet] via GitHub
  • Re: [I] copy_array panics on offset overflow instead of returning an error [datafusion-comet] via GitHub
  • Re: [I] copy_array panics on offset overflow instead of returning an error [datafusion-comet] via GitHub
• [I] Signed-zero array test fixtures claim the literal case agrees with Spark when it does not [datafusion-comet] via GitHub
  • Re: [I] Signed-zero array test fixtures claim the literal case agrees with Spark when it does not [datafusion-comet] via GitHub
• [I] array_distinct and array_union diverge from Spark on -0.0 for Spark versions without SPARK-54918 [datafusion-comet] via GitHub
  • Re: [I] array_distinct and array_union diverge from Spark on -0.0 for Spark versions without SPARK-54918 [datafusion-comet] via GitHub
  • Re: [I] array_distinct and array_union diverge from Spark on -0.0 for Spark versions without SPARK-54918 [datafusion-comet] via GitHub
• [PR] Pwmj metrics accounting [datafusion] via GitHub
  • Re: [PR] Pwmj metrics accounting [datafusion] via GitHub
  • Re: [PR] Pwmj metrics accounting [datafusion] via GitHub
  • Re: [PR] Pwmj metrics accounting [datafusion] via GitHub
  • Re: [PR] Pwmj metrics accounting [datafusion] via GitHub
  • Re: [PR] Pwmj metrics accounting [datafusion] via GitHub
  • Re: [PR] Pwmj metrics accounting [datafusion] via GitHub
  • Re: [PR] Pwmj metrics accounting [datafusion] via GitHub
  • Re: [PR] Pwmj metrics accounting [datafusion] via GitHub
• Re: [I] Cleanup aggregation planning [datafusion] via GitHub
  • Re: [I] Cleanup aggregation planning [datafusion] via GitHub
• [PR] Optimize NULL-containing string IN list pruning [datafusion] via GitHub
  • Re: [PR] Optimize NULL-containing string IN list pruning [datafusion] via GitHub
  • Re: [PR] Optimize NULL-containing string IN list pruning [datafusion] via GitHub
  • Re: [PR] Optimize NULL-containing string IN list pruning [datafusion] via GitHub
• Re: [I] CSE may extract Window functions into a Project node [datafusion] via GitHub
• Re: [I] chore: Fix Scala code warnings [datafusion-comet] via GitHub
  • Re: [I] chore: Fix Scala code warnings [datafusion-comet] via GitHub
  • Re: [I] chore: Fix Scala code warnings [datafusion-comet] via GitHub
• Re: [PR] Add xmlnamespace support mssql [datafusion-sqlparser-rs] via GitHub
  • Re: [PR] Add xmlnamespace support mssql [datafusion-sqlparser-rs] via GitHub
  • Re: [PR] Add xmlnamespace support mssql [datafusion-sqlparser-rs] via GitHub
• Re: [PR] fix: flush FilterExec output on pending inut for unbounded inputs [datafusion] via GitHub
  • Re: [PR] fix: flush FilterExec output on pending inut for unbounded inputs [datafusion] via GitHub
• Re: [I] External sort can lose reserved spill workspace when memory availability decreases [datafusion] via GitHub
• [PR] bench: add PWMJ vs NestedLoopJoin criterion coverage for LeftMark/Rig… [datafusion] via GitHub
  • Re: [PR] bench: add PWMJ vs NestedLoopJoin criterion coverage for LeftMark/Rig… [datafusion] via GitHub
  • Re: [PR] bench: add PWMJ vs NestedLoopJoin criterion coverage for LeftMark/Rig… [datafusion] via GitHub
  • Re: [PR] bench: add PWMJ vs NestedLoopJoin criterion coverage for LeftMark/Rig… [datafusion] via GitHub
  • Re: [PR] bench: add PWMJ vs NestedLoopJoin criterion coverage for LeftMark/Rig… [datafusion] via GitHub
  • Re: [PR] bench: add PWMJ vs NestedLoopJoin criterion coverage for LeftMark/Rig… [datafusion] via GitHub
  • Re: [PR] bench: add PWMJ vs NestedLoopJoin criterion coverage for LeftMark/Rig… [datafusion] via GitHub
• Re: [I] Substrait consumer: DuplicateUnqualifiedField when chaining two Window relations with a carried window column [datafusion] via GitHub
• Re: [I] Consider an enclosing-range fallback for `IN` lists above `max_in_list_size` [datafusion] via GitHub
• [PR] fix: avoid unsafe set rewrites for compound IN lists [datafusion] via GitHub
  • Re: [PR] fix: avoid unsafe set rewrites for compound IN lists [datafusion] via GitHub
• [PR] fix: stabilize grouped correlation with centered moments [datafusion] via GitHub
  • Re: [PR] fix: stabilize grouped correlation with centered moments [datafusion] via GitHub
• [PR] fix: reject NULL partition keys during partitioned writes [datafusion] via GitHub
• Re: [I] Support zero copy hash repartitioning for Hash Join [datafusion] via GitHub
  • Re: [I] Support zero copy hash repartitioning for Hash Join [datafusion] via GitHub
• Re: [I] [DISCUSSION] Maybe support planning scalar function in `ExprPlanner` as well? [datafusion] via GitHub
• Re: [PR] Support reading files with a boolean SBBF written by Parquet Java [datafusion] via GitHub
• [PR] add benches array [datafusion-comet] via GitHub
  • Re: [PR] chore: add benches array functions [datafusion-comet] via GitHub
  • Re: [PR] chore: add benches array functions [datafusion-comet] via GitHub
  • Re: [PR] chore: add benches array functions [datafusion-comet] via GitHub
  • Re: [PR] chore: add benches array functions [datafusion-comet] via GitHub
• Re: [I] Comet native Parquet scan reads full nested columns despite pruned Spark ReadSchema [datafusion-comet] via GitHub
  • Re: [I] Comet native Parquet scan reads full nested columns despite pruned Spark ReadSchema [datafusion-comet] via GitHub
  • Re: [I] Comet native Parquet scan reads full nested columns despite pruned Spark ReadSchema [datafusion-comet] via GitHub
• [I] [Bug] spark factorial: BIGINT overflow should wrap in non-ANSI mode and raise CAST_OVERFLOW in ANSI mode, as in Spark [datafusion] via GitHub
• Re: [I] Fix remaining reported issues for arrays of map [datafusion-comet] via GitHub
• [PR] fix: reject a zero `objectstore_writer_buffer_size` instead of hanging [datafusion] via GitHub
• Re: [I] [Discussion] Should hash join order be based on memory size? [datafusion] via GitHub
  • Re: [I] [Discussion] Should hash join order be based on memory size? [datafusion] via GitHub
• [PR] fix: compare struct and map scalars with a dynamic comparator [datafusion] via GitHub
• [PR] [Comet] Push native join runtime filters into Parquet scans [datafusion-comet] via GitHub
  • Re: [PR] feat: push native join runtime filters into Parquet scans [datafusion-comet] via GitHub
  • Re: [PR] feat: push native join runtime filters into Parquet scans [datafusion-comet] via GitHub
  • Re: [PR] feat: push native join runtime filters into Parquet scans [datafusion-comet] via GitHub
  • Re: [PR] feat: native dynamic filter pushdown for hash join into Parquet scans [datafusion-comet] via GitHub
  • Re: [PR] feat: native dynamic filter pushdown for hash join into Parquet scans [datafusion-comet] via GitHub
  • Re: [PR] feat: native dynamic filter pushdown for hash join into Parquet scans [datafusion-comet] via GitHub
  • Re: [PR] feat: native dynamic filter pushdown for hash join into Parquet scans [datafusion-comet] via GitHub
  • Re: [PR] feat: native dynamic filter pushdown for hash join into Parquet scans [datafusion-comet] via GitHub
• [I] Native Iceberg writer raises a different exception than iceberg-java for unclustered input to a clustered writer [datafusion-comet] via GitHub
• [I] Pin Range satisfaction vs advertised split-point domain [datafusion] via GitHub
• [PR] [Comet] Update Celeborn map-key assertion for Arrow 59.2 [datafusion-comet] via GitHub
  • Re: [PR] fix: Update Celeborn map-key assertion for Arrow 59.2 [datafusion-comet] via GitHub
• [PR] fix: restore columnar transitions under the native Iceberg write [datafusion-comet] via GitHub
  • Re: [PR] fix: restore columnar transitions under the native Iceberg write [datafusion-comet] via GitHub
  • Re: [PR] fix: restore columnar transitions under the native Iceberg write [datafusion-comet] via GitHub
  • Re: [PR] fix: restore columnar transitions under the native Iceberg write [datafusion-comet] via GitHub
  • Re: [PR] fix: restore columnar transitions under the native Iceberg write [datafusion-comet] via GitHub
  • Re: [PR] fix: restore columnar transitions under the native Iceberg write [datafusion-comet] via GitHub
  • Re: [PR] fix: restore columnar transitions under the native Iceberg write [datafusion-comet] via GitHub
  • Re: [PR] fix: restore columnar transitions under the native Iceberg write [datafusion-comet] via GitHub
  • Re: [PR] fix: restore columnar transitions under the native Iceberg write [datafusion-comet] via GitHub
  • Re: [PR] fix: restore columnar transitions under the native Iceberg write [datafusion-comet] via GitHub
  • Re: [PR] fix: restore columnar transitions under the native Iceberg write [datafusion-comet] via GitHub
• Re: [PR] feat: stream aggregates over grouped input [datafusion] via GitHub
• Re: [PR] feat: add Grouped equivalence properties [datafusion] via GitHub
• Re: [PR] feat: bind task-owned RSS callbacks to native shuffle plans (4/n) [datafusion-comet] via GitHub
• Re: [PR] feat: add RSS partition writer and task-owned JNI callback (1/n) [datafusion-comet] via GitHub
• Re: [PR] feat: support Apache Celeborn remote shuffle service [datafusion-comet] via GitHub
• Re: [PR] feat: complete Celeborn map-side shuffle push lifecycle (6/n) [datafusion-comet] via GitHub
• Re: [PR] feat: add Celeborn shuffle manager and partition pusher (5/n) [datafusion-comet] via GitHub
• Re: [PR] feat: support `WindowGroupLimitExec` [datafusion-comet] via GitHub
• Re: [PR] fix: revert unsafe partial aggregates after final fallback [datafusion-comet] via GitHub
• [I] enable_url_table returns a handle on a different session, unlike every other with_* derivation [datafusion-python] via GitHub
• [I] Add an agent skill for datafusion-python extension authors [datafusion-python] via GitHub
• [I] Push native join runtime filters into scans to reduce I/O and decoding [datafusion-comet] via GitHub
  • Re: [I] Push native join runtime filters into scans to reduce I/O and decoding [datafusion-comet] via GitHub
• [I] Native Iceberg write panics generating the partition path for a timestamptz partition column [datafusion-comet] via GitHub
• [I] Native Iceberg write panics in construct_partition_summaries after partition spec evolution [datafusion-comet] via GitHub
• [I] Native Iceberg write fails with "No column with source column id" after an identity partition field is dropped [datafusion-comet] via GitHub
• [PR] feat: route unrecognized StaticInvoke and Invoke through the codegen dispatcher [datafusion-comet] via GitHub
  • Re: [PR] feat: route unrecognized StaticInvoke and Invoke through the codegen dispatcher [datafusion-comet] via GitHub
  • Re: [PR] feat: route unrecognized StaticInvoke and Invoke through the codegen dispatcher [datafusion-comet] via GitHub
  • Re: [PR] feat: route unrecognized StaticInvoke and Invoke through the codegen dispatcher [datafusion-comet] via GitHub
  • Re: [PR] feat: route unrecognized StaticInvoke and Invoke through the codegen dispatcher [datafusion-comet] via GitHub
• [I] Native Iceberg writer does not honour write.target-file-size-bytes the same way as iceberg-java [datafusion-comet] via GitHub
• [I] Native Iceberg write drops a ColumnarToRow transition when AQE is disabled [datafusion-comet] via GitHub
  • Re: [I] Native Iceberg write drops a ColumnarToRow transition when AQE is disabled [datafusion-comet] via GitHub
• [PR] Arrays zip codegen [datafusion-comet] via GitHub
  • Re: [PR] feat: route `arrays_zip` on map element types through the codegen dispatcher [datafusion-comet] via GitHub
• Re: [PR] Add atomic SessionContext.with_extensions API [datafusion-python] via GitHub
• Re: [I] [DISCUSS] Is Comet ready to move to top-level Apache Comet project? [datafusion-comet] via GitHub
• Re: [PR] Add optional native Lance scan support [datafusion-comet] via GitHub
  • Re: [PR] Add optional native Lance scan support [datafusion-comet] via GitHub
• [I] feat: add native concat_ws support for array<string> arguments [datafusion-comet] via GitHub
  • Re: [I] feat: add native concat_ws support for array<string> arguments [datafusion-comet] via GitHub
• [I] Expose Python UDF inlining as a public, reusable extension codec [datafusion-python] via GitHub
• [I] Report which extension codec handled each node of a serialized plan [datafusion-python] via GitHub
• [I] Decode Python UDFs opaquely so a scheduler needs no Python interpreter [datafusion-python] via GitHub
• [I] Gate pyo3/extension-module so the crate can be linked as a Rust dependency and tested [datafusion-python] via GitHub
• Re: [I] [FEATURE] Carry Spark VariantType identity through Comet schema serialization [datafusion-comet] via GitHub
• Re: [PR] feat: full native make_interval with 2.6x faster than spark and 1.1x faster than codegen dispatch [datafusion-comet] via GitHub
• [I] perf: coalesce small shuffle-read batches before native sort [datafusion-comet] via GitHub
  • Re: [I] perf: coalesce small shuffle-read batches before native sort [datafusion-comet] via GitHub
  • Re: [I] perf: coalesce small shuffle-read batches before native sort [datafusion-comet] via GitHub
• [I] Comet's explode is slower than Spark when the element is a struct, worst on nested rows [datafusion-comet] via GitHub
  • Re: [I] Comet's explode is slower than Spark when the element is a struct, worst on nested rows [datafusion-comet] via GitHub
  • Re: [I] Comet's explode is slower than Spark when the element is a struct, worst on nested rows [datafusion-comet] via GitHub
  • Re: [I] Comet's explode is slower than Spark when the element is a struct, worst on nested rows [datafusion-comet] via GitHub
• [I] concat_ws with array<string> arguments fails natively instead of flattening like Spark [datafusion-comet] via GitHub
  • Re: [I] concat_ws with array<string> arguments fails natively instead of flattening like Spark [datafusion-comet] via GitHub
• [PR] Add docs section for debugging for contributors [datafusion] via GitHub
  • Re: [PR] Add docs section for debugging for contributors [datafusion] via GitHub
  • Re: [PR] Add docs section for debugging for contributors [datafusion] via GitHub
  • Re: [PR] Add docs section for debugging for contributors [datafusion] via GitHub
• [PR] fix: apply Spark's Parquet conversion rules to nested struct/list/map fields [datafusion-comet] via GitHub
  • Re: [PR] fix: apply Spark's Parquet conversion rules to nested struct/list/map fields [datafusion-comet] via GitHub
• [PR] fix: match Spark's ANSI bound check for float/double to integral casts [datafusion-comet] via GitHub
  • Re: [PR] fix: match Spark's ANSI bound check for float/double to integral casts [datafusion-comet] via GitHub
  • Re: [PR] fix: match Spark's ANSI bound check for float/double to integral casts [datafusion-comet] via GitHub
• Re: [PR] fix: GROUP BY returns two rows for the NULL group with a nullable UNIQUE constraint [datafusion] via GitHub
• [PR] fix: correctly rounded decimal to double/float cast matching BigDecimal.doubleValue/floatValue [datafusion-comet] via GitHub
  • Re: [PR] fix: correctly rounded decimal to double/float cast matching BigDecimal.doubleValue/floatValue [datafusion-comet] via GitHub
  • Re: [PR] fix: correctly rounded decimal to double/float cast matching BigDecimal.doubleValue/floatValue [datafusion-comet] via GitHub
  • Re: [PR] fix: correctly rounded decimal to double/float cast matching BigDecimal.doubleValue/floatValue [datafusion-comet] via GitHub
  • Re: [PR] fix: correctly rounded decimal to double/float cast matching BigDecimal.doubleValue/floatValue [datafusion-comet] via GitHub
• [PR] fix: align string to timestamp parsing with Spark's segment rules [datafusion-comet] via GitHub
  • Re: [PR] fix: align string to timestamp parsing with Spark's segment rules [datafusion-comet] via GitHub
  • Re: [PR] fix: align string to timestamp parsing with Spark's segment rules [datafusion-comet] via GitHub
  • Re: [PR] fix: align string to timestamp parsing with Spark's segment rules [datafusion-comet] via GitHub
  • Re: [PR] fix: align string to timestamp parsing with Spark's segment rules [datafusion-comet] via GitHub
  • Re: [PR] fix: align string to timestamp parsing with Spark's segment rules [datafusion-comet] via GitHub
  • Re: [PR] fix: align string to timestamp parsing with Spark's segment rules [datafusion-comet] via GitHub

• Earlier messages
• Later messages