Messages by Thread
-
-
Re: [I] Represent Hive/value partitioning without claiming Partitioning::Hash [datafusion]
via GitHub
-
Re: [PR] fix: validate parquet encoding when configured [datafusion]
via GitHub
-
Re: [PR] Parser: Fix array suffixes after a `>>` that closes two nested types [datafusion-sqlparser-rs]
via GitHub
-
Re: [PR] test: combine datafusion-ffi test binaries into ffi_integration [datafusion]
via GitHub
-
Re: [PR] perf: decode Parquet footers for statistics in parallel during planning [datafusion]
via GitHub
-
Re: [PR] fix: support binaryAsString in the native Parquet scan [datafusion-comet]
via GitHub
-
Re: [PR] fix: keep EliminateGroupByConstant from requalifying group expressions [datafusion]
via GitHub
-
Re: [I] Spark 3.4: native Parquet scan returns NULL for a struct matched by field id under a new name when Spark prunes its fields [datafusion-comet]
via GitHub
-
Re: [PR] fix: align Union inputs after leaf expression pushdown [datafusion]
via GitHub
-
Re: [PR] fix: size aggregate spill batches by bytes as well as rows [datafusion]
via GitHub
-
[I] perf: Celeborn push completion is polled every 10 ms and the transport hook proxies every channel of the shared factory [datafusion-comet]
via GitHub
-
[I] perf: a shuffle over a native plan takes the JVM row shuffle for round-robin and nested hash partitioning [datafusion-comet]
via GitHub
-
[I] perf: the native Celeborn writer sends one frame per partition chunk with no coalescing [datafusion-comet]
via GitHub
-
[PR] fix: preserve Parquet root field IDs after nested pruning [datafusion-comet]
via GitHub
-
[I] Native hashing of NullType and ANSI interval values fails with "Unsupported data type in hasher" [datafusion-comet]
via GitHub
-
[PR] fix: reserve hash table growth before aggregating a batch [datafusion]
via GitHub
-
[I] SanityCheckPlan error when sorting by a function of a product above a projection with a computed column [datafusion]
via GitHub
-
[I] perf: the JVM columnar shuffle insert loop boxes, locks and copies every row more than once [datafusion-comet]
via GitHub
-
[I] perf: the native Celeborn writer pays a large fixed cost per frame (admission, schema, estimates, re-encodes) [datafusion-comet]
via GitHub
-
[I] bench: add the shuffle benchmark cases that the #5905 review found missing [datafusion-comet]
via GitHub
-
[I] perf: the native Celeborn writer reserves about 7x a batch and holds it through the network round trip [datafusion-comet]
via GitHub
-
[I] perf: the JVM shuffle's native radix sort always runs six passes, and the JVM reserves scratch it never uses [datafusion-comet]
via GitHub
-
[I] perf: the native multi-partition shuffle writer copies each partition's tail batch and flushes the output file once per partition [datafusion-comet]
via GitHub
-
[I] perf: use extend when buffering partition row indices in the native shuffle writer [datafusion-comet]
via GitHub
-
[I] perf: NativeBatchDecoderIterator discards its direct buffer after every block over 128 KiB [datafusion-comet]
via GitHub
-
[I] perf: the shuffle reader builds a new decompressor and zeroes a fresh body buffer for every block [datafusion-comet]
via GitHub
-
[I] perf: a stage that reads shuffle and ends in columnar-to-row crosses the FFI boundary twice per batch [datafusion-comet]
via GitHub
-
Re: [PR] fix: avoid quadratic row-buffer accounting in sort merges [datafusion]
via GitHub
-
[PR] perf: share one DiskManager per executor across native plans [datafusion-comet]
via GitHub
-
[PR] Parser: Move CREATE statement parsing into parser/create.rs [datafusion-sqlparser-rs]
via GitHub
-
[PR] Deny `clippy::should_panic_without_expect` [datafusion-sqlparser-rs]
via GitHub
-
[PR] Commit the fuzz crate's Cargo.lock [datafusion-sqlparser-rs]
via GitHub
-
Re: [I] Double-negated child check and an always-true guard in the shuffle support logic [datafusion-comet]
via GitHub
-
Re: [I] Crate name `datafusion-iceberg` is taken on crates.io; switch back to `iceberg-datafusion` [datafusion-iceberg]
via GitHub
-
[PR] perf: start bypass shuffle partition writers on small pages that grow [datafusion-comet]
via GitHub
-
Re: [PR] PostgreSQL: Remove unreachable empty op_types branch in operator FUNCTION items [datafusion-sqlparser-rs]
via GitHub
-
[I] perf: Arrow IPC write paths still copy through Channels.newChannel's 8 KiB buffer [datafusion-comet]
via GitHub
-
[I] JVM columnar shuffle panics on a struct column with a NullType field [datafusion-comet]
via GitHub
-
Re: [I] Cancelling a task doesn't stop its native plan until the plan produces its next batch [datafusion-comet]
via GitHub
-
[I] perf: the sort-based JVM columnar shuffle is 2x slower than Spark at 2,000 partitions because every partition write pays builder, file and codec setup [datafusion-comet]
via GitHub
-
Re: [PR] SQLite: Support comma-separated WITHOUT ROWID and STRICT table options [datafusion-sqlparser-rs]
via GitHub
-
[I] perf: every native plan creates and deletes a temp directory on each Spark local disk, even when it never spills [datafusion-comet]
via GitHub
-
[PR] feat: report the ordering and partitioning of materialized CTE scans [datafusion]
via GitHub
-
Re: [PR] feat: support col reference for percentiles [datafusion]
via GitHub
-
Re: [PR] Deny `clippy::manual_assert_eq` [datafusion-sqlparser-rs]
via GitHub
-
Re: [I] Native Parquet scan returns a non-null array or struct of NULLs where Spark returns NULL because no requested nested field is in the file [datafusion-comet]
via GitHub
-
Re: [I] Share Parquet footer and page index metadata across tasks in an executor [datafusion-comet]
via GitHub
-
Re: [PR] minor: fold tpcds_planning test into core_integration binary [datafusion]
via GitHub
-
Re: [I] TUI: collapse repeated sort-order blocks in JobsData::sort_jobs and share table scrolling helpers [datafusion-ballista]
via GitHub
-
Re: [PR] fix: stop building maps natively when the key type holds a FLOAT or DOUBLE [datafusion-comet]
via GitHub
-
Re: [PR] fix: read fs.comet.libhdfs.schemes for the scan rule from the Hadoop conf [datafusion-comet]
via GitHub
-
Re: [PR] fix: treat -0.0 and 0.0 as one value in arrays_overlap on Spark 4.0.5, 4.1.4 and 4.2.1 [datafusion-comet]
via GitHub
-
Re: [PR] feat: report the sort order of sorted Iceberg scans to DataFusion [datafusion-iceberg]
via GitHub
-
Re: [I] CometScanRule reads fs.comet.libhdfs.schemes from SQLConf, but native reads it from the Hadoop conf [datafusion-comet]
via GitHub
-
[PR] fix: [branch-1.1] return an empty array from CASE WHEN for an empty batch (#6763) [datafusion-comet]
via GitHub
-
Re: [PR] perf: append sort-merge join output pairs in bulk [datafusion]
via GitHub
-
Re: [PR] bench: test Arrow RLE dictionary reader optimization [datafusion]
via GitHub
-
Re: [PR] fix: validate parquet.coerce_int96_tz when it is set. [datafusion]
via GitHub