github  

Messages by Thread

• [I] date_trunc SECOND/MILLISECOND wraps on Spark 4.2, where Spark raises long overflow [datafusion-comet] via GitHub
  • Re: [I] date_trunc SECOND/MILLISECOND wraps on Spark 4.2, where Spark raises long overflow [datafusion-comet] via GitHub
• Re: [I] Metadata tables ignore the scan projection, so projected and aggregate queries return wrong columns or fail [datafusion-iceberg] via GitHub
• [I] Design question: long-term, should native Iceberg scans run through DataFusion's ParquetSource instead of iceberg-rust's ArrowReader? [datafusion-comet] via GitHub
  • Re: [I] Design question: long-term, should native Iceberg scans run through DataFusion's ParquetSource instead of iceberg-rust's ArrowReader? [datafusion-comet] via GitHub
  • Re: [I] Design question: long-term, should native Iceberg scans run through DataFusion's ParquetSource instead of iceberg-rust's ArrowReader? [datafusion-comet] via GitHub
  • Re: [I] Design question: long-term, should native Iceberg scans run through DataFusion's ParquetSource instead of iceberg-rust's ArrowReader? [datafusion-comet] via GitHub
  • Re: [I] Design question: long-term, should native Iceberg scans run through DataFusion's ParquetSource instead of iceberg-rust's ArrowReader? [datafusion-comet] via GitHub
  • Re: [I] Design question: long-term, should native Iceberg scans run through DataFusion's ParquetSource instead of iceberg-rust's ArrowReader? [datafusion-comet] via GitHub
• [PR] perf: share one StatisticsContext across all physical optimizer rules [datafusion] via GitHub
  • Re: [PR] perf: share one StatisticsContext across all physical optimizer rules [datafusion] via GitHub
  • Re: [PR] perf: share one StatisticsContext across all physical optimizer rules [datafusion] via GitHub
  • Re: [PR] perf: share one StatisticsContext across all physical optimizer rules [datafusion] via GitHub
  • Re: [PR] perf: share one StatisticsContext across all physical optimizer rules [datafusion] via GitHub
  • Re: [PR] perf: share one StatisticsContext across all physical optimizer rules [datafusion] via GitHub
  • Re: [PR] perf: share one StatisticsContext across all physical optimizer rules [datafusion] via GitHub
  • Re: [PR] perf: share one StatisticsContext across all physical optimizer rules [datafusion] via GitHub
  • Re: [PR] perf: share one StatisticsContext across all physical optimizer rules [datafusion] via GitHub
  • Re: [PR] perf: share one StatisticsContext across all physical optimizer rules [datafusion] via GitHub
  • Re: [PR] perf: share one StatisticsContext across all physical optimizer rules [datafusion] via GitHub
  • Re: [PR] perf: share one StatisticsContext across all physical optimizer rules [datafusion] via GitHub
  • Re: [PR] perf: share one StatisticsContext across all physical optimizer rules [datafusion] via GitHub
  • Re: [PR] perf: share one StatisticsContext across all physical optimizer rules [datafusion] via GitHub
  • Re: [PR] perf: share one StatisticsContext across all physical optimizer rules [datafusion] via GitHub
  • Re: [PR] perf: share one StatisticsContext across all physical optimizer rules [datafusion] via GitHub
  • Re: [PR] perf: share one StatisticsContext across all physical optimizer rules [datafusion] via GitHub
  • Re: [PR] perf: share one StatisticsContext across all physical optimizer rules [datafusion] via GitHub
  • Re: [PR] perf: share one StatisticsContext across all physical optimizer rules [datafusion] via GitHub
  • Re: [PR] perf: share one StatisticsContext across all physical optimizer rules [datafusion] via GitHub
  • Re: [PR] perf: share one StatisticsContext across all physical optimizer rules [datafusion] via GitHub
  • Re: [PR] perf: share one StatisticsContext across all physical optimizer rules [datafusion] via GitHub
  • Re: [PR] perf: share one StatisticsContext across all physical optimizer rules [datafusion] via GitHub
  • Re: [PR] perf: share one StatisticsContext across all physical optimizer rules [datafusion] via GitHub
  • Re: [PR] perf: share one StatisticsContext across all physical optimizer rules [datafusion] via GitHub
  • Re: [PR] perf: share one StatisticsContext across all physical optimizer rules [datafusion] via GitHub
  • Re: [PR] perf: share one StatisticsContext across all physical optimizer rules [datafusion] via GitHub
  • Re: [PR] perf: share one StatisticsContext across all physical optimizer rules [datafusion] via GitHub
  • Re: [PR] perf: share one StatisticsContext across all physical optimizer rules [datafusion] via GitHub
  • Re: [PR] perf: share one StatisticsContext across all physical optimizer rules [datafusion] via GitHub
  • Re: [PR] perf: share one StatisticsContext across all physical optimizer rules [datafusion] via GitHub
• [PR] Preserve list element metadata when rebuilding scalar values [datafusion] via GitHub
  • Re: [PR] Preserve list element metadata when rebuilding scalar values [datafusion] via GitHub
• [I] Native `mode` treats all NaNs as one value on Spark 3.4, where Spark counts each NaN separately [datafusion-comet] via GitHub
• Re: [I] Null treatment of `sort` does not follow DataFusion's default [datafusion-python] via GitHub
• [PR] Fix/exact hashtable accounting group values column [datafusion] via GitHub
  • Re: [PR] fix: use HashTable::allocation_size() in GroupValuesColumn::size() [datafusion] via GitHub
• Re: [PR] feat(datafusion): make IcebergTableProvider::try_new public [datafusion-iceberg] via GitHub
  • Re: [PR] feat(datafusion): make IcebergTableProvider::try_new public [datafusion-iceberg] via GitHub
• Re: [I] Iceberg plan nodes can't be rebuilt by a PhysicalExtensionCodec [datafusion-iceberg] via GitHub
• [PR] chore(deps): bump urllib3 from 2.6.3 to 2.8.0 [datafusion-sandbox] via GitHub
• Re: [PR] chore(deps): bump urllib3 from 2.6.3 to 2.7.0 [datafusion-sandbox] via GitHub
  • Re: [PR] chore(deps): bump urllib3 from 2.6.3 to 2.7.0 [datafusion-sandbox] via GitHub
• Re: [PR] chore: add benches for remaining comet native kernels [datafusion-comet] via GitHub
• Re: [I] Dictionary performance improvement tickets [datafusion] via GitHub
  • Re: [I] Dictionary performance improvement tickets [datafusion] via GitHub
• Re: [PR] perf: retain batched build storage for large hash joins [datafusion] via GitHub
• [PR] feat: run nullable nested float sort keys natively in strict floating-point mode [datafusion-comet] via GitHub
  • Re: [PR] feat: run nullable nested float sort keys natively in strict floating-point mode [datafusion-comet] via GitHub
  • Re: [PR] feat: run nullable nested float sort keys natively in strict floating-point mode [datafusion-comet] via GitHub
  • Re: [PR] feat: run nullable nested float sort keys natively in strict floating-point mode [datafusion-comet] via GitHub
  • Re: [PR] feat: run nullable nested float sort keys natively in strict floating-point mode [datafusion-comet] via GitHub
  • Re: [PR] feat: run nullable nested float sort keys natively in strict floating-point mode [datafusion-comet] via GitHub
  • Re: [PR] feat: run nullable nested float sort keys natively in strict floating-point mode [datafusion-comet] via GitHub
  • Re: [PR] feat: run nullable nested float sort keys natively in strict floating-point mode [datafusion-comet] via GitHub
  • Re: [PR] feat: run nullable nested float sort keys natively in strict floating-point mode [datafusion-comet] via GitHub
• [PR] chore: update documentation links for 1.1.0 release [datafusion-comet] via GitHub
  • Re: [PR] chore: update documentation links for 1.1.0 release [datafusion-comet] via GitHub
• [PR] perf: backport bitmap pruning for runtime join filters [datafusion-comet] via GitHub
  • Re: [PR] perf: backport bitmap pruning for runtime join filters [datafusion-comet] via GitHub
• [PR] fix: include try_add suggestion in ANSI SUM overflow error [datafusion-comet] via GitHub
  • Re: [PR] fix: include try_add suggestion in ANSI SUM overflow error [datafusion-comet] via GitHub
  • Re: [PR] fix: include try_add suggestion in ANSI SUM overflow error [datafusion-comet] via GitHub
  • Re: [PR] fix: include try_add suggestion in ANSI SUM overflow error [datafusion-comet] via GitHub
  • Re: [PR] fix: include try_add suggestion in ANSI SUM overflow error [datafusion-comet] via GitHub
  • Re: [PR] fix: include try_add suggestion in ANSI SUM overflow error [datafusion-comet] via GitHub
  • Re: [PR] fix: include try_add suggestion in ANSI SUM overflow error [datafusion-comet] via GitHub
• [PR] test: add optimizer controls and config isolation to SQL fixtures [datafusion-comet] via GitHub
  • Re: [PR] test: add optimizer controls and config isolation to SQL fixtures [datafusion-comet] via GitHub
• Re: [PR] fix: resolve Azure ABFS authentication through Hadoop and keep ambient credentials out [datafusion-comet] via GitHub
  • Re: [PR] fix: resolve Azure ABFS authentication through Hadoop and keep ambient credentials out [datafusion-comet] via GitHub
  • Re: [PR] fix: resolve Azure ABFS authentication through Hadoop and keep ambient credentials out [datafusion-comet] via GitHub
  • Re: [PR] fix: resolve Azure ABFS authentication through Hadoop and keep ambient credentials out [datafusion-comet] via GitHub
  • Re: [PR] fix: resolve Azure ABFS authentication through Hadoop and keep ambient credentials out [datafusion-comet] via GitHub
  • Re: [PR] fix: resolve Azure ABFS authentication through Hadoop and keep ambient credentials out [datafusion-comet] via GitHub
  • Re: [PR] fix: resolve Azure ABFS authentication through Hadoop and keep ambient credentials out [datafusion-comet] via GitHub
  • Re: [PR] fix: resolve Azure ABFS authentication through Hadoop and keep ambient credentials out [datafusion-comet] via GitHub
  • Re: [PR] fix: resolve Azure ABFS authentication through Hadoop and keep ambient credentials out [datafusion-comet] via GitHub
• [PR] feature(physical-plan): add NullAwareMode to SortMergeJoinExec [datafusion] via GitHub
  • Re: [PR] feature(physical-plan): add NullAwareMode to SortMergeJoinExec [datafusion] via GitHub
  • Re: [PR] feature(physical-plan): add NullAwareMode to SortMergeJoinExec [datafusion] via GitHub
• [PR] perf: convert Spark's unsafe rows to Arrow straight from their memory [datafusion-comet] via GitHub
  • Re: [PR] perf: convert Spark's unsafe rows to Arrow straight from their memory [datafusion-comet] via GitHub
  • Re: [PR] perf: convert Spark's unsafe rows to Arrow straight from their memory [datafusion-comet] via GitHub
  • Re: [PR] perf: convert Spark's unsafe rows to Arrow straight from their memory [datafusion-comet] via GitHub
  • Re: [PR] perf: convert Spark's unsafe rows to Arrow straight from their memory [datafusion-comet] via GitHub
  • Re: [PR] perf: convert Spark's unsafe rows to Arrow straight from their memory [datafusion-comet] via GitHub
  • Re: [PR] perf: convert Spark's unsafe rows to Arrow straight from their memory [datafusion-comet] via GitHub
  • Re: [PR] perf: convert Spark's unsafe rows to Arrow straight from their memory [datafusion-comet] via GitHub
  • Re: [PR] perf: convert Spark's unsafe rows to Arrow straight from their memory [datafusion-comet] via GitHub
  • Re: [PR] perf: convert Spark's unsafe rows to Arrow straight from their memory [datafusion-comet] via GitHub
  • Re: [PR] perf: convert Spark's unsafe rows to Arrow straight from their memory [datafusion-comet] via GitHub
  • Re: [PR] perf: convert Spark's unsafe rows to Arrow straight from their memory [datafusion-comet] via GitHub
  • Re: [PR] perf: convert Spark's unsafe rows to Arrow straight from their memory [datafusion-comet] via GitHub
  • Re: [PR] perf: convert Spark's unsafe rows to Arrow straight from their memory [datafusion-comet] via GitHub
  • Re: [PR] perf: convert Spark's unsafe rows to Arrow straight from their memory [datafusion-comet] via GitHub
• Re: [I] Consolidate date/timestamp truncation and xxhash64 with upstream DataFusion implementations [datafusion-comet] via GitHub
• [PR] docs(bench): add a Trino 483 column to the TPC-H SF1000 results [datafusion-ballista] via GitHub
  • Re: [PR] docs(bench): add a Trino 483 column to the TPC-H SF1000 results [datafusion-ballista] via GitHub
  • Re: [PR] docs(bench): add a Trino 483 column to the TPC-H SF1000 results [datafusion-ballista] via GitHub
• [PR] perf: reuse Arrow buffers across batches in Spark-to-Arrow conversion [datafusion-comet] via GitHub
  • Re: [PR] perf: reuse Arrow buffers across batches in Spark-to-Arrow conversion [datafusion-comet] via GitHub
  • Re: [PR] perf: reuse Arrow buffers across batches in Spark-to-Arrow conversion [datafusion-comet] via GitHub
  • Re: [PR] perf: reuse Arrow buffers across batches in Spark-to-Arrow conversion [datafusion-comet] via GitHub
  • Re: [PR] perf: reuse Arrow buffers across batches in Spark-to-Arrow conversion [datafusion-comet] via GitHub
• [I] feat: Treat all NaNs as the same value (`-NaN = NaN`) [datafusion] via GitHub
  • Re: [I] feat: Treat all NaNs as the same value (`-NaN = NaN`) [datafusion] via GitHub
  • Re: [I] feat: Treat all NaNs as the same value (`-NaN = NaN`) [datafusion] via GitHub
• [PR] docs: add 1.1.0 changelog [datafusion-comet] via GitHub
  • Re: [PR] docs: add 1.1.0 changelog [datafusion-comet] via GitHub
• [I] Explore Kubernetes in-place pod memory resize (Spark 4.2 ExecutorResizePlugin) for Comet's untracked native memory [datafusion-comet] via GitHub
• [I] Native Iceberg scan reads every field of a nested column even when Spark prunes it [datafusion-comet] via GitHub
• Re: [PR] test: move mixed-type shift coverage to SQL fixtures [datafusion-comet] via GitHub
  • Re: [PR] test: move mixed-type shift coverage to SQL fixtures [datafusion-comet] via GitHub
• [PR] feat: admit every array and map type in Spark-to-Comet conversion [datafusion-comet] via GitHub
  • Re: [PR] feat: admit every array and map type in Spark-to-Comet conversion [datafusion-comet] via GitHub
  • Re: [PR] feat: admit every array and map type in Spark-to-Comet conversion [datafusion-comet] via GitHub
  • Re: [PR] feat: admit every array and map type in Spark-to-Comet conversion [datafusion-comet] via GitHub
  • Re: [PR] feat: admit every array and map type in Spark-to-Comet conversion [datafusion-comet] via GitHub
  • Re: [PR] feat: admit every array and map type in Spark-to-Comet conversion [datafusion-comet] via GitHub
  • Re: [PR] feat: admit every array and map type in Spark-to-Comet conversion [datafusion-comet] via GitHub
• [I] Build multi-column null-aware NOT IN on the subquery side (multi-key null-aware RightAnti) [datafusion] via GitHub
  • Re: [I] Build multi-column null-aware NOT IN on the subquery side (multi-key null-aware RightAnti) [datafusion] via GitHub
• Re: [PR] fix: validate temporal display formats [datafusion] via GitHub
• [PR] perf: read only the nested fields Spark keeps in the native Iceberg scan [datafusion-comet] via GitHub
  • Re: [PR] perf: read only the nested fields Spark keeps in the native Iceberg scan [datafusion-comet] via GitHub
  • Re: [PR] perf: read only the nested fields Spark keeps in the native Iceberg scan [datafusion-comet] via GitHub
  • Re: [PR] perf: read only the nested fields Spark keeps in the native Iceberg scan [datafusion-comet] via GitHub
  • Re: [PR] perf: read only the nested fields Spark keeps in the native Iceberg scan [datafusion-comet] via GitHub
  • Re: [PR] perf: read only the nested fields Spark keeps in the native Iceberg scan [datafusion-comet] via GitHub
  • Re: [PR] perf: read only the nested fields Spark keeps in the native Iceberg scan [datafusion-comet] via GitHub
  • Re: [PR] perf: read only the nested fields Spark keeps in the native Iceberg scan [datafusion-comet] via GitHub
  • Re: [PR] perf: read only the nested fields Spark keeps in the native Iceberg scan [datafusion-comet] via GitHub
  • Re: [PR] perf: read only the nested fields Spark keeps in the native Iceberg scan [datafusion-comet] via GitHub
  • Re: [PR] perf: read only the nested fields Spark keeps in the native Iceberg scan [datafusion-comet] via GitHub
  • Re: [PR] perf: read only the nested fields Spark keeps in the native Iceberg scan [datafusion-comet] via GitHub
• [I] Narrow multi-column NOT IN UNKNOWN checks by treating NOT NULL tuple elements as scope keys [datafusion] via GitHub
  • Re: [I] Narrow multi-column NOT IN UNKNOWN checks by treating NOT NULL tuple elements as scope keys [datafusion] via GitHub
• Re: [PR] Add Apache DataFusion Comet 1.1.0 release blog post [datafusion-site] via GitHub
  • Re: [PR] Add Apache DataFusion Comet 1.1.0 release blog post [datafusion-site] via GitHub
• [I] Admit all array and map types in CometSparkToColumnarExec [datafusion-comet] via GitHub
• [I] Row-input conversionTime metric includes the time the child spends producing rows [datafusion-comet] via GitHub
• [I] Write strings and nested values in bulk when converting Spark rows to Arrow [datafusion-comet] via GitHub
  • Re: [I] Write strings and nested values in bulk when converting Spark rows to Arrow [datafusion-comet] via GitHub
• [I] Reuse Arrow buffers across batches in Spark-to-Arrow conversion instead of reallocating and zeroing them [datafusion-comet] via GitHub
• Re: [PR] improve Spark from_utc_timestamp compatibility [datafusion] via GitHub
• Re: [PR] feat: add Nix flake and direnv support [datafusion] via GitHub
  • Re: [PR] feat: add Nix flake and direnv support [datafusion] via GitHub
  • Re: [PR] feat: add Nix flake and direnv support [datafusion] via GitHub
  • Re: [PR] feat: add Nix flake and direnv support [datafusion] via GitHub
• [I] Iceberg data tables whose names end with a metadata-table name fall back to Spark [datafusion-comet] via GitHub
  • Re: [I] Iceberg data tables whose names end with a metadata-table name fall back to Spark [datafusion-comet] via GitHub
• [I] Broadcast hash join on string keys drops rows when the build side is a large DISTINCT [datafusion-comet] via GitHub
  • Re: [I] Broadcast hash join on string keys drops rows when the build side is a large DISTINCT [datafusion-comet] via GitHub
  • Re: [I] Broadcast hash join on string keys drops rows when the build side is a large DISTINCT [datafusion-comet] via GitHub
• Re: [I] Clarify how to run native Comet -> Celeborn shuffle end-to-end [datafusion-comet] via GitHub
  • Re: [I] Clarify how to run native Comet -> Celeborn shuffle end-to-end [datafusion-comet] via GitHub
• [I] Decide scan parallelism from the estimated row count, not from statistics precision [datafusion] via GitHub
• Re: [I] Spark errors after the first batch of a JVM input stream reach the user as CometNativeException [datafusion-comet] via GitHub
• Re: [PR] feat: support multi-column NOT IN subqueries with null-aware joins [datafusion] via GitHub
  • Re: [PR] feat: support multi-column NOT IN subqueries with null-aware joins [datafusion] via GitHub
  • Re: [PR] feat: support multi-column NOT IN subqueries with null-aware joins [datafusion] via GitHub
  • Re: [PR] feat: support multi-column NOT IN subqueries with null-aware joins [datafusion] via GitHub
  • Re: [PR] feat: support multi-column NOT IN subqueries with null-aware joins [datafusion] via GitHub
  • Re: [PR] feat: support multi-column NOT IN subqueries with null-aware joins [datafusion] via GitHub
  • Re: [PR] feat: support multi-column NOT IN subqueries with null-aware joins [datafusion] via GitHub
  • Re: [PR] feat: support multi-column NOT IN subqueries with null-aware joins [datafusion] via GitHub
  • Re: [PR] feat: support multi-column NOT IN subqueries with null-aware joins [datafusion] via GitHub
  • Re: [PR] feat: support multi-column NOT IN subqueries with null-aware joins [datafusion] via GitHub
  • Re: [PR] feat: support multi-column NOT IN subqueries with null-aware joins [datafusion] via GitHub
  • Re: [PR] feat: support multi-column NOT IN subqueries with null-aware joins [datafusion] via GitHub
• Re: [I] [Epic] Fuzz data generators never produce NULLs [datafusion] via GitHub
• Re: [PR] perf(shuffle-writer): coalesce small batches before writing passthrough shuffle files [datafusion-ballista] via GitHub
  • Re: [PR] perf(shuffle-writer): coalesce small batches before writing passthrough shuffle files [datafusion-ballista] via GitHub
• Re: [I] Account the native Iceberg writer's buffers in Comet's memory pool [datafusion-comet] via GitHub
• Re: [PR] feat: charge native write buffers to the task's memory pool [datafusion-comet] via GitHub
• Re: [I] Charge native write buffers to the shared off-heap memory pool [datafusion-comet] via GitHub
• Re: [PR] fix: handle dictionary nulls in null-aware left mark joins [datafusion] via GitHub
  • Re: [PR] fix: handle dictionary nulls in null-aware left mark joins [datafusion] via GitHub
  • Re: [PR] fix: handle dictionary nulls in null-aware left mark joins [datafusion] via GitHub
  • Re: [PR] fix: handle dictionary nulls in null-aware left mark joins [datafusion] via GitHub
  • Re: [PR] fix: handle dictionary nulls in null-aware left mark joins [datafusion] via GitHub
• Re: [PR] docs: check timezone handling against the contributor guide in the PR review skills [datafusion-comet] via GitHub
• [PR] test: generate NULLs in equivalence fuzz tests [datafusion] via GitHub
  • Re: [PR] test: generate NULLs in equivalence fuzz tests [datafusion] via GitHub
• Re: [PR] feat: expose per-scan S3 HTTP attempts and retries [datafusion-comet] via GitHub
  • Re: [PR] feat: expose per-scan S3 HTTP attempts and retries [datafusion-comet] via GitHub
• [I] Vectorize the lower-bound check in native date_trunc for fine units [datafusion-comet] via GitHub
• Re: [I] Let the data source veto `ExpressionPlacement::MoveTowardsLeafNodes` per function [datafusion] via GitHub
  • Re: [I] Let the data source veto `ExpressionPlacement::MoveTowardsLeafNodes` per function [datafusion] via GitHub
• Re: [I] push_down_filter and common_sub_expression_eliminate fight between them [datafusion] via GitHub

• Earlier messages
• Later messages