(spark) branch master updated: [SPARK-53362][ML][CONNECT] Fix IDFModel local loader bug

2025-08-25 Thread weichenxu123
This is an automated email from the ASF dual-hosted git repository. weichenxu123 pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/spark.git The following commit(s) were added to refs/heads/master by this push: new c13c10fa194a [SPARK-53362][ML][CONNECT

(spark) branch master updated: [SPARK-53328][ML][CONNECT] Improve debuggability for SparkML-connect

2025-08-21 Thread weichenxu123
This is an automated email from the ASF dual-hosted git repository. weichenxu123 pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/spark.git The following commit(s) were added to refs/heads/master by this push: new 12c87cef0d0f [SPARK-53328][ML][CONNECT

(spark) branch master updated: [SPARK-53336][ML][CONNECT] Reset `MLCache.totalMLCacheSizeBytes` when `MLCache.clear()` is called

2025-08-21 Thread weichenxu123
This is an automated email from the ASF dual-hosted git repository. weichenxu123 pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/spark.git The following commit(s) were added to refs/heads/master by this push: new 7e73d0ec8763 [SPARK-53336][ML][CONNECT

(spark) branch master updated: [SPARK-52675][ML][CONNECT] Interrupt hanging ML handlers in tests

2025-07-06 Thread weichenxu123
This is an automated email from the ASF dual-hosted git repository. weichenxu123 pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/spark.git The following commit(s) were added to refs/heads/master by this push: new d89a4af9d761 [SPARK-52675][ML][CONNECT

(spark) branch master updated (81ca4fd5479c -> 618b3da9f642)

2025-06-23 Thread weichenxu123
This is an automated email from the ASF dual-hosted git repository. weichenxu123 pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/spark.git from 81ca4fd5479c [SPARK-52499][SQL] Add more data type tests for SQL UDFs add 618b3da9f642 [SPARK-52534][ML

(spark) branch master updated: [SPARK-52470][ML][CONNECT] Support model summary offloading

2025-06-17 Thread weichenxu123
This is an automated email from the ASF dual-hosted git repository. weichenxu123 pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/spark.git The following commit(s) were added to refs/heads/master by this push: new fd74b5ec6bd6 [SPARK-52470][ML][CONNECT

(spark) branch master updated: [SPARK-52130][FOLLOW-UP][ML][CONNECT] Refine error message when model.summary is evicted

2025-05-30 Thread weichenxu123
This is an automated email from the ASF dual-hosted git repository. weichenxu123 pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/spark.git The following commit(s) were added to refs/heads/master by this push: new 6a89f654bb54 [SPARK-52130][FOLLOW-UP][ML

(spark) branch branch-4.0 updated: [SPARK-52259][ML][CONNECT] Fix Param class binary compatibility

2025-05-22 Thread weichenxu123
This is an automated email from the ASF dual-hosted git repository. weichenxu123 pushed a commit to branch branch-4.0 in repository https://gitbox.apache.org/repos/asf/spark.git The following commit(s) were added to refs/heads/branch-4.0 by this push: new b0932404575a [SPARK-52259][ML

(spark) branch master updated (7fee2912ba8b -> 8f699a415410)

2025-05-22 Thread weichenxu123
This is an automated email from the ASF dual-hosted git repository. weichenxu123 pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/spark.git from 7fee2912ba8b [SPARK-52224][CONNECT][PYTHON] Introduce pyyaml as a dependency for the Python client add

(spark) branch master updated: [SPARK-52229][ML][CONNECT] Improve model size estimation

2025-05-20 Thread weichenxu123
This is an automated email from the ASF dual-hosted git repository. weichenxu123 pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/spark.git The following commit(s) were added to refs/heads/master by this push: new b825e7437ca5 [SPARK-52229][ML][CONNECT

(spark) branch master updated: [SPARK-52192][ML][CONNECT] MLCache loading path check

2025-05-18 Thread weichenxu123
This is an automated email from the ASF dual-hosted git repository. weichenxu123 pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/spark.git The following commit(s) were added to refs/heads/master by this push: new 7e363747b15e [SPARK-52192][ML][CONNECT

(spark) branch master updated: [SPARK-52191][ML][CONNECT] Remove Java deserializer in model local path loader

2025-05-18 Thread weichenxu123
This is an automated email from the ASF dual-hosted git repository. weichenxu123 pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/spark.git The following commit(s) were added to refs/heads/master by this push: new 7fbfeae25175 [SPARK-52191][ML][CONNECT

(spark) branch branch-4.0 updated (a61ddb802b4d -> 6ffc4ed2564e)

2025-05-16 Thread weichenxu123
This is an automated email from the ASF dual-hosted git repository. weichenxu123 pushed a change to branch branch-4.0 in repository https://gitbox.apache.org/repos/asf/spark.git from a61ddb802b4d [SPARK-50762][SQL][TESTS] Add more scalar SQL UDF SQL query tests add 6ffc4ed2564e [SPARK

(spark) branch master updated: [SPARK-52122][ML][CONNECT] Fix DefaultParamsReader RCE vulnerability

2025-05-16 Thread weichenxu123
This is an automated email from the ASF dual-hosted git repository. weichenxu123 pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/spark.git The following commit(s) were added to refs/heads/master by this push: new da2c7dd83a92 [SPARK-52122][ML][CONNECT

(spark) branch master updated: [SPARK-52130][ML][CONNECT] Refine error message, and hide internal spark config

2025-05-15 Thread weichenxu123
This is an automated email from the ASF dual-hosted git repository. weichenxu123 pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/spark.git The following commit(s) were added to refs/heads/master by this push: new f77ab1ec0f9b [SPARK-52130][ML][CONNECT

(spark) branch master updated: [SPARK-52057][ML][CONNECT] Collect Tree size limit warning messages to client

2025-05-11 Thread weichenxu123
This is an automated email from the ASF dual-hosted git repository. weichenxu123 pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/spark.git The following commit(s) were added to refs/heads/master by this push: new b50690a89a81 [SPARK-52057][ML][CONNECT

(spark) branch master updated: [SPARK-52051][ML][CONNECT] Enable model summary when memory control is enabled

2025-05-09 Thread weichenxu123
This is an automated email from the ASF dual-hosted git repository. weichenxu123 pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/spark.git The following commit(s) were added to refs/heads/master by this push: new 568f92017fe1 [SPARK-52051][ML][CONNECT

(spark) branch master updated: [SPARK-52013][CONNECT][ML] Remove `SparkConnectClient.ml_caches`

2025-05-08 Thread weichenxu123
This is an automated email from the ASF dual-hosted git repository. weichenxu123 pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/spark.git The following commit(s) were added to refs/heads/master by this push: new edc631211452 [SPARK-52013][CONNECT][ML

(spark) branch master updated: [SPARK-51974][CONNECT][ML] Limit model size and per-session model cache size

2025-05-07 Thread weichenxu123
This is an automated email from the ASF dual-hosted git repository. weichenxu123 pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/spark.git The following commit(s) were added to refs/heads/master by this push: new ca77925c86d6 [SPARK-51974][CONNECT][ML

(spark) branch master updated (e857f43cde5a -> 894d82898673)

2025-05-05 Thread weichenxu123
This is an automated email from the ASF dual-hosted git repository. weichenxu123 pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/spark.git from e857f43cde5a [MINOR][PS][DOC] Update pandas API on Spark option doc add 894d82898673 [SPARK-51947] Spark

(spark) branch master updated: [SPARK-51867][ML][FOLLOW-UP][ML] Use private to avoid exposing Data class

2025-04-30 Thread weichenxu123
This is an automated email from the ASF dual-hosted git repository. weichenxu123 pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/spark.git The following commit(s) were added to refs/heads/master by this push: new 7019d5e63b72 [SPARK-51867][ML][FOLLOW-UP

(spark) branch master updated (86bf4c84805e -> 6f9bf73c345d)

2025-04-29 Thread weichenxu123
This is an automated email from the ASF dual-hosted git repository. weichenxu123 pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/spark.git from 86bf4c84805e [SPARK-51931][SQL] Add maxBytesPerOutputBatch to limit the number of bytes of Arrow output batch

(spark) branch master updated: [SPARK-51873][ML] For OneVsRest algorithm, allow using save / load to replace cache

2025-04-23 Thread weichenxu123
This is an automated email from the ASF dual-hosted git repository. weichenxu123 pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/spark.git The following commit(s) were added to refs/heads/master by this push: new d9816b709003 [SPARK-51873][ML] For

(spark) branch master updated: [SPARK-51856][ML][CONNECT] Update model size API to count distributed DataFrame size

2025-04-22 Thread weichenxu123
This is an automated email from the ASF dual-hosted git repository. weichenxu123 pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/spark.git The following commit(s) were added to refs/heads/master by this push: new cdd52963281a [SPARK-51856][ML][CONNECT

(spark) branch master updated: [SPARK-51551][ML][PYTHON][CONNECT] For tuning algorithm, allow using save / load to replace cache

2025-03-24 Thread weichenxu123
This is an automated email from the ASF dual-hosted git repository. weichenxu123 pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/spark.git The following commit(s) were added to refs/heads/master by this push: new fe44069e3a10 [SPARK-51551][ML][PYTHON

(spark) branch master updated: [SPARK-51340][ML][CONNECT] Model size estimation

2025-03-19 Thread weichenxu123
This is an automated email from the ASF dual-hosted git repository. weichenxu123 pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/spark.git The following commit(s) were added to refs/heads/master by this push: new 25358ebb68f0 [SPARK-51340][ML][CONNECT

(spark) branch master updated: [SPARK-49615] Bugfix: Make ML column schema validation conforms with spark config `spark.sql.caseSensitive`

2024-10-11 Thread weichenxu123
This is an automated email from the ASF dual-hosted git repository. weichenxu123 pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/spark.git The following commit(s) were added to refs/heads/master by this push: new 8e1d317307d9 [SPARK-49615] Bugfix: Make ML

(spark) branch master updated: [SPARK-48463][ML] Make Binarizer, Bucketizer, VectorAssembler, FeatureHasher, QuantizeDiscretizer, OnehotEncoder, StopWordsRemover, Imputer, Interactor supporting nested

2024-08-13 Thread weichenxu123
This is an automated email from the ASF dual-hosted git repository. weichenxu123 pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/spark.git The following commit(s) were added to refs/heads/master by this push: new e7e082663b94 [SPARK-48463][ML] Make

(spark) branch master updated: [SPARK-48970][PYTHON][ML] Avoid using SparkSession.getActiveSession in spark ML reader/writer

2024-07-23 Thread weichenxu123
This is an automated email from the ASF dual-hosted git repository. weichenxu123 pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/spark.git The following commit(s) were added to refs/heads/master by this push: new fba4c8c20e52 [SPARK-48970][PYTHON][ML

(spark) branch master updated: [SPARK-48941][PYTHON][ML] Replace RDD read / write API invocation with Dataframe read / write API

2024-07-22 Thread weichenxu123
This is an automated email from the ASF dual-hosted git repository. weichenxu123 pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/spark.git The following commit(s) were added to refs/heads/master by this push: new be1afd504a44 [SPARK-48941][PYTHON][ML

(spark) branch master updated: [SPARK-48463][ML] Make StringIndexer supporting nested input columns

2024-07-15 Thread weichenxu123
This is an automated email from the ASF dual-hosted git repository. weichenxu123 pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/spark.git The following commit(s) were added to refs/heads/master by this push: new 9bff2c8bc505 [SPARK-48463][ML] Make

(spark) branch master updated: [SPARK-48883][ML][R] Replace RDD read / write API invocation with Dataframe read / write API

2024-07-12 Thread weichenxu123
This is an automated email from the ASF dual-hosted git repository. weichenxu123 pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/spark.git The following commit(s) were added to refs/heads/master by this push: new 0fa5787d0a6b [SPARK-48883][ML][R] Replace

(spark) branch master updated: [SPARK-47663][CORE][TESTS] add end to end test for task limiting according to different cpu and gpu configurations

2024-04-02 Thread weichenxu123
This is an automated email from the ASF dual-hosted git repository. weichenxu123 pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/spark.git The following commit(s) were added to refs/heads/master by this push: new eb9b12692601 [SPARK-47663][CORE][TESTS

(spark) branch master updated (081809667611 -> c4e4497ff7e7)

2024-02-18 Thread weichenxu123
This is an automated email from the ASF dual-hosted git repository. weichenxu123 pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/spark.git from 081809667611 [MINOR][SQL] Remove `unsupportedOperationMsg` from `CaseInsensitiveStringMap` add

[spark] branch master updated (8394ebb52b9 -> e1a7b84f47b)

2023-10-11 Thread weichenxu123
This is an automated email from the ASF dual-hosted git repository. weichenxu123 pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/spark.git from 8394ebb52b9 [SPARK-45469][CORE][SQL][CONNECT][PYTHON] Replace `toIterator` with `iterator` for `IterableOnce

[spark] branch master updated: [SPARK-45386][SQL]: Fix correctness issue with persist using StorageLevel.NONE on Dataset (#43188)

2023-10-02 Thread weichenxu123
This is an automated email from the ASF dual-hosted git repository. weichenxu123 pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/spark.git The following commit(s) were added to refs/heads/master by this push: new a0c9ab63f3b [SPARK-45386][SQL]: Fix

[spark] branch branch-3.5 updated: [SPARK-44908][ML][CONNECT] Fix cross validator foldCol param functionality

2023-08-23 Thread weichenxu123
This is an automated email from the ASF dual-hosted git repository. weichenxu123 pushed a commit to branch branch-3.5 in repository https://gitbox.apache.org/repos/asf/spark.git The following commit(s) were added to refs/heads/branch-3.5 by this push: new 40ccabfd681 [SPARK-44908][ML

[spark] branch master updated: [SPARK-44908][ML][CONNECT] Fix cross validator foldCol param functionality

2023-08-23 Thread weichenxu123
This is an automated email from the ASF dual-hosted git repository. weichenxu123 pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/spark.git The following commit(s) were added to refs/heads/master by this push: new 0d1b5975b2d [SPARK-44908][ML][CONNECT] Fix

[spark] branch branch-3.5 updated: [SPARK-44909][ML] Skip starting torch distributor log streaming server when it is not available

2023-08-23 Thread weichenxu123
This is an automated email from the ASF dual-hosted git repository. weichenxu123 pushed a commit to branch branch-3.5 in repository https://gitbox.apache.org/repos/asf/spark.git The following commit(s) were added to refs/heads/branch-3.5 by this push: new 4f61662e91a [SPARK-44909][ML] Skip

[spark] branch master updated (00cd5e846b6 -> 80668dc1a36)

2023-08-23 Thread weichenxu123
This is an automated email from the ASF dual-hosted git repository. weichenxu123 pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/spark.git from 00cd5e846b6 [SPARK-44899][PYTHON][DOCS] Refine the docstring of DataFrame.collect add 80668dc1a36 [SPARK

[spark] branch master updated: [SPARK-44264][ML][PYTHON] Incorporating FunctionPickler Into TorchDistributor

2023-07-18 Thread weichenxu123
This is an automated email from the ASF dual-hosted git repository. weichenxu123 pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/spark.git The following commit(s) were added to refs/heads/master by this push: new 2ab70576d68 [SPARK-44264][ML][PYTHON

[spark] branch master updated (0d90f2a8ea0 -> 054f94af95f)

2023-07-11 Thread weichenxu123
This is an automated email from the ASF dual-hosted git repository. weichenxu123 pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/spark.git from 0d90f2a8ea0 [SPARK-44264][ML][PYTHON] Write a Deepspeed Distributed Learning Class DeepspeedTorchDistributor

[spark] branch master updated: [SPARK-43983][PYTHON][ML][CONNECT] Implement cross validator estimator

2023-07-10 Thread weichenxu123
This is an automated email from the ASF dual-hosted git repository. weichenxu123 pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/spark.git The following commit(s) were added to refs/heads/master by this push: new 01918bb9017 [SPARK-43983][PYTHON][ML

[spark] branch master updated (7bc28d54f83 -> 7fcabef2874)

2023-07-03 Thread weichenxu123
This is an automated email from the ASF dual-hosted git repository. weichenxu123 pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/spark.git from 7bc28d54f83 [SPARK-44269][SQL] Assign names to the error class _LEGACY_ERROR_TEMP_[2310-2314] add

[spark] branch master updated (0865c0db923 -> 35b3a18ff04)

2023-06-20 Thread weichenxu123
This is an automated email from the ASF dual-hosted git repository. weichenxu123 pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/spark.git from 0865c0db923 [SPARK-43944][SPARK-43942][SQL][FOLLOWUP] Directly leverage `UnresolvedFunction` for functions

[spark] branch master updated: [SPARK-43982][ML][PYTHON][CONNECT] Implement pipeline estimator for ML on spark connect

2023-06-19 Thread weichenxu123
This is an automated email from the ASF dual-hosted git repository. weichenxu123 pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/spark.git The following commit(s) were added to refs/heads/master by this push: new 6c0c226d901 [SPARK-43982][ML][PYTHON

[spark] branch master updated: [SPARK-43097][FOLLOW-UP][ML] Improve logistic regression model saving

2023-06-17 Thread weichenxu123
This is an automated email from the ASF dual-hosted git repository. weichenxu123 pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/spark.git The following commit(s) were added to refs/heads/master by this push: new 4e0bd3c5717 [SPARK-43097][FOLLOW-UP][ML

[spark] branch master updated: [SPARK-43981][PYTHON][ML] Basic saving / loading implementation for ML on spark connect

2023-06-13 Thread weichenxu123
This is an automated email from the ASF dual-hosted git repository. weichenxu123 pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/spark.git The following commit(s) were added to refs/heads/master by this push: new a5d3bea04eb [SPARK-43981][PYTHON][ML

[spark] branch master updated (fead8a7962a -> 89de4f79e7f)

2023-06-07 Thread weichenxu123
This is an automated email from the ASF dual-hosted git repository. weichenxu123 pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/spark.git from fead8a7962a [SPARK-43993][SQL][TESTS] Add tests for cache artifacts add 89de4f79e7f [SPARK-43790][PYTHON

[spark] branch master updated: [SPARK-43097][ML] New pyspark ML logistic regression estimator implemented on top of distributor

2023-06-06 Thread weichenxu123
This is an automated email from the ASF dual-hosted git repository. weichenxu123 pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/spark.git The following commit(s) were added to refs/heads/master by this push: new 2a82b42bcb1 [SPARK-43097][ML] New pyspark

[spark] branch master updated (51a919ea8d6 -> 1df1d7661a3)

2023-06-05 Thread weichenxu123
This is an automated email from the ASF dual-hosted git repository. weichenxu123 pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/spark.git from 51a919ea8d6 [SPARK-43973][SS][UI] Structured Streaming UI should display failed queries correctly add

[spark] branch master updated (c618dabc96a -> fc3489d8bb6)

2023-06-05 Thread weichenxu123
This is an automated email from the ASF dual-hosted git repository. weichenxu123 pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/spark.git from c618dabc96a Revert "[SPARK-43911][SQL] Use toSet to deduplicate the iterator data to prevent the creati

[spark] branch master updated: [SPARK-43516][ML][FOLLOW-UP] Drop vector type support in Distributed ML for spark connect

2023-06-02 Thread weichenxu123
This is an automated email from the ASF dual-hosted git repository. weichenxu123 pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/spark.git The following commit(s) were added to refs/heads/master by this push: new c3b62708cd6 [SPARK-43516][ML][FOLLOW-UP

[spark] branch master updated: [SPARK-41593][FOLLOW-UP][ML] Torch distributor log streaming server: Avoid duplicated log to stdout redirection

2023-06-01 Thread weichenxu123
This is an automated email from the ASF dual-hosted git repository. weichenxu123 pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/spark.git The following commit(s) were added to refs/heads/master by this push: new 480b14f4b45 [SPARK-41593][FOLLOW-UP][ML

[spark] branch master updated: [SPARK-43081][ML][FOLLOW-UP] Improve torch distributor data loader code

2023-05-31 Thread weichenxu123
This is an automated email from the ASF dual-hosted git repository. weichenxu123 pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/spark.git The following commit(s) were added to refs/heads/master by this push: new c2060e7c0a3 [SPARK-43081][ML][FOLLOW-UP

[spark] branch master updated: [SPARK-41593][FOLLOW-UP] Fix the case torch distributor logging server not shut down

2023-05-30 Thread weichenxu123
This is an automated email from the ASF dual-hosted git repository. weichenxu123 pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/spark.git The following commit(s) were added to refs/heads/master by this push: new e1619653895 [SPARK-41593][FOLLOW-UP] Fix

[spark] branch master updated (7c7b9585a2a -> 0e8e4ae47fb)

2023-05-24 Thread weichenxu123
This is an automated email from the ASF dual-hosted git repository. weichenxu123 pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/spark.git from 7c7b9585a2a [SPARK-43546][PYTHON][CONNECT][TESTS] Complete parity tests of Pandas UDF add 0e8e4ae47fb

[spark] branch master updated (aed6a47580e -> abd864766b0)

2023-04-30 Thread weichenxu123
This is an automated email from the ASF dual-hosted git repository. weichenxu123 pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/spark.git from aed6a47580e [SPARK-43320][SQL][HIVE] Directly call Hive 2.3.9 API add abd864766b0 [SPARK-43081][ML][CONNECT

[spark] branch master updated: [SPARK-42929] make mapInPandas / mapInArrow support "is_barrier"

2023-03-27 Thread weichenxu123
This is an automated email from the ASF dual-hosted git repository. weichenxu123 pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/spark.git The following commit(s) were added to refs/heads/master by this push: new 2a1ac07132b [SPARK-42929] make mapInPandas

[spark] branch master updated: [SPARK-42896][SQL][PYTHON] Make `mapInPandas` / `mapInArrow` support barrier mode execution

2023-03-26 Thread weichenxu123
This is an automated email from the ASF dual-hosted git repository. weichenxu123 pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/spark.git The following commit(s) were added to refs/heads/master by this push: new 06bf544973f [SPARK-42896][SQL][PYTHON

[spark] branch master updated: [SPARK-42732][PYSPARK][CONNECT] Support spark connect session getActiveSession method

2023-03-14 Thread weichenxu123
This is an automated email from the ASF dual-hosted git repository. weichenxu123 pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/spark.git The following commit(s) were added to refs/heads/master by this push: new 753864fedee [SPARK-42732][PYSPARK][CONNECT

[spark] branch master updated (b414b895ffd -> d5b08f8d99b)

2023-01-16 Thread weichenxu123
This is an automated email from the ASF dual-hosted git repository. weichenxu123 pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/spark.git from b414b895ffd [SPARK-41994] Assign SQLSTATE's (1/2) add d5b08f8d99b [SPARK-40264][ML] add batch_infe

[spark] branch master updated: [SPARK-41949][CORE][PYTHON] Make stage scheduling support local-cluster mode

2023-01-10 Thread weichenxu123
This is an automated email from the ASF dual-hosted git repository. weichenxu123 pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/spark.git The following commit(s) were added to refs/heads/master by this push: new 8871f6dfb82 [SPARK-41949][CORE][PYTHON

[spark] branch branch-3.2 updated: [SPARK-41188][CORE][ML] Set executorEnv OMP_NUM_THREADS to be spark.task.cpus by default for spark executor JVM processes

2022-11-19 Thread weichenxu123
This is an automated email from the ASF dual-hosted git repository. weichenxu123 pushed a commit to branch branch-3.2 in repository https://gitbox.apache.org/repos/asf/spark.git The following commit(s) were added to refs/heads/branch-3.2 by this push: new 3b9cca7aa32 [SPARK-41188][CORE][ML

[spark] branch branch-3.3 updated: [SPARK-41188][CORE][ML] Set executorEnv OMP_NUM_THREADS to be spark.task.cpus by default for spark executor JVM processes

2022-11-19 Thread weichenxu123
This is an automated email from the ASF dual-hosted git repository. weichenxu123 pushed a commit to branch branch-3.3 in repository https://gitbox.apache.org/repos/asf/spark.git The following commit(s) were added to refs/heads/branch-3.3 by this push: new f431cdf0944 [SPARK-41188][CORE][ML

[spark] branch master updated: [SPARK-41188][CORE][ML] Set executorEnv OMP_NUM_THREADS to be spark.task.cpus by default for spark executor JVM processes

2022-11-19 Thread weichenxu123
This is an automated email from the ASF dual-hosted git repository. weichenxu123 pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/spark.git The following commit(s) were added to refs/heads/master by this push: new 82a41d8ca27 [SPARK-41188][CORE][ML] Set

[spark] branch branch-3.1 updated: [SPARK-35542][ML] Fix: Bucketizer created for multiple columns with parameters splitsArray, inputCols and outputCols can not be loaded after saving it

2022-08-18 Thread weichenxu123
This is an automated email from the ASF dual-hosted git repository. weichenxu123 pushed a commit to branch branch-3.1 in repository https://gitbox.apache.org/repos/asf/spark.git The following commit(s) were added to refs/heads/branch-3.1 by this push: new a1a2534a01f [SPARK-35542][ML] Fix

[spark] branch branch-3.2 updated: [SPARK-35542][ML] Fix: Bucketizer created for multiple columns with parameters splitsArray, inputCols and outputCols can not be loaded after saving it

2022-08-18 Thread weichenxu123
This is an automated email from the ASF dual-hosted git repository. weichenxu123 pushed a commit to branch branch-3.2 in repository https://gitbox.apache.org/repos/asf/spark.git The following commit(s) were added to refs/heads/branch-3.2 by this push: new 3943427b847 [SPARK-35542][ML] Fix

[spark] branch branch-3.3 updated: [SPARK-35542][ML] Fix: Bucketizer created for multiple columns with parameters splitsArray, inputCols and outputCols can not be loaded after saving it

2022-08-18 Thread weichenxu123
This is an automated email from the ASF dual-hosted git repository. weichenxu123 pushed a commit to branch branch-3.3 in repository https://gitbox.apache.org/repos/asf/spark.git The following commit(s) were added to refs/heads/branch-3.3 by this push: new 87f957dea86 [SPARK-35542][ML] Fix

[spark] branch master updated: [SPARK-35542][ML] Fix: Bucketizer created for multiple columns with parameters splitsArray, inputCols and outputCols can not be loaded after saving it

2022-08-18 Thread weichenxu123
This is an automated email from the ASF dual-hosted git repository. weichenxu123 pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/spark.git The following commit(s) were added to refs/heads/master by this push: new 876ce6a5df1 [SPARK-35542][ML] Fix

[spark] branch branch-3.1 updated: [SPARK-40079] Add Imputer inputCols validation for empty input case

2022-08-15 Thread weichenxu123
This is an automated email from the ASF dual-hosted git repository. weichenxu123 pushed a commit to branch branch-3.1 in repository https://gitbox.apache.org/repos/asf/spark.git The following commit(s) were added to refs/heads/branch-3.1 by this push: new f2453e8a129 [SPARK-40079] Add

[spark] branch branch-3.2 updated: [SPARK-40079] Add Imputer inputCols validation for empty input case

2022-08-15 Thread weichenxu123
This is an automated email from the ASF dual-hosted git repository. weichenxu123 pushed a commit to branch branch-3.2 in repository https://gitbox.apache.org/repos/asf/spark.git The following commit(s) were added to refs/heads/branch-3.2 by this push: new 2b54b48cd85 [SPARK-40079] Add

[spark] branch branch-3.3 updated: [SPARK-40079] Add Imputer inputCols validation for empty input case

2022-08-15 Thread weichenxu123
This is an automated email from the ASF dual-hosted git repository. weichenxu123 pushed a commit to branch branch-3.3 in repository https://gitbox.apache.org/repos/asf/spark.git The following commit(s) were added to refs/heads/branch-3.3 by this push: new 2ee196dbb0b [SPARK-40079] Add

[spark] branch master updated: [SPARK-40079] Add Imputer inputCols validation for empty input case

2022-08-15 Thread weichenxu123
This is an automated email from the ASF dual-hosted git repository. weichenxu123 pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/spark.git The following commit(s) were added to refs/heads/master by this push: new 87094f89655 [SPARK-40079] Add Imputer

[spark] branch master updated: [SPARK-39071][SQL][PYTHON] Add unwrap_udt function for unwrapping UserDefinedType columns

2022-05-10 Thread weichenxu123
This is an automated email from the ASF dual-hosted git repository. weichenxu123 pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/spark.git The following commit(s) were added to refs/heads/master by this push: new daedcd29630 [SPARK-39071][SQL][PYTHON] Add

[spark] branch master updated: [SPARK-36642][SQL] Add df.withMetadata pyspark API

2021-09-23 Thread weichenxu123
This is an automated email from the ASF dual-hosted git repository. weichenxu123 pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/spark.git The following commit(s) were added to refs/heads/master by this push: new ef7441b [SPARK-36642][SQL] Add

[spark] branch master updated: [SPARK-36642][SQL] Add df.withMetadata: a syntax suger to update the metadata of a dataframe

2021-09-07 Thread weichenxu123
This is an automated email from the ASF dual-hosted git repository. weichenxu123 pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/spark.git The following commit(s) were added to refs/heads/master by this push: new cb30683 [SPARK-36642][SQL] Add

[spark] branch branch-3.1 updated: [SPARK-35142][PYTHON][ML] Fix incorrect return type for `rawPredictionUDF` in `OneVsRestModel`

2021-04-21 Thread weichenxu123
This is an automated email from the ASF dual-hosted git repository. weichenxu123 pushed a commit to branch branch-3.1 in repository https://gitbox.apache.org/repos/asf/spark.git The following commit(s) were added to refs/heads/branch-3.1 by this push: new 0208810 [SPARK-35142][PYTHON][ML

[spark] branch master updated (43ad939 -> b6350f5)

2021-04-21 Thread weichenxu123
This is an automated email from the ASF dual-hosted git repository. weichenxu123 pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/spark.git. from 43ad939 [SPARK-35152][SQL] ANSI mode: IntegralDivide throws exception on overflow add b6350f5 [SPARK

[spark] branch branch-3.1 updated: [SPARK-31768][ML][FOLLOWUP] add getMetrics in Evaluators: cleanup

2021-01-25 Thread weichenxu123
This is an automated email from the ASF dual-hosted git repository. weichenxu123 pushed a commit to branch branch-3.1 in repository https://gitbox.apache.org/repos/asf/spark.git The following commit(s) were added to refs/heads/branch-3.1 by this push: new 4cf94f3 [SPARK-31768][ML

[spark] branch master updated (d1177b5 -> cb37c96)

2021-01-25 Thread weichenxu123
This is an automated email from the ASF dual-hosted git repository. weichenxu123 pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/spark.git. from d1177b5 [SPARK-34192][SQL] Move char padding to write side and remove length check on read side too add

[spark] branch branch-3.0 updated: [MINOR][ML] Increase the timeout for StreamingLinearRegressionSuite to 60s

2021-01-19 Thread weichenxu123
This is an automated email from the ASF dual-hosted git repository. weichenxu123 pushed a commit to branch branch-3.0 in repository https://gitbox.apache.org/repos/asf/spark.git The following commit(s) were added to refs/heads/branch-3.0 by this push: new 5a93bcb [MINOR][ML] Increase the

[spark] branch branch-3.1 updated: [MINOR][ML] Increase the timeout for StreamingLinearRegressionSuite to 60s

2021-01-19 Thread weichenxu123
This is an automated email from the ASF dual-hosted git repository. weichenxu123 pushed a commit to branch branch-3.1 in repository https://gitbox.apache.org/repos/asf/spark.git The following commit(s) were added to refs/heads/branch-3.1 by this push: new 32e61b0 [MINOR][ML] Increase the

[spark] branch master updated (32dad1d -> f7ff7ff)

2021-01-19 Thread weichenxu123
This is an automated email from the ASF dual-hosted git repository. weichenxu123 pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/spark.git. from 32dad1d [SPARK-34149][SQL] Refresh cache in v2 `ALTER TABLE .. ADD PARTITION` add f7ff7ff [MINOR][ML

[spark] branch master updated (66cc129 -> f354883)

2021-01-15 Thread weichenxu123
This is an automated email from the ASF dual-hosted git repository. weichenxu123 pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/spark.git. from 66cc129 [SPARK-34132][DOCS][R] Update Roxygen version references to 7.1.1 add f354883 [SPARK-34080][ML

[spark] branch branch-3.1 updated: [MINOR][ML] Increase Bounded MLOR (without regularization) test error tolerance

2020-12-08 Thread weichenxu123
This is an automated email from the ASF dual-hosted git repository. weichenxu123 pushed a commit to branch branch-3.1 in repository https://gitbox.apache.org/repos/asf/spark.git The following commit(s) were added to refs/heads/branch-3.1 by this push: new b0a70ab [MINOR][ML] Increase

[spark] branch master updated (3ac70f1 -> f021f6d)

2020-12-08 Thread weichenxu123
This is an automated email from the ASF dual-hosted git repository. weichenxu123 pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/spark.git. from 3ac70f1 [SPARK-33695][BUILD] Upgrade to jackson to 2.10.5 and jackson-databind to 2.10.5.1 add f021f6d

[spark] branch branch-3.0 updated: [SPARK-33592][ML][PYTHON][3.0] Backport Fix: Pyspark ML Validator params in estimatorParamMaps may be lost after saving and reloading

2020-12-06 Thread weichenxu123
This is an automated email from the ASF dual-hosted git repository. weichenxu123 pushed a commit to branch branch-3.0 in repository https://gitbox.apache.org/repos/asf/spark.git The following commit(s) were added to refs/heads/branch-3.0 by this push: new 8acbe5b [SPARK-33592][ML][PYTHON

[spark] branch master updated (63f9d47 -> 7e759b2)

2020-12-03 Thread weichenxu123
This is an automated email from the ASF dual-hosted git repository. weichenxu123 pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/spark.git. from 63f9d47 [SPARK-33634][SQL][TESTS] Use Analyzer in PlanResolutionSuite add 7e759b2 [SPARK-33520][ML

[spark] branch master updated (a180e02 -> 689c294)

2020-11-18 Thread weichenxu123
This is an automated email from the ASF dual-hosted git repository. weichenxu123 pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/spark.git. from a180e02 [SPARK-32852][SQL][DOC][FOLLOWUP] Revise the documentation of spark.sql.hive.metastore.jars add

[spark] branch master updated (a180e02 -> 689c294)

2020-11-18 Thread weichenxu123
This is an automated email from the ASF dual-hosted git repository. weichenxu123 pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/spark.git. from a180e02 [SPARK-32852][SQL][DOC][FOLLOWUP] Revise the documentation of spark.sql.hive.metastore.jars add

[spark] branch master updated (a180e02 -> 689c294)

2020-11-18 Thread weichenxu123
This is an automated email from the ASF dual-hosted git repository. weichenxu123 pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/spark.git. from a180e02 [SPARK-32852][SQL][DOC][FOLLOWUP] Revise the documentation of spark.sql.hive.metastore.jars add

[spark] branch master updated (a180e02 -> 689c294)

2020-11-18 Thread weichenxu123
This is an automated email from the ASF dual-hosted git repository. weichenxu123 pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/spark.git. from a180e02 [SPARK-32852][SQL][DOC][FOLLOWUP] Revise the documentation of spark.sql.hive.metastore.jars add

[spark] branch master updated (a180e02 -> 689c294)

2020-11-18 Thread weichenxu123
This is an automated email from the ASF dual-hosted git repository. weichenxu123 pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/spark.git. from a180e02 [SPARK-32852][SQL][DOC][FOLLOWUP] Revise the documentation of spark.sql.hive.metastore.jars add

[spark] branch master updated (4335af0 -> a288716)

2020-11-12 Thread weichenxu123
This is an automated email from the ASF dual-hosted git repository. weichenxu123 pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/spark.git. from 4335af0 [MINOR][DOC] spark.executor.memoryOverhead is not cluster-mode only add a288716 [SPARK-32907

[spark] branch master updated (4335af0 -> a288716)

2020-11-12 Thread weichenxu123
This is an automated email from the ASF dual-hosted git repository. weichenxu123 pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/spark.git. from 4335af0 [MINOR][DOC] spark.executor.memoryOverhead is not cluster-mode only add a288716 [SPARK-32907

[spark] branch master updated (4335af0 -> a288716)

2020-11-12 Thread weichenxu123
This is an automated email from the ASF dual-hosted git repository. weichenxu123 pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/spark.git. from 4335af0 [MINOR][DOC] spark.executor.memoryOverhead is not cluster-mode only add a288716 [SPARK-32907

[spark] branch master updated (4335af0 -> a288716)

2020-11-12 Thread weichenxu123
This is an automated email from the ASF dual-hosted git repository. weichenxu123 pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/spark.git. from 4335af0 [MINOR][DOC] spark.executor.memoryOverhead is not cluster-mode only add a288716 [SPARK-32907

[spark] branch master updated (4335af0 -> a288716)

2020-11-12 Thread weichenxu123
This is an automated email from the ASF dual-hosted git repository. weichenxu123 pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/spark.git. from 4335af0 [MINOR][DOC] spark.executor.memoryOverhead is not cluster-mode only add a288716 [SPARK-32907

[spark] 01/02: init

2020-04-20 Thread weichenxu123
This is an automated email from the ASF dual-hosted git repository. weichenxu123 pushed a commit to branch fix_pipeline_tuning in repository https://gitbox.apache.org/repos/asf/spark.git commit c834fe8f335dc74db6346d82b5ce4cf742cba9bb Author: Weichen Xu AuthorDate: Mon Apr 20 17:04:12 2020

  1   2   >