[
https://issues.apache.org/jira/browse/CASSANDRA-20997?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18071681#comment-18071681
]
Arvind Kandpal commented on CASSANDRA-20997:
--------------------------------------------
I tried to fix the Python 3.13 CI test environment issues. I am not completely
sure if this is 100% up to the mark architecturally for the project, but it
works locally. Please take a look and let me know your thoughts!
Here is a quick summary of the changes (done in trunk branch Patch):
# Switched from the OS virtualenv to the native python -m venv to avoid the
pipes module crash on Python 3.13.
# Added pyasyncore and setuptools to the test venv setup so the driver doesn't
crash (since asyncore was removed in 3.12+).
# Fixed the cqlsh prompt Unicode chunk boundary bug in run_cqlsh.py by adding
errors="replace" to the utf-8 decoder.
# Added a symlink to force the Docker python3 command to point to the
requested version so it doesn't default to 3.10.
Here is the local test output showing all tests passing successfully on Python
3.13:
{code:java}
~/Desktop/cassandra-github-code/cassandra$ java_version=11 python_version=3.13
.build/docker/run-tests.sh -a cqlsh-test
Docker version 29.2.1, build a5c7197
Found build image locally.
Running container
cassandra_ubuntu-test_cqlsh-test_jdk11_arch-x86_64_python3-13_1_1__98vnor
1d30034f6135e6349e88b36cf388d5d27684724e254835f14db27b90b1c2455f
Using Java 11
Running run-tests.sh -a cqlsh-test -c 1/1
Apache Ant(TM) version 1.10.12 compiled on January 17 1970
git version 2.34.1
openjdk version "11.0.30" 2026-01-20
OpenJDK Runtime Environment (build 11.0.30+7-post-Ubuntu-1ubuntu122.04)
OpenJDK 64-Bit Server VM (build 11.0.30+7-post-Ubuntu-1ubuntu122.04, mixed
mode, sharing)
javac 11.0.30
Note:
/home/cassandra/cassandra/modules/accord/accord-core/src/main/java/accord/utils/async/AsyncChains.java
uses or overrides a deprecated API.
Note: Recompile with -Xlint:deprecation for details.
Note: Some input files use unchecked or unsafe operations.
Note: Recompile with -Xlint:unchecked for details.
Note: Some input files use unchecked or unsafe operations.
Note: Recompile with -Xlint:unchecked for details.
Note: Some input files use unchecked or unsafe operations.
Note: Recompile with -Xlint:unchecked for details.
git.is-available=true
Repository state is dirty
.build/docker/run-tests.sh | 1 +
.build/docker/ubuntu-test.docker | 10 ++++++----
.build/run-python-dtests.sh | 5 +++--
pylib/cassandra-cqlsh-tests.sh | 6 ++++--
pylib/cqlshlib/test/run_cqlsh.py | 2 +-
5 files changed, 15 insertions(+), 9 deletions(-)
git.worktree.name=cassandra
git.branch.name=CASSANDRA-20997-python3.13-support
Collecting pip<25.0
Using cached pip-24.3.1-py3-none-any.whl.metadata (3.7 kB)
Collecting setuptools<70.0.0,>=65.5.0
Using cached setuptools-69.5.1-py3-none-any.whl.metadata (6.2 kB)
Collecting wheel
Using cached wheel-0.46.3-py3-none-any.whl.metadata (2.4 kB)
Collecting pyasyncore
Downloading pyasyncore-1.0.5-py3-none-any.whl.metadata (4.3 kB)
Collecting packaging>=24.0 (from wheel)
Using cached packaging-26.0-py3-none-any.whl.metadata (3.3 kB)
Using cached pip-24.3.1-py3-none-any.whl (1.8 MB)
Using cached setuptools-69.5.1-py3-none-any.whl (894 kB)
Using cached wheel-0.46.3-py3-none-any.whl (30 kB)
Downloading pyasyncore-1.0.5-py3-none-any.whl (10 kB)
Using cached packaging-26.0-py3-none-any.whl (74 kB)
Installing collected packages: pyasyncore, setuptools, pip, packaging, wheel
Attempting uninstall: pip
Found existing installation: pip 25.3
Uninstalling pip-25.3:
Successfully uninstalled pip-25.3
Successfully installed packaging-26.0 pip-24.3.1 pyasyncore-1.0.5
setuptools-69.5.1 wheel-0.46.3
[notice] A new release of pip is available: 24.3.1 -> 26.0.1
[notice] To update, run: pip install --upgrade pip
Obtaining cassandra-driver from
git+https://github.com/apache/[email protected]#egg=cassandra-driver
(from -r /home/cassandra/cassandra/pylib/requirements.txt (line 20))
Cloning https://github.com/apache/cassandra-python-driver.git (to revision
3.29.0) to ./build/venv/src/cassandra-driver
Running command git clone --filter=blob:none --quiet
https://github.com/apache/cassandra-python-driver.git
/home/cassandra/cassandra/build/venv/src/cassandra-driver
Running command git checkout -q 5770b6d3489910aa1e7af9a652681a22b17b498a
Resolved https://github.com/apache/cassandra-python-driver.git to commit
5770b6d3489910aa1e7af9a652681a22b17b498a
Preparing metadata (setup.py): started
Preparing metadata (setup.py): finished with status 'done'
Obtaining ccm from
git+https://github.com/apache/cassandra-ccm.git@cassandra-test#egg=ccm (from -r
/home/cassandra/cassandra/pylib/requirements.txt (line 22))
Cloning https://github.com/apache/cassandra-ccm.git (to revision
cassandra-test) to ./build/venv/src/ccm
Running command git clone --filter=blob:none --quiet
https://github.com/apache/cassandra-ccm.git
/home/cassandra/cassandra/build/venv/src/ccm
Running command git checkout -q f55ac7c916764b90cca4c9086f6f8ff1e57645fe
Resolved https://github.com/apache/cassandra-ccm.git to commit
f55ac7c916764b90cca4c9086f6f8ff1e57645fe
Preparing metadata (setup.py): started
Preparing metadata (setup.py): finished with status 'done'
Collecting coverage (from -r /home/cassandra/cassandra/pylib/requirements.txt
(line 23))
Downloading
coverage-7.13.5-cp313-cp313-manylinux1_x86_64.manylinux_2_28_x86_64.manylinux_2_5_x86_64.whl.metadata
(8.5 kB)
Collecting pytest (from -r /home/cassandra/cassandra/pylib/requirements.txt
(line 24))
Downloading pytest-9.0.2-py3-none-any.whl.metadata (7.6 kB)
Collecting wcwidth (from -r /home/cassandra/cassandra/pylib/requirements.txt
(line 25))
Downloading wcwidth-0.6.0-py3-none-any.whl.metadata (30 kB)
Collecting geomet<0.3,>=0.1 (from cassandra-driver->-r
/home/cassandra/cassandra/pylib/requirements.txt (line 20))
Using cached geomet-0.2.1.post1-py3-none-any.whl.metadata (1.0 kB)
Collecting pyYaml (from ccm->-r
/home/cassandra/cassandra/pylib/requirements.txt (line 22))
Downloading
pyyaml-6.0.3-cp313-cp313-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl.metadata
(2.4 kB)
Collecting six>=1.4.1 (from ccm->-r
/home/cassandra/cassandra/pylib/requirements.txt (line 22))
Downloading six-1.17.0-py2.py3-none-any.whl.metadata (1.7 kB)
Collecting psutil (from ccm->-r
/home/cassandra/cassandra/pylib/requirements.txt (line 22))
Downloading
psutil-7.2.2-cp36-abi3-manylinux2010_x86_64.manylinux_2_12_x86_64.manylinux_2_28_x86_64.whl.metadata
(22 kB)
Collecting packaging<21 (from ccm->-r
/home/cassandra/cassandra/pylib/requirements.txt (line 22))
Using cached packaging-20.9-py2.py3-none-any.whl.metadata (13 kB)
Collecting iniconfig>=1.0.1 (from pytest->-r
/home/cassandra/cassandra/pylib/requirements.txt (line 24))
Downloading iniconfig-2.3.0-py3-none-any.whl.metadata (2.5 kB)
INFO: pip is looking at multiple versions of pytest to determine which version
is compatible with other requirements. This could take a while.
Collecting pytest (from -r /home/cassandra/cassandra/pylib/requirements.txt
(line 24))
Downloading pytest-9.0.1-py3-none-any.whl.metadata (7.6 kB)
Downloading pytest-9.0.0-py3-none-any.whl.metadata (7.6 kB)
Downloading pytest-8.4.2-py3-none-any.whl.metadata (7.7 kB)
Collecting pluggy<2,>=1.5 (from pytest->-r
/home/cassandra/cassandra/pylib/requirements.txt (line 24))
Downloading pluggy-1.6.0-py3-none-any.whl.metadata (4.8 kB)
Collecting pygments>=2.7.2 (from pytest->-r
/home/cassandra/cassandra/pylib/requirements.txt (line 24))
Downloading pygments-2.20.0-py3-none-any.whl.metadata (2.5 kB)
Collecting click (from geomet<0.3,>=0.1->cassandra-driver->-r
/home/cassandra/cassandra/pylib/requirements.txt (line 20))
Downloading click-8.3.2-py3-none-any.whl.metadata (2.6 kB)
Collecting pyparsing>=2.0.2 (from packaging<21->ccm->-r
/home/cassandra/cassandra/pylib/requirements.txt (line 22))
Downloading pyparsing-3.3.2-py3-none-any.whl.metadata (5.8 kB)
Downloading
coverage-7.13.5-cp313-cp313-manylinux1_x86_64.manylinux_2_28_x86_64.manylinux_2_5_x86_64.whl
(253 kB)
Downloading pytest-8.4.2-py3-none-any.whl (365 kB)
Downloading wcwidth-0.6.0-py3-none-any.whl (94 kB)
Using cached geomet-0.2.1.post1-py3-none-any.whl (18 kB)
Downloading iniconfig-2.3.0-py3-none-any.whl (7.5 kB)
Using cached packaging-20.9-py2.py3-none-any.whl (40 kB)
Downloading pluggy-1.6.0-py3-none-any.whl (20 kB)
Downloading pygments-2.20.0-py3-none-any.whl (1.2 MB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1.2/1.2 MB 298.4 kB/s eta 0:00:00
Downloading six-1.17.0-py2.py3-none-any.whl (11 kB)
Downloading
psutil-7.2.2-cp36-abi3-manylinux2010_x86_64.manylinux_2_12_x86_64.manylinux_2_28_x86_64.whl
(155 kB)
Downloading
pyyaml-6.0.3-cp313-cp313-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl
(801 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 801.6/801.6 kB 674.3 kB/s eta 0:00:00
Downloading pyparsing-3.3.2-py3-none-any.whl (122 kB)
Downloading click-8.3.2-py3-none-any.whl (108 kB)
Installing collected packages: wcwidth, six, pyYaml, pyparsing, pygments,
psutil, pluggy, iniconfig, coverage, click, packaging, geomet, pytest, ccm,
cassandra-driver
Attempting uninstall: packaging
Found existing installation: packaging 26.0
Uninstalling packaging-26.0:
Successfully uninstalled packaging-26.0
DEPRECATION: Legacy editable install of ccm from
git+https://github.com/apache/cassandra-ccm.git@cassandra-test#egg=ccm (from -r
/home/cassandra/cassandra/pylib/requirements.txt (line 22)) (setup.py develop)
is deprecated. pip 25.0 will enforce this behaviour change. A possible
replacement is to add a pyproject.toml or enable --use-pep517, and use
setuptools >= 64. If the resulting installation is not behaving as expected,
try using --config-settings editable_mode=compat. Please consult the setuptools
documentation for more information. Discussion can be found at
https://github.com/pypa/pip/issues/11457
Running setup.py develop for ccm
DEPRECATION: Legacy editable install of cassandra-driver from
git+https://github.com/apache/[email protected]#egg=cassandra-driver
(from -r /home/cassandra/cassandra/pylib/requirements.txt (line 20)) (setup.py
develop) is deprecated. pip 25.0 will enforce this behaviour change. A possible
replacement is to add a pyproject.toml or enable --use-pep517, and use
setuptools >= 64. If the resulting installation is not behaving as expected,
try using --config-settings editable_mode=compat. Please consult the setuptools
documentation for more information. Discussion can be found at
https://github.com/pypa/pip/issues/11457
Running setup.py develop for cassandra-driver
ERROR: pip's dependency resolver does not currently take into account all the
packages that are installed. This behaviour is the source of the following
dependency conflicts.
wheel 0.46.3 requires packaging>=24.0, but you have packaging 20.9 which is
incompatible.
Successfully installed cassandra-driver-3.29.0 ccm-3.1.6.dev1347 click-8.3.2
coverage-7.13.5 geomet-0.2.1.post1 iniconfig-2.3.0 packaging-20.9 pluggy-1.6.0
psutil-7.2.2 pyYaml-6.0.3 pygments-2.20.0 pyparsing-3.3.2 pytest-8.4.2
six-1.17.0 wcwidth-0.6.0
[notice] A new release of pip is available: 24.3.1 -> 26.0.1
[notice] To update, run: pip install --upgrade pip
-e
git+https://github.com/apache/cassandra-python-driver.git@1a947f8437d10e52a66488c1eab45456d089a92d#egg=cassandra_driver
-e
git+https://github.com/apache/cassandra-ccm.git@0b19c8eef37db2e11ecf5b27c42c883be88830ce#egg=ccm
click==8.3.2
coverage==7.13.5
geomet==0.2.1.post1
iniconfig==2.3.0
packaging==20.9
pluggy==1.6.0
psutil==7.2.2
pyasyncore==1.0.5
Pygments==2.20.0
pyparsing==3.3.2
pytest==8.4.2
PyYAML==6.0.3
setuptools==69.5.1
six==1.17.0
wcwidth==0.6.0
wheel==0.46.3
/home/cassandra/cassandra/build/venv/bin/ccm:4: DeprecationWarning:
pkg_resources is deprecated as an API. See
https://setuptools.pypa.io/en/latest/pkg_resources.html
__import__('pkg_resources').require('ccm==3.1.6.dev1347')
/home/cassandra/cassandra/build/venv/src/ccm/ccmlib/common.py:744:
SyntaxWarning: invalid escape sequence '\d'
ver_pattern = '\"(\d+\.\d+).*\"'
/home/cassandra/cassandra/build/venv/src/ccm/ccmlib/common.py:748:
SyntaxWarning: invalid escape sequence '\d'
ver_pattern = '\"(\d+).*\"'
/home/cassandra/cassandra/build/venv/src/ccm/ccmlib/cmds/cluster_cmds.py:136:
SyntaxWarning: invalid escape sequence '\w'
if common.is_win() and os.path.exists('c:\windows\system32\java.exe'):
/home/cassandra/cassandra/build/venv/src/ccm/ccmlib/cmds/cluster_cmds.py:137:
SyntaxWarning: invalid escape sequence '\w'
print_("""WARN: c:\windows\system32\java.exe exists.
/home/cassandra/cassandra/build/venv/src/ccm/ccmlib/repository.py:259:
SyntaxWarning: invalid escape sequence '\['
elif re.search('\[.*?(ahead|behind).*?\]', status.decode("utf-8")) is not
None: # status looks like '## trunk...origin/trunk [ahead 1, behind 29]\n'
/home/cassandra/cassandra/build/venv/src/ccm/ccmlib/repository.py:450:
SyntaxWarning: invalid escape sequence '\.'
tag_regex = re.compile('^refs/tags/cassandra-([0-9]+\.[0-9]+\.[0-9]+-.*$)')
/home/cassandra/cassandra/build/venv/src/ccm/ccmlib/repository.py:453:
SyntaxWarning: invalid escape sequence '\.'
tag_regex = re.compile('^refs/tags/cassandra-([0-9]+\.[0-9]+\.[0-9]+$)')
/home/cassandra/cassandra/build/venv/src/ccm/ccmlib/node.py:110: SyntaxWarning:
invalid escape sequence '\s'
_sstable_regexp =
re.compile('((?P<keyspace>[^\s-]+)-(?P<cf>[^\s-]+)-)?(?P<tmp>tmp(link)?-)?(?P<version>[^\s-]+)-(?P<number>\d+)-(?P<format>([a-z]+)-)?(?P<suffix>[a-zA-Z]+)\.[a-zA-Z0-9]+$')
/home/cassandra/cassandra/build/venv/src/ccm/ccmlib/node.py:139: SyntaxWarning:
invalid escape sequence '\.'
match = re.search('name="base\.version" value="([0-9.]+)[^"]*"', line)
/home/cassandra/cassandra/build/venv/src/ccm/ccmlib/node.py:406: SyntaxWarning:
invalid escape sequence '\.'
return float('.'.join(re.split('\.|-',version.vstring)[:2]))
/home/cassandra/cassandra/build/venv/src/ccm/ccmlib/node.py:1720:
SyntaxWarning: invalid escape sequence '\l'
for_statement = "for %%i in (\"%CASSANDRA_HOME%\lib\*.jar\")"
/home/cassandra/cassandra/build/venv/src/ccm/ccmlib/node.py:1931:
SyntaxWarning: invalid escape sequence '\s'
jmx_port_pattern = '^\s+\$JMX_PORT='
/home/cassandra/cassandra/build/venv/src/ccm/ccmlib/node.py:2245:
SyntaxWarning: invalid escape sequence '\s'
common.replace_in_file(dst, "^\s+\$JMX_PORT=", " $JMX_PORT=\"" +
self.jmx_port + "\"")
/home/cassandra/cassandra/build/venv/src/ccm/ccmlib/remote.py:205:
SyntaxWarning: "is not" with 'str' literal. Did you mean "!="?
if not profile is None and not profile is "None":
/home/cassandra/cassandra/build/venv/src/ccm/ccmlib/dse/dse_node.py:500:
SyntaxWarning: invalid escape sequence '\d'
node_num = re.search(u'node(\d+)', self.name).group(1)
/home/cassandra/cassandra/build/venv/src/ccm/ccmlib/dse/dse_node.py:574:
SyntaxWarning: invalid escape sequence '\.'
match = re.search('^dse(?:-core)?-([0-9.]+)(?:-.*)?\.jar', file)
/home/cassandra/cassandra/build/venv/src/ccm/ccmlib/hcd/hcd_cluster.py:141:
SyntaxWarning: invalid escape sequence '\.'
match = re.search('^hcd(?:-core)?-([0-9.]+)(?:-.*)?\.jar', file)
test does not appear to be a valid cluster (use ccm list to view valid clusters)
/home/cassandra/cassandra/build/venv/bin/ccm:4: DeprecationWarning:
pkg_resources is deprecated as an API. See
https://setuptools.pypa.io/en/latest/pkg_resources.html
__import__('pkg_resources').require('ccm==3.1.6.dev1347')
Current cluster is now: test
11:02:05,301 ccm DEBUG using balanced tokens for non-vnode cluster
/home/cassandra/cassandra/build/venv/bin/ccm:4: DeprecationWarning:
pkg_resources is deprecated as an API. See
https://setuptools.pypa.io/en/latest/pkg_resources.html
__import__('pkg_resources').require('ccm==3.1.6.dev1347')
/home/cassandra/cassandra/build/venv/bin/ccm:4: DeprecationWarning:
pkg_resources is deprecated as an API. See
https://setuptools.pypa.io/en/latest/pkg_resources.html
__import__('pkg_resources').require('ccm==3.1.6.dev1347')
/home/cassandra/cassandra/build/venv/bin/ccm:4: DeprecationWarning:
pkg_resources is deprecated as an API. See
https://setuptools.pypa.io/en/latest/pkg_resources.html
__import__('pkg_resources').require('ccm==3.1.6.dev1347')
11:02:11,629 ccm INFO Supported Java versions for Cassandra distribution in
'/home/cassandra/cassandra': [11, 17, 21]
11:02:11,777 ccm INFO node1: Using the current Java 11 available on PATH for
the current invocation of Cassandra 6.0.
11:02:12,131 ccm INFO Supported Java versions for Cassandra distribution in
'/home/cassandra/cassandra': [11, 17, 21]
11:02:12,306 ccm INFO node1: Using the current Java 11 available on PATH for
the current invocation of Cassandra 6.0.
11:02:12,442 ccm INFO Starting node1 with
JAVA_HOME=/usr/lib/jvm/java-11-openjdk-amd64 java_version=11
cassandra_version=6.0, install_dir=/home/cassandra/cassandra
============================= test session starts ==============================
platform linux -- Python 3.13.12, pytest-8.4.2, pluggy-1.6.0
rootdir: /home/cassandra/cassandra/pylib
collected 135 items
test/test_authproviderhandling.py ................ [ 11%]
test/test_constants.py . [ 12%]
test/test_copyutil.py . [ 13%]
test/test_cql_parsing.py ......................... [ 31%]
test/test_cqlsh_completion.py .......................................... [ 62%]
..... [ 66%]
test/test_cqlsh_output.py ........................................ [ 96%]
test/test_unicode.py ..... [100%]
=============================== warnings summary ===============================
saferscanner.py:23
/home/cassandra/cassandra/pylib/cqlshlib/saferscanner.py:23:
DeprecationWarning: module 'sre_constants' is deprecated
from sre_constants import BRANCH, SUBPATTERN, GROUPREF, GROUPREF_IGNORE,
GROUPREF_EXISTS
../../build/venv/src/cassandra-driver/cassandra/util.py:43
/home/cassandra/cassandra/build/venv/src/cassandra-driver/cassandra/util.py:43:
DeprecationWarning: datetime.datetime.utcfromtimestamp() is deprecated and
scheduled for removal in a future version. Use timezone-aware objects to
represent datetimes in UTC: datetime.datetime.fromtimestamp(timestamp,
datetime.UTC).
UTC_DATETIME_EPOC = datetime.datetime.utcfromtimestamp(0)
-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
- generated xml file: /home/cassandra/cassandra/build/test/output/cqlshlib.xml -
================= 135 passed, 2 warnings in 833.46s (0:13:53) ==================
/home/cassandra/cassandra/build/venv/bin/ccm:4: DeprecationWarning:
pkg_resources is deprecated as an API. See
https://setuptools.pypa.io/en/latest/pkg_resources.html
__import__('pkg_resources').require('ccm==3.1.6.dev1347')
Generating Test Summary for test files found under
/home/cassandra/cassandra/build/test/output
[Fatal Error] cqlshlib.xml:1:21429: XML document structures must start and end
within the same entity.
The file /home/cassandra/cassandra/build/test/output/cqlshlib.xml is not a
valid XML document. It is possibly corrupted.
[Test Summary] Run: 0, Failed: 0, Errors: 0, Skipped: 0
+ exit 0
+ exit 0
{code}
> Support Python 3.13
> -------------------
>
> Key: CASSANDRA-20997
> URL: https://issues.apache.org/jira/browse/CASSANDRA-20997
> Project: Apache Cassandra
> Issue Type: Improvement
> Components: Build, Dependencies, Tool/cqlsh
> Reporter: fmaurer
> Assignee: Arvind Kandpal
> Priority: Normal
> Time Spent: 2h 50m
> Remaining Estimate: 0h
>
> Using the cassandra install for debian using the latest deb of 5.0.6, the
> database does not come up with JDK-21:
> {{java.lang.UnsupportedOperationException: The Security Manager is deprecated
> and will be removed in a future release}}
>
> This can be fixed with
> {{JVM_EXTRA_OPTS="$JVM_EXTRA_OPTS -Djava.security.manager=allow"}}
> {{in /etc/cassandra/cassandra-env.sh}}
> then after {{systemctl restart cassandra}}
> the `cqlsh` fails with
> {{Warning: unsupported version of Python, required 3.6-3.11 but found 3.13}}
> {{No appropriate Python interpreter found.}}
>
> I would like to see a project of this size to support a more recent
> technology stack as well.
> Is there any plan to fix these issues? Thanks!
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
