Source: grpc Version: 1.51.1-5 Severity: serious Tags: ftbfs patch The abseil 20240722.0 transition has started (#1081553) and grpc FTBFS with the new version:
/usr/bin/c++ -I/build/reproducible-path/grpc-1.51.1/include -I/build/reproducible-path/grpc-1.51.1 -I/build/reproducible-path/grpc-1.51.1/third_party/address_sorting/include -I/build/reproducible-path/grpc-1.51.1/src/core/ext/upb-generated -I/build/reproducible-path/grpc-1.51.1/src/core/ext/upbdefs-generated -I/build/reproducible-path/grpc-1.51.1/third_party/upb -I/build/reproducible-path/grpc-1.51.1/third_party/xxhash -g -O2 -ffile-prefix-map=/build/reproducible-path/grpc-1.51.1=. -fstack-protector-strong -fstack-clash-protection -Wformat -Werror=format-security -fcf-protection -Wdate-time -D_FORTIFY_SOURCE=2 -std=c++14 -fPIC -pthread -DNOMINMAX -MD -MT CMakeFiles/grpc.dir/src/core/lib/iomgr/wakeup_fd_pipe.cc.o -MF CMakeFiles/grpc.dir/src/core/lib/iomgr/wakeup_fd_pipe.cc.o.d -o CMakeFiles/grpc.dir/src/core/lib/iomgr/wakeup_fd_pipe.cc.o -c /build/reproducible-path/grpc-1.51.1/src/core/lib/iomgr/wakeup_fd_pipe.cc /build/reproducible-path/grpc-1.51.1/src/core/lib/iomgr/tcp_posix.cc: In function ‘void update_rcvlowat({anonymous}::grpc_tcp*)’: /build/reproducible-path/grpc-1.51.1/src/core/lib/iomgr/tcp_posix.cc:861:19: error: ‘StrCat’ is not a member of ‘absl’ 861 | absl::StrCat("Cannot set SO_RCVLOWAT on fd=", tcp->fd, | ^~~~~~ /build/reproducible-path/grpc-1.51.1/src/core/lib/iomgr/tcp_posix.cc: In function ‘bool tcp_do_read({anonymous}::grpc_tcp*, grpc_error_handle*)’: /build/reproducible-path/grpc-1.51.1/src/core/lib/iomgr/tcp_posix.cc:951:58: error: ‘StrCat’ is not a member of ‘absl’ 951 | tcp_annotate_error(absl::InternalError(absl::StrCat( | ^~~~~~ This looks like some missing #includes. I'll pick my way through and find them. In the meantime, I see that 1.59.5-1 already supports the newer abseil, here's a patch to get 1.59.5-1 building gain (essentially, merging the latest commit from my MR to unstable).
diff --git a/debian/changelog b/debian/changelog index 993c66177d..48f17b9a92 100644 --- a/debian/changelog +++ b/debian/changelog @@ -1,3 +1,12 @@ +grpc (1.59.5-1.1) unstable; urgency=medium + + [ Stefano Rivera <stefa...@debian.org> ] + * Patch: Cython 3 support. + * Patch: Python 3.13 support (closes: #1084603). + * Patch: GCC-14 support (closes: #1075045). + + -- Stefano Rivera <stefa...@debian.org> Mon, 10 Mar 2025 09:06:20 -0400 + grpc (1.59.5-1) experimental; urgency=medium * New major upstream release. diff --git a/debian/control b/debian/control index 38aff01ec8..4cf6990138 100644 --- a/debian/control +++ b/debian/control @@ -12,7 +12,7 @@ Build-Depends: dpkg-dev (>= 1.22.5), debhelper-compat (= 13), cmake, pkgconf, ruby-rspec, ruby-simplecov, dh-python, python3-all-dev, - cython3-legacy, + cython3, python3-six, python3-setuptools, python3-pkg-resources, diff --git a/debian/patches/cython3-deadlocks.patch b/debian/patches/cython3-deadlocks.patch new file mode 100644 index 0000000000..fd0362de99 --- /dev/null +++ b/debian/patches/cython3-deadlocks.patch @@ -0,0 +1,250 @@ +From d6cdc919ba40eb650ed29f4ef3ed2813528db28e Mon Sep 17 00:00:00 2001 +From: Xuan Wang <xua...@google.com> +Date: Mon, 14 Oct 2024 15:16:27 -0700 +Subject: [PATCH 1/3] [Cython AIO] Fix Aio tests time out issue (#37917) + +### What's happening +Some of our asyncio tests began timing out following a Cython upgrade to 3.0. This issue occurs consistently across both our Bazel and setup.py test environments. + +### Why the time out +After some investigation, we found that our code here:https://github.com/grpc/grpc/blob/4ffcdd4ab7976cac397fcff25b6f952d0b5c8168/src/python/grpcio/grpc/_cython/_cygrpc/aio/completion_queue.pyx.pxi#L115-L116 + +Was translated to this in Cython 0.29: +``` +__pyx_f_7_cython_6cygrpc__unified_socket_write(__pyx_v_self->_write_fd); +``` + +And it changed to this in Cython 3.0: +``` +__pyx_f_7_cython_6cygrpc__unified_socket_write(__pyx_v_self->_write_fd); if (unlikely(__Pyx_ErrOccurredWithGIL())) __PYX_ERR(7, 136, __pyx_L1_error) +``` + +Which indicates that this `nogil` function `_unified_socket_write` now requires GIL. + +### What's new in Cython 3 +* Cython 3 `cdef` functions with `void` return type will default to use `except *` as exception specification. +* If function have `void` return type and defined as `nogil`, Cython will always re-acquire the GIL after the function call to check if an exception has been raised. +* In some cases, this will cause a deadlock, especially if the function was called inside another `nogil` function. + +### What's the fix +* This PR changes those functions to use `noexcept` as exception specification since we don't expect them to throw any exception, and this is also the suggested workarounds in Cython documentation: https://cython.readthedocs.io/en/latest/src/userguide/language_basics.html#error-return-values + +### Test +* Tested locally by running `bazel test`, time our rate decreased from 5% to 0.3% in 3000 runs and 10s test time out. +<!-- + +If you know who should review your pull request, please assign it to that +person, otherwise the pull request would get assigned randomly. + +If your pull request is for a specific language, please add the appropriate +lang label. + +--> + +Closes #37917 + +COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/37917 from XuanWang-Amos:fix_cython_aio 063d27aee9192a26f36d16e6f056dcab1a48b9c9 +PiperOrigin-RevId: 685851320 +--- + .../grpc/_cython/_cygrpc/aio/completion_queue.pxd.pxi | 4 ++-- + .../grpc/_cython/_cygrpc/aio/completion_queue.pyx.pxi | 6 +++--- + 2 files changed, 5 insertions(+), 5 deletions(-) + +Origin: upstream, https://github.com/grpc/grpc/pull/38003 +--- a/src/python/grpcio/grpc/_cython/_cygrpc/aio/completion_queue.pxd.pxi ++++ b/src/python/grpcio/grpc/_cython/_cygrpc/aio/completion_queue.pxd.pxi +@@ -23,7 +23,7 @@ + int win_socket_send "send" (WIN_SOCKET s, const char *buf, int len, int flags) + + +-cdef void _unified_socket_write(int fd) nogil ++cdef void _unified_socket_write(int fd) noexcept nogil + + + cdef class BaseCompletionQueue: +--- a/src/python/grpcio/grpc/_cython/_cygrpc/aio/completion_queue.pyx.pxi ++++ b/src/python/grpcio/grpc/_cython/_cygrpc/aio/completion_queue.pyx.pxi +@@ -22,12 +22,12 @@ + cdef bint _has_fd_monitoring = True + + IF UNAME_SYSNAME == "Windows": +- cdef void _unified_socket_write(int fd) nogil: ++ cdef void _unified_socket_write(int fd) noexcept nogil: + win_socket_send(<WIN_SOCKET>fd, b"1", 1, 0) + ELSE: + from posix cimport unistd + +- cdef void _unified_socket_write(int fd) nogil: ++ cdef void _unified_socket_write(int fd) noexcept nogil: + unistd.write(fd, b"1", 1) + + +--- a/src/python/grpcio/grpc/_cython/_cygrpc/credentials.pxd.pxi ++++ b/src/python/grpcio/grpc/_cython/_cygrpc/credentials.pxd.pxi +@@ -26,9 +26,9 @@ + grpc_credentials_plugin_metadata_cb cb, void *user_data, + grpc_metadata creds_md[GRPC_METADATA_CREDENTIALS_PLUGIN_SYNC_MAX], + size_t *num_creds_md, grpc_status_code *status, +- const char **error_details) except * with gil ++ const char **error_details) except -1 with gil + +-cdef void _destroy(void *state) except * with gil ++cdef void _destroy(void *state) noexcept nogil + + + cdef class MetadataPluginCallCredentials(CallCredentials): +--- a/src/python/grpcio/grpc/_cython/_cygrpc/credentials.pyx.pxi ++++ b/src/python/grpcio/grpc/_cython/_cygrpc/credentials.pyx.pxi +@@ -12,6 +12,7 @@ + # See the License for the specific language governing permissions and + # limitations under the License. + ++import atexit + + def _spawn_callback_in_thread(cb_func, args): + t = ForkManagedThread(target=cb_func, args=args) +@@ -41,7 +42,7 @@ + grpc_metadata creds_md[GRPC_METADATA_CREDENTIALS_PLUGIN_SYNC_MAX], + size_t *num_creds_md, + grpc_status_code *status, +- const char **error_details) except * with gil: ++ const char **error_details) except -1 with gil: + cdef size_t metadata_count + cdef grpc_metadata *c_metadata + def callback(metadata, grpc_status_code status, bytes error_details): +@@ -65,11 +66,59 @@ + return 0 # Asynchronous return + + +-cdef void _destroy(void *state) except * with gil: +- cpython.Py_DECREF(<object>state) ++# Protects access to GIL from _destroy() and to g_shutting_down. ++# Do NOT hold this while holding GIL to prevent a deadlock. ++cdef mutex g_shutdown_mu ++# Number of C-core clean up calls in progress. Set to -1 when Python is shutting ++# down. ++cdef int g_shutting_down = 0 ++ ++# This is called by C-core when the plugin is destroyed, which may race between ++# GIL destruction during process shutdown. Since GIL destruction happens after ++# Python's exit handlers, we mark that Python is shutting down from an exit ++# handler and don't grab GIL in this function afterwards using a C mutex. ++cdef void _destroy(void *state) noexcept nogil: ++ global g_shutdown_mu ++ global g_shutting_down ++ g_shutdown_mu.lock() ++ if g_shutting_down > -1: ++ g_shutting_down += 1 ++ g_shutdown_mu.unlock() ++ with gil: ++ cpython.Py_DECREF(<object>state) ++ g_shutdown_mu.lock() ++ g_shutting_down -= 1 ++ g_shutdown_mu.unlock() + grpc_shutdown() + + ++g_shutdown_handler_registered = False ++ ++def _maybe_register_shutdown_handler(): ++ global g_shutdown_handler_registered ++ if g_shutdown_handler_registered: ++ return ++ g_shutdown_handler_registered = True ++ atexit.register(_on_shutdown) ++ ++cdef void _on_shutdown() noexcept nogil: ++ global g_shutdown_mu ++ global g_shutting_down ++ # Wait for up to ~2s if C-core is still cleaning up. ++ cdef int wait_ms = 10 ++ while wait_ms < 1500: ++ g_shutdown_mu.lock() ++ if g_shutting_down == 0: ++ g_shutting_down = -1 ++ g_shutdown_mu.unlock() ++ return ++ g_shutdown_mu.unlock() ++ with gil: ++ time.sleep(wait_ms / 1000) ++ wait_ms = wait_ms * 2 ++ with gil: ++ _LOGGER.error('Timed out waiting for C-core clean-up') ++ + cdef class MetadataPluginCallCredentials(CallCredentials): + + def __cinit__(self, metadata_plugin, name): +@@ -83,6 +132,7 @@ + c_metadata_plugin.state = <void *>self._metadata_plugin + c_metadata_plugin.type = self._name + cpython.Py_INCREF(self._metadata_plugin) ++ _maybe_register_shutdown_handler() + fork_handlers_and_grpc_init() + # TODO(yihuazhang): Expose min_security_level via the Python API so that + # applications can decide what minimum security level their plugins require. +--- a/src/python/grpcio/grpc/_cython/_cygrpc/records.pxd.pxi ++++ b/src/python/grpcio/grpc/_cython/_cygrpc/records.pxd.pxi +@@ -14,8 +14,8 @@ + + + cdef bytes _slice_bytes(grpc_slice slice) +-cdef grpc_slice _copy_slice(grpc_slice slice) nogil +-cdef grpc_slice _slice_from_bytes(bytes value) nogil ++cdef grpc_slice _copy_slice(grpc_slice slice) noexcept nogil ++cdef grpc_slice _slice_from_bytes(bytes value) noexcept nogil + + + cdef class CallDetails: +--- a/src/python/grpcio/grpc/_cython/_cygrpc/records.pyx.pxi ++++ b/src/python/grpcio/grpc/_cython/_cygrpc/records.pyx.pxi +@@ -18,12 +18,12 @@ + cdef size_t length = grpc_slice_length(slice) + return (<const char *>start)[:length] + +-cdef grpc_slice _copy_slice(grpc_slice slice) nogil: ++cdef grpc_slice _copy_slice(grpc_slice slice) noexcept nogil: + cdef void *start = grpc_slice_start_ptr(slice) + cdef size_t length = grpc_slice_length(slice) + return grpc_slice_from_copied_buffer(<const char *>start, length) + +-cdef grpc_slice _slice_from_bytes(bytes value) nogil: ++cdef grpc_slice _slice_from_bytes(bytes value) noexcept nogil: + cdef const char *value_ptr + cdef size_t length + with gil: +--- a/src/python/grpcio/grpc/_cython/_cygrpc/security.pxd.pxi ++++ b/src/python/grpcio/grpc/_cython/_cygrpc/security.pxd.pxi +@@ -14,4 +14,4 @@ + + + cdef grpc_ssl_roots_override_result ssl_roots_override_callback( +- char **pem_root_certs) nogil ++ char **pem_root_certs) noexcept nogil +--- a/src/python/grpcio/grpc/_cython/_cygrpc/security.pyx.pxi ++++ b/src/python/grpcio/grpc/_cython/_cygrpc/security.pyx.pxi +@@ -18,7 +18,7 @@ + + + cdef grpc_ssl_roots_override_result ssl_roots_override_callback( +- char **pem_root_certs) nogil: ++ char **pem_root_certs) noexcept nogil: + with gil: + pkg = __name__ + if pkg.endswith('.cygrpc'): +--- a/tools/distrib/python/grpcio_tools/grpc_tools/_protoc_compiler.pyx ++++ b/tools/distrib/python/grpcio_tools/grpc_tools/_protoc_compiler.pyx +@@ -39,12 +39,12 @@ + vector[string]* include_path, + vector[pair[string, string]]* files_out, + vector[cProtocError]* errors, +- vector[cProtocWarning]* wrnings) nogil except + ++ vector[cProtocWarning]* wrnings) except + nogil + int protoc_get_services(char* protobuf_path, + vector[string]* include_path, + vector[pair[string, string]]* files_out, + vector[cProtocError]* errors, +- vector[cProtocWarning]* wrnings) nogil except + ++ vector[cProtocWarning]* wrnings) except + nogil + + def run_main(list args not None): + cdef char **argv = <char **>stdlib.malloc(len(args)*sizeof(char *)) diff --git a/debian/patches/cython3-noexcept.patch b/debian/patches/cython3-noexcept.patch new file mode 100644 index 0000000000..5c7b34b888 --- /dev/null +++ b/debian/patches/cython3-noexcept.patch @@ -0,0 +1,170 @@ +From 4bea12353c2f0f5a52746821f1e3f961592c7ca0 Mon Sep 17 00:00:00 2001 +From: Atri Bhattacharya <a.bhattacha...@uliege.be> +Date: Tue, 2 Apr 2024 10:05:04 -0700 +Subject: [PATCH] [python] Cython 3 compatibility: declare functions noexcept. + (#35995) + +In Cython 3, cdef functions that really will not raise exceptions must be declared as `noexcept`. Fixed by this commit. + +Update requirements to `cython >= 3.0` in requirements*.txt and setup.py. + +Fixes issue #33918. + +<!-- + +If you know who should review your pull request, please assign it to that +person, otherwise the pull request would get assigned randomly. + +If your pull request is for a specific language, please add the appropriate +lang label. + +--> + +Closes #35995 + +COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/35995 from badshah400:master b3277bac1585ddee88a170b0a95c260d909cce9c +PiperOrigin-RevId: 621214091 +--- + requirements.bazel.txt | 2 +- + requirements.txt | 2 +- + setup.py | 2 +- + .../grpcio/grpc/_cython/_cygrpc/aio/callback_common.pxd.pxi | 2 +- + .../grpcio/grpc/_cython/_cygrpc/aio/callback_common.pyx.pxi | 2 +- + src/python/grpcio/grpc/_cython/_cygrpc/credentials.pyx.pxi | 2 +- + src/python/grpcio/grpc/_cython/_cygrpc/fork_posix.pxd.pxi | 6 +++--- + src/python/grpcio/grpc/_cython/_cygrpc/fork_posix.pyx.pxi | 6 +++--- + src/python/grpcio/grpc/_cython/_cygrpc/vtable.pyx.pxi | 6 +++--- + 9 files changed, 15 insertions(+), 15 deletions(-) + +Origin: upstream, https://github.com/grpc/grpc/commit/4bea12353c2f0f5a52746821f1e3f961592c7ca0 +Bug-upstream: https://github.com/grpc/grpc/issues/33918 +--- a/requirements.bazel.txt ++++ b/requirements.bazel.txt +@@ -1,6 +1,6 @@ + # GRPC Python setup requirements + coverage==4.5.4 +-cython==0.29.21 ++cython==3.0.0 + protobuf>=3.5.0.post1, < 4.0dev + wheel==0.38.1 + oauth2client==4.1.0 +--- a/requirements.txt ++++ b/requirements.txt +@@ -1,5 +1,5 @@ + # GRPC Python setup requirements + coverage>=4.0 +-cython>=0.29.8,<3.0.0rc1 ++cython>=3.0.0 + protobuf>=4.21.3,<5.0dev + wheel>=0.29 +--- a/setup.py ++++ b/setup.py +@@ -573,7 +573,7 @@ + sys.stderr.write( + "We could not find Cython. Setup may take 10-20 minutes.\n" + ) +- SETUP_REQUIRES += ("cython>=0.23,<3.0.0rc1",) ++ SETUP_REQUIRES += ('cython>=3.0.0',) + + COMMAND_CLASS = { + "doc": commands.SphinxDocumentation, +--- a/src/python/grpcio/grpc/_cython/_cygrpc/aio/callback_common.pxd.pxi ++++ b/src/python/grpcio/grpc/_cython/_cygrpc/aio/callback_common.pxd.pxi +@@ -48,7 +48,7 @@ + @staticmethod + cdef void functor_run( + grpc_completion_queue_functor* functor, +- int succeed) ++ int succeed) noexcept + + cdef grpc_completion_queue_functor *c_functor(self) + +--- a/src/python/grpcio/grpc/_cython/_cygrpc/aio/callback_common.pyx.pxi ++++ b/src/python/grpcio/grpc/_cython/_cygrpc/aio/callback_common.pyx.pxi +@@ -50,7 +50,7 @@ + @staticmethod + cdef void functor_run( + grpc_completion_queue_functor* functor, +- int success): ++ int success) noexcept: + cdef CallbackContext *context = <CallbackContext *>functor + cdef object waiter = <object>context.waiter + if not waiter.cancelled(): +--- a/src/python/grpcio/grpc/_cython/_cygrpc/credentials.pyx.pxi ++++ b/src/python/grpcio/grpc/_cython/_cygrpc/credentials.pyx.pxi +@@ -316,7 +316,7 @@ + return credentials + + cdef grpc_ssl_certificate_config_reload_status _server_cert_config_fetcher_wrapper( +- void* user_data, grpc_ssl_server_certificate_config **config) with gil: ++ void* user_data, grpc_ssl_server_certificate_config **config) noexcept with gil: + # This is a credentials.ServerCertificateConfig + cdef ServerCertificateConfig cert_config = None + if not user_data: +--- a/src/python/grpcio/grpc/_cython/_cygrpc/fork_posix.pxd.pxi ++++ b/src/python/grpcio/grpc/_cython/_cygrpc/fork_posix.pxd.pxi +@@ -12,10 +12,10 @@ + # See the License for the specific language governing permissions and + # limitations under the License. + +-cdef void __prefork() nogil ++cdef void __prefork() noexcept nogil + + +-cdef void __postfork_parent() nogil ++cdef void __postfork_parent() noexcept nogil + + +-cdef void __postfork_child() nogil +\ No newline at end of file ++cdef void __postfork_child() noexcept nogil +--- a/src/python/grpcio/grpc/_cython/_cygrpc/fork_posix.pyx.pxi ++++ b/src/python/grpcio/grpc/_cython/_cygrpc/fork_posix.pyx.pxi +@@ -35,7 +35,7 @@ + + _fork_handler_failed = False + +-cdef void __prefork() nogil: ++cdef void __prefork() noexcept nogil: + with gil: + global _fork_handler_failed + _fork_handler_failed = False +@@ -49,14 +49,14 @@ + _fork_handler_failed = True + + +-cdef void __postfork_parent() nogil: ++cdef void __postfork_parent() noexcept nogil: + with gil: + with _fork_state.fork_in_progress_condition: + _fork_state.fork_in_progress = False + _fork_state.fork_in_progress_condition.notify_all() + + +-cdef void __postfork_child() nogil: ++cdef void __postfork_child() noexcept nogil: + with gil: + try: + if _fork_handler_failed: +--- a/src/python/grpcio/grpc/_cython/_cygrpc/vtable.pyx.pxi ++++ b/src/python/grpcio/grpc/_cython/_cygrpc/vtable.pyx.pxi +@@ -13,16 +13,16 @@ + # limitations under the License. + + # TODO(https://github.com/grpc/grpc/issues/15662): Reform this. +-cdef void* _copy_pointer(void* pointer): ++cdef void* _copy_pointer(void* pointer) noexcept: + return pointer + + + # TODO(https://github.com/grpc/grpc/issues/15662): Reform this. +-cdef void _destroy_pointer(void* pointer): ++cdef void _destroy_pointer(void* pointer) noexcept: + pass + + +-cdef int _compare_pointer(void* first_pointer, void* second_pointer): ++cdef int _compare_pointer(void* first_pointer, void* second_pointer) noexcept: + if first_pointer < second_pointer: + return -1 + elif first_pointer > second_pointer: diff --git a/debian/patches/gcc-14-2.patch b/debian/patches/gcc-14-2.patch new file mode 100644 index 0000000000..e0a0d0a21e --- /dev/null +++ b/debian/patches/gcc-14-2.patch @@ -0,0 +1,28 @@ +From f55bf225da0eafeeebffd507dcb57c625933d105 Mon Sep 17 00:00:00 2001 +From: alto-ruby <altoru...@gmail.com> +Date: Tue, 17 Sep 2024 18:09:05 -0700 +Subject: [PATCH] [Ruby] fix incompatible pointer type of + grpc_compression_algorithm_name (#37741) + +fixes #37731 + +Closes #37741 + +COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/37741 from alto-ruby:fix-incompatible-pointer-type-in-grpc_compression_algorithm_name 18a89be44d54e8c8bd614f1cee626d1db598a45a +PiperOrigin-RevId: 675779382 +--- + src/ruby/ext/grpc/rb_compression_options.c | 2 +- + 1 file changed, 1 insertion(+), 1 deletion(-) + +Origin: upstream, https://github.com/grpc/grpc/pull/37741 +--- a/src/ruby/ext/grpc/rb_compression_options.c ++++ b/src/ruby/ext/grpc/rb_compression_options.c +@@ -296,7 +296,7 @@ + * Fails if the enum value is invalid. */ + VALUE grpc_rb_compression_options_algorithm_value_to_name_internal( + grpc_compression_algorithm internal_value) { +- char* algorithm_name = NULL; ++ const char* algorithm_name = NULL; + + if (!grpc_compression_algorithm_name(internal_value, &algorithm_name)) { + rb_raise(rb_eArgError, "Failed to convert algorithm value to name"); diff --git a/debian/patches/no-embed-libs.patch b/debian/patches/no-embed-libs.patch index cf3be6c548..ba361ed078 100644 --- a/debian/patches/no-embed-libs.patch +++ b/debian/patches/no-embed-libs.patch @@ -1,6 +1,6 @@ --- a/src/ruby/ext/grpc/extconf.rb +++ b/src/ruby/ext/grpc/extconf.rb -@@ -70,10 +70,12 @@ end +@@ -69,10 +69,12 @@ # Don't embed on TruffleRuby (constant-time crypto is unsafe with Sulong, slow build times) ENV['EMBED_OPENSSL'] = (RUBY_ENGINE != 'truffleruby').to_s diff --git a/debian/patches/no-opencensus.patch b/debian/patches/no-opencensus.patch index 9146a7fe86..02c1748d8f 100644 --- a/debian/patches/no-opencensus.patch +++ b/debian/patches/no-opencensus.patch @@ -6,9 +6,9 @@ Last-Update: 2022-11-14 --- ---- grpc-1.50.1.orig/CMakeLists.txt -+++ grpc-1.50.1/CMakeLists.txt -@@ -366,6 +366,7 @@ if (NOT EXISTS ${CMAKE_CURRENT_SOURCE_DI +--- a/CMakeLists.txt ++++ b/CMakeLists.txt +@@ -382,6 +382,7 @@ ) endif() # Setup external proto library at third_party/opencensus-proto/src with 2 download URLs @@ -16,7 +16,7 @@ Last-Update: 2022-11-14 if (NOT EXISTS ${CMAKE_CURRENT_SOURCE_DIR}/third_party/opencensus-proto/src) # Download the archive via HTTP, validate the checksum, and extract to third_party/opencensus-proto/src. download_archive( -@@ -384,6 +385,7 @@ if (NOT EXISTS ${CMAKE_CURRENT_SOURCE_DI +@@ -400,6 +401,7 @@ opencensus-proto-0.3.0/src ) endif() diff --git a/debian/patches/python3.13-pipes.patch b/debian/patches/python3.13-pipes.patch new file mode 100644 index 0000000000..3e361cf177 --- /dev/null +++ b/debian/patches/python3.13-pipes.patch @@ -0,0 +1,110 @@ +From dfa9d853fff02212ff232da5b39b76adf3ecce83 Mon Sep 17 00:00:00 2001 +From: Leonardo Pistone <lepist...@spotify.com> +Date: Thu, 7 Dec 2023 12:59:29 -0800 +Subject: [PATCH] Fix Python DeprecationWarning: 'pipes' (#34941) + +Starting from Python 3.11, the pipes module produces this warning: + +DeprecationWarning: 'pipes' is deprecated and slated for removal in Python 3.13 + +Turns out that in this repo the pipes module is only used for the +"quote" function which is turn directly taken from the shlex module [1]. + +The shlex module is not deprecated as of today and is already used in +other places in this repo. The function shlex.quote has been around +since the ancient Python 3.3. + +[1] https://github.com/python/cpython/blob/3.11/Lib/pipes.py#L64-L66 + +<!-- + +If you know who should review your pull request, please assign it to that +person, otherwise the pull request would get assigned randomly. + +If your pull request is for a specific language, please add the appropriate +lang label. + +--> + +Closes #34941 + +COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/34941 from lepistone:deprecate-python-pipes 233c54c135542178703aa700a2dddadc895fedb0 +PiperOrigin-RevId: 588883480 +--- + tools/run_tests/run_performance_tests.py | 10 +++++----- + tools/run_tests/run_tests.py | 4 ++-- + 2 files changed, 7 insertions(+), 7 deletions(-) + +Origin: upstream, https://github.com/grpc/grpc/pull/34941 +Bug-Debian: https://bugs.debian.org/1084603 +--- a/tools/run_tests/run_performance_tests.py ++++ b/tools/run_tests/run_performance_tests.py +@@ -21,8 +21,8 @@ + import itertools + import json + import os +-import pipes + import re ++import shlex + import sys + import time + +@@ -121,7 +121,7 @@ + if bq_result_table: + cmd += 'BQ_RESULT_TABLE="%s" ' % bq_result_table + cmd += "tools/run_tests/performance/run_qps_driver.sh " +- cmd += "--scenarios_json=%s " % pipes.quote( ++ cmd += "--scenarios_json=%s " % shlex.quote( + json.dumps({"scenarios": [scenario_json]}) + ) + cmd += "--scenario_result_file=scenario_result.json " +@@ -135,7 +135,7 @@ + user_at_host = "%s@%s" % (_REMOTE_HOST_USERNAME, remote_host) + cmd = 'ssh %s "cd ~/performance_workspace/grpc/ && "%s' % ( + user_at_host, +- pipes.quote(cmd), ++ shlex.quote(cmd), + ) + + return jobset.JobSpec( +@@ -157,7 +157,7 @@ + user_at_host = "%s@%s" % (_REMOTE_HOST_USERNAME, remote_host) + cmd = 'ssh %s "cd ~/performance_workspace/grpc/ && "%s' % ( + user_at_host, +- pipes.quote(cmd), ++ shlex.quote(cmd), + ) + + return jobset.JobSpec( +@@ -192,7 +192,7 @@ + user_at_host = "%s@%s" % (_REMOTE_HOST_USERNAME, client_host) + cmd = 'ssh %s "cd ~/performance_workspace/grpc/ && "%s' % ( + user_at_host, +- pipes.quote(cmd), ++ shlex.quote(cmd), + ) + + return jobset.JobSpec( +--- a/tools/run_tests/run_tests.py ++++ b/tools/run_tests/run_tests.py +@@ -26,10 +26,10 @@ + import multiprocessing + import os + import os.path +-import pipes + import platform + import random + import re ++import shlex + import socket + import subprocess + import sys +@@ -479,7 +479,7 @@ + cmdline = [binary] + target["args"] + shortname = target.get( + "shortname", +- " ".join(pipes.quote(arg) for arg in cmdline), ++ " ".join(shlex.quote(arg) for arg in cmdline), + ) + shortname += shortname_ext + out.append( diff --git a/debian/patches/python3.13-pyeval-initthreads.patch b/debian/patches/python3.13-pyeval-initthreads.patch new file mode 100644 index 0000000000..361cf46b99 --- /dev/null +++ b/debian/patches/python3.13-pyeval-initthreads.patch @@ -0,0 +1,39 @@ +From d583a79750a7f3cf2c955d880ebfbfd5b8aa1675 Mon Sep 17 00:00:00 2001 +From: "Benjamin A. Beasley" <c...@musicinmybrain.net> +Date: Thu, 2 Nov 2023 07:00:37 -0400 +Subject: [PATCH] [Python] Do not call PyEval_InitThreads +MIME-Version: 1.0 +Content-Type: text/plain; charset=UTF-8 +Content-Transfer-Encoding: 8bit + +Since Python 3.7, it’s called by Py_Initialize and doesn’t need to be called manually, so this removal has no effect. + +Since Python 3.9, it does nothing, and in Python 3.13, it will be removed, which makes this change necessary. + +References: + +https://docs.python.org/3/c-api/init.html#c.PyEval_InitThreads + +https://github.com/python/cpython/issues/105182 +--- + src/python/grpcio/grpc/_cython/cygrpc.pyx | 7 ------- + 1 file changed, 7 deletions(-) + +Origin: upstream, https://github.com/grpc/grpc/pull/34857 +--- a/src/python/grpcio/grpc/_cython/cygrpc.pyx ++++ b/src/python/grpcio/grpc/_cython/cygrpc.pyx +@@ -80,14 +80,7 @@ + # + # initialize gRPC + # +-cdef extern from "Python.h": +- +- int PyEval_InitThreads() +- + cdef _initialize(): +- # We have Python callbacks called by c-core threads, this ensures the GIL +- # is initialized. +- PyEval_InitThreads() + grpc_set_ssl_roots_override_callback( + <grpc_ssl_roots_override_callback>ssl_roots_override_callback) + diff --git a/debian/patches/series b/debian/patches/series index 0824a05bd9..5f8120e39d 100644 --- a/debian/patches/series +++ b/debian/patches/series @@ -15,3 +15,8 @@ add_grpc_libdir.patch link_python_all_absl.patch fix-run_poll_channels_loop-FTBFS.patch ruby_no-incompatible-pointer-types.patch +cython3-noexcept.patch +python3.13-pyeval-initthreads.patch +cython3-deadlocks.patch +python3.13-pipes.patch +gcc-14-2.patch