[issue43514] Disallow fork in a subinterpreter affects multiprocessing plugin

2021-03-16 Thread Frank Ueberschar


New submission from Frank Ueberschar :

Related to this issue https://bugs.python.org/issue34651, our Bareos libcloud 
plugin cannot be run with Python > 3.7. We are using subprocesses in a 
C-subinterpreter environment. 

Is there a way to circumvent rewriting our code completely?

--
components: C API
messages: 388841
nosy: franku
priority: normal
severity: normal
status: open
title: Disallow fork in a subinterpreter affects multiprocessing plugin
type: behavior
versions: Python 3.8

___
Python tracker 
<https://bugs.python.org/issue43514>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue43514] Disallow fork in a subinterpreter affects multiprocessing plugin

2021-03-16 Thread Frank Ueberschar


Frank Ueberschar  added the comment:

Here is part of the gdb backtrace:

[franku@franku py3plug-fd-libcloud]$ sbin/bareos_fd-py3plug-fd-libcloud -f -c 
etc/bareos   
Fatal Python error: _PyInterpreterState_DeleteExceptMain: not main interpreter
Python runtime state: initialized

Current thread 0x7f0dc2ab7640 (most recent call first):
  File "/usr/lib64/python3.9/multiprocessing/popen_fork.py", line 66 in _launch
  File "/usr/lib64/python3.9/multiprocessing/popen_fork.py", line 19 in __init__
  File "/usr/lib64/python3.9/multiprocessing/context.py", line 277 in _Popen
  File "/usr/lib64/python3.9/multiprocessing/context.py", line 224 in _Popen
  File "/usr/lib64/python3.9/multiprocessing/process.py", line 121 in start
  File 
"/home/franku/git/bareos/master/core/src/plugins/filed/python/libcloud/BareosLibcloudApi.py",
 line 102 in __init__
  File 
"/home/franku/git/bareos/master/core/src/plugins/filed/python/libcloud/BareosFdPluginLibcloud.py",
 line 267 in start_backup_job
  File 
"/home/franku/git/bareos/master/core/src/plugins/filed/python/pyfiles/BareosFdPluginBaseclass.py",
 line 285 in handle_plugin_event
  File 
"/home/franku/git/bareos/master/core/src/plugins/filed/python/pyfiles/BareosFdWrapper.py",
 line 38 in handle_plugin_event
Fatal Python error: _PyInterpreterState_DeleteExceptMain: not main interpreter
Python runtime state: initialized

--

___
Python tracker 
<https://bugs.python.org/issue43514>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue43514] Disallow fork in a subinterpreter affects multiprocessing plugin

2021-03-16 Thread Frank Ueberschar


Frank Ueberschar  added the comment:

Initialization of the Python interpreter in C-code is here: 
https://github.com/bareos/bareos/blob/fb76608092ba204ce43cd7c262619e01f9d6a2d6/core/src/plugins/filed/python/python-fd.cc#L189

The actual Python code that starts the child processes is here:
https://github.com/bareos/bareos/blob/fb76608092ba204ce43cd7c262619e01f9d6a2d6/core/src/plugins/filed/python/libcloud/BareosLibcloudApi.py#L110

Where this is one of the actual Processes: 
https://github.com/bareos/bareos/blob/fb76608092ba204ce43cd7c262619e01f9d6a2d6/core/src/plugins/filed/python/libcloud/bareos_libcloud_api/bucket_explorer.py#L45

Inherited from this base class: 
https://github.com/bareos/bareos/blob/fb76608092ba204ce43cd7c262619e01f9d6a2d6/core/src/plugins/filed/python/libcloud/bareos_libcloud_api/process_base.py#L36

--

___
Python tracker 
<https://bugs.python.org/issue43514>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue43514] Disallow fork in a subinterpreter affects multiprocessing plugin

2021-03-16 Thread Frank Ueberschar


Frank Ueberschar  added the comment:

These lines correspond (due to dirty working copy): 

File 
"/home/franku/git/bareos/master/core/src/plugins/filed/python/libcloud/BareosLibcloudApi.py",
 line 102 in __init__

The actual Python code that starts the child processes is here:
https://github.com/bareos/bareos/blob/fb76608092ba204ce43cd7c262619e01f9d6a2d6/core/src/plugins/filed/python/libcloud/BareosLibcloudApi.py#L110

--

___
Python tracker 
<https://bugs.python.org/issue43514>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com