[issue37292] _xxsubinterpreters: Can't unpickle objects defined in __main__

[Crusader Ky]

New submission from Crusader Ky :

As of CPython 3.8.0b1:

If one pickles an object that is defined in the __main__ module, sends it to a 
subinterpreter as bytes, and then tries unpickling it there, it fails saying 
that __main__ doesn't define it.


import _xxsubinterpreters as interpreters
import pickle


class C:
pass


c = C()

interp_id = interpreters.create()
c_bytes = pickle.dumps(c)
interpreters.run_string(
interp_id,
"import pickle; pickle.loads(c_bytes)",
shared={"c_bytes": c_bytes},
)


If the above is executed directly with the python command-line, it fails. If 
it's imported from another module, it works.
One would expected behaviour compatible with sub-processes spawned with the 
spawn method, where the__main__ of the parent process is visible to the 
subprocess too.

Workarounds:
1 - define everything that must be pickled in an imported module
2 - use CloudPickle

--
messages: 345680
nosy: Crusader Ky, eric.snow
priority: normal
severity: normal
status: open
title: _xxsubinterpreters: Can't unpickle objects defined in __main__
type: behavior
versions: Python 3.9

___
Python tracker 
<https://bugs.python.org/issue37292>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37292] _xxsubinterpreters: Can't unpickle objects defined in __main__

[Crusader Ky]

Change by Crusader Ky :


--
versions: +Python 3.8

___
Python tracker 
<https://bugs.python.org/issue37292>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37293] concurrent.futures.InterpreterPoolExecutor

[Crusader Ky]

New submission from Crusader Ky :

As one of the logical consequences to PEP 554, it would be neat to have a 
concurrent.futures.InterpreterPoolExecutor.

I wrote the initial code at https://github.com/crusaderky/subinterpreters_tests 
- currently missing unit tests and pickle5 buffers support.

If everybody is happy with the design, I'll start working on a PR as soon as 
the GIL becomes per-interpreter (it's kinda pointless before that).

--
components: Extension Modules
messages: 345681
nosy: Crusader Ky, eric.snow
priority: normal
severity: normal
status: open
title: concurrent.futures.InterpreterPoolExecutor
type: enhancement
versions: Python 3.9

___
Python tracker 
<https://bugs.python.org/issue37293>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue22239] asyncio: nested event loop

[Crusader Ky]

Change by Crusader Ky :


--
nosy: +Crusader Ky

___
Python tracker 
<https://bugs.python.org/issue22239>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36555] PEP484 @overload vs. str/bytes

[Crusader Ky]

New submission from Crusader Ky :

An exceedingly common pattern in many Python libraries is for a function to 
accept either a string or a list of strings, and change the function output 
accordingly.

This however does not play nice with @typing.overload, as a str variable is 
also an Iterable[str] that yields individual characters; a bytes variable is 
also an Iterable[bytes].

The example below confuses tools like mypy:

@overload
def f(x: str) -> int
   ...

@overload
def f(x: Iterable[str]) -> List[int]
   ...

def f(x):
if isinstance(x, str):
return len(x)
return [len(i) for i in x]


mypy output:

error: Overloaded function signatures 1 and 2 overlap with incompatible return 
types


The proposed solution is to modify PEP484 to specify that, in case of 
ambiguity, whatever overloaded typing is defined first wins. This would be 
coherent with the behaviour of @functools.singledispatch.

--
components: Library (Lib)
messages: 339610
nosy: Crusader Ky
priority: normal
severity: normal
status: open
title: PEP484 @overload vs. str/bytes
type: behavior
versions: Python 3.5, Python 3.6, Python 3.7

___
Python tracker 
<https://bugs.python.org/issue36555>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com