[issue28053] parameterize what serialization is used in multiprocessing

2017-07-14 Thread Will S

Will S added the comment:

Documentation would be appreciated. I have a project that uses BaseManager, 
Client, and Listener to create some servers and clients. I would like to update 
the project to work with Python 3 and would prefer to update the clients and 
the servers separately (i.e. switch the client to Python 3 while the server is 
run with Python 2.7). However, BaseManager uses connection.Client which uses 
connection._ConnectionBase which uses reduction.ForkingPickler without a 
protocol argument. It seems the default protocol is 3 on Python 3.6 and 2 on 
Python 2.7 (contrary to the comment above about v2 being used). I just want to 
set the protocol version to 2 in Python 3.6. Can I do that with the changes 
added by this patch?

I tried creating pickle2reducer.py like this:

from multiprocessing.reduction import ForkingPickler, AbstractReducer

class ForkingPickler2(ForkingPickler):
def __init__(self, *args):
if len(args) > 1:
args[1] = 2
else:
args.append(2)
super().__init__(*args)

@classmethod
def dumps(cls, obj, protocol=2):
return ForkingPickler.dumps(obj, protocol)


def dump(obj, file, protocol=2):
ForkingPickler2(file, protocol).dump(obj)


class Pickle2Reducer(AbstractReducer):
ForkingPickler = ForkingPickler2
register = ForkingPickler2.register
dump = dump

and then putting

import pickle2reducer
multiprocessing.reducer = pickle2reducer.Pickle2Reducer()

at the top of my module before

import multiprocessing.connection

but I still see "ValueError: unsupported pickle protocol: 3" on the server when 
I connect with a Python 3.6 client.

--
nosy: +Will S

___
Python tracker 
<http://bugs.python.org/issue28053>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue28053] parameterize what serialization is used in multiprocessing

2017-07-28 Thread Will S

Will S added the comment:

Just to follow up in case anyone comes across my last message later:

I just had to change the last line from

multiprocessing.reducer = pickle2reducer.Pickle2Reducer()

to

multiprocessing.context._default_context.reducer = 
pickle2reducer.Pickle2Reducer()

--

___
Python tracker 
<http://bugs.python.org/issue28053>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com