Hi All,
As part of our scikit-learn development and our effort to provide better
parallelism for python, we rely heavily on dynamic classes and functions
pickling. For this
usage we use cloudpickle, but it suffers from performance issues due to
its pure python implementation. After long discuss
Hi All,
As part of our scikit-learn development and our effort to provide better
parallelism for python, we rely heavily on dynamic classes and functions
pickling. For this
usage we use cloudpickle, but it suffers from performance issues due to
its pure python implementation. After long discuss
Hi Serhiy,
Do you have any bug / issue to track the work you want to do to add native
pickling support for locally defined function and classes by serializing
the code objects like cloudpickle does?
Is this work public on some git branch on GitHub or somewhere else?
Cheers,
--
Olivier
__
28.03.18 23:19, Antoine Pitrou пише:
Agreed. Do you know by which timeframe you'll know which opcodes you
want to add?
I'm currently in the middle of the first part, trying to implement
pickling local classes with static and class methods without creating
loops. Other parts exist just like g
On 29 March 2018 at 04:39, Antoine Pitrou wrote:
>
> Hi,
>
> I'd like to submit this PEP for discussion. It is quite specialized
> and the main target audience of the proposed changes is
> users and authors of applications/libraries transferring large amounts
> of data (read: the scientific compu
On Thu, 29 Mar 2018 11:25:13 +
Nathaniel Smith wrote:
>
> > I doubt it'll be a problem to pickle though as it'll use some form of
> > versioning even in NONPORTABLE mode right?
> >
>
> I guess the (merged, but undocumented?) changes in
> https://bugs.python.org/issue28053 should make it po
On Thu, Mar 29, 2018, 02:02 Chris Angelico wrote:
> On Thu, Mar 29, 2018 at 7:56 PM, Paul Moore wrote:
> > On 29 March 2018 at 09:49, Chris Angelico wrote:
> >> On Thu, Mar 29, 2018 at 7:18 PM, Nathaniel Smith wrote:
> >>> Another example is the multiprocessing module: it's very safe to
> >>>
On Thu, Mar 29, 2018 at 7:56 PM, Paul Moore wrote:
> On 29 March 2018 at 09:49, Chris Angelico wrote:
>> On Thu, Mar 29, 2018 at 7:18 PM, Nathaniel Smith wrote:
>>> Another example is the multiprocessing module: it's very safe to
>>> assume that the parent and the child are using the same interp
On 29 March 2018 at 09:49, Chris Angelico wrote:
> On Thu, Mar 29, 2018 at 7:18 PM, Nathaniel Smith wrote:
>> Another example is the multiprocessing module: it's very safe to
>> assume that the parent and the child are using the same interpreter
>> :-). There's no fundamental reason you shouldn't
On Thu, Mar 29, 2018 at 7:18 PM, Nathaniel Smith wrote:
> Another example is the multiprocessing module: it's very safe to
> assume that the parent and the child are using the same interpreter
> :-). There's no fundamental reason you shouldn't be able to send
> bytecode between them.
You put a sm
On Thu, 29 Mar 2018 01:40:17 +
Robert Collins wrote:
> >
> > Data sharing
> >
> >
> > If you pickle and then unpickle an object in the same process, passing
> > out-of-band buffer views, then the unpickled object may be backed by the
> > same buffer as the original pickled object.
On Thu, Mar 29, 2018 at 12:56 AM, Chris Jerdonek
wrote:
> On Wed, Mar 28, 2018 at 6:15 PM, Nathaniel Smith wrote:
>> On Wed, Mar 28, 2018 at 1:03 PM, Serhiy Storchaka
>> wrote:
>>> 28.03.18 21:39, Antoine Pitrou пише:
I'd like to submit this PEP for discussion. It is quite specialized
>>>
On Wed, Mar 28, 2018 at 6:15 PM, Nathaniel Smith wrote:
> On Wed, Mar 28, 2018 at 1:03 PM, Serhiy Storchaka wrote:
>> 28.03.18 21:39, Antoine Pitrou пише:
>>> I'd like to submit this PEP for discussion. It is quite specialized
>>> and the main target audience of the proposed changes is
>>> users
On 3/28/2018 9:15 PM, Nathaniel Smith wrote:
There's obviously some tension here between pickle's use as a
persistent storage format, and its use as a transient wire format. For
the former, you definitely can't store code objects because there's no
forwards- or backwards-compatibility guarantee
One question..
On Thu., 29 Mar. 2018, 07:42 Antoine Pitrou, wrote:
> ...
>
===
>
> Mutability
> --
>
> PEP 3118 buffers [#pep-3118]_ can be readonly or writable. Some objects,
> such as Numpy arrays, need to be backed by a mutable buffer for full
> operation. Pickle consumers that
On Wed, Mar 28, 2018 at 1:03 PM, Serhiy Storchaka wrote:
> 28.03.18 21:39, Antoine Pitrou пише:
>> I'd like to submit this PEP for discussion. It is quite specialized
>> and the main target audience of the proposed changes is
>> users and authors of applications/libraries transferring large amoun
On Wed, 28 Mar 2018 23:03:08 +0300
Serhiy Storchaka wrote:
> 28.03.18 21:39, Antoine Pitrou пише:
> > I'd like to submit this PEP for discussion. It is quite specialized
> > and the main target audience of the proposed changes is
> > users and authors of applications/libraries transferring lar
28.03.18 21:39, Antoine Pitrou пише:
> I'd like to submit this PEP for discussion. It is quite specialized
> and the main target audience of the proposed changes is
> users and authors of applications/libraries transferring large amounts
> of data (read: the scientific computing & data science ec
Hi,
I'd like to submit this PEP for discussion. It is quite specialized
and the main target audience of the proposed changes is
users and authors of applications/libraries transferring large amounts
of data (read: the scientific computing & data science ecosystems).
https://www.python.org/dev/p
19 matches
Mail list logo
