[Python-Dev] Pure pickle bechmark.

[Bhavishya]

Hello,

1).I was going through the code of *python pickle* to search any
optimization possibility.But the only thing that I found very alarming was
again the import time(I tried with lazy-import but it didn't helped much.)

I found py3 to be ~45 times slower on* initial imports(very raw
measure..using "time." ) *as compared to py2 on an usual example.

py3->
./python -c '
favorite_color = { "lion": "yellow", "kitty": "red" }
pickle.dump( favorite_color, open( "save.p", "wb" ) )'
0.009715557098388672(time taken to do initial imports...measured using
*time.time()* )

py2->
./python -c '
favorite_color = { "lion": "yellow", "kitty": "red" }
pickle.dump( favorite_color, open( "save.p", "wb" ) )'
0.000236034393311(time taken to do initial imports...measured using
*time.time()* )

Do you have any thought/ideas on improving this?


Thank You.
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Pure pickle bechmark.

[MRAB]

On 2017-07-09 15:08, Bhavishya wrote:

Hello,

1).I was going through the code of *python pickle* to search any 
optimization possibility.But the only thing that I found very alarming 
was again the import time(I tried with lazy-import but it didn't helped 
much.)


I found py3 to be ~45 times slower on*initial imports(very raw 
measure..using "time." ) *as compared to py2 on an usual example.


py3->
./python -c '
favorite_color = { "lion": "yellow", "kitty": "red" }
pickle.dump( favorite_color, open( "save.p", "wb" ) )'
0.009715557098388672(time taken to do initial imports...measured using 
*time.time()* )


py2->
./python -c '
favorite_color = { "lion": "yellow", "kitty": "red" }
pickle.dump( favorite_color, open( "save.p", "wb" ) )'
0.000236034393311(time taken to do initial imports...measured using 
*time.time()* )


Do you have any thought/ideas on improving this?


Python 3 is using Unicode strings, whereas Python 2 is using bytestrings.
What you show above are very short (in time) examples (less than 1/100 
of a second), so they're not that meaningful.


If you had timed pickling a substantial object (the same object in both 
cases) and it took a significant amount of time and you found a 
significant slowdown, then it would be worth looking into further.

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Pure pickle bechmark.

[Antoine Pitrou]

Hi,

On Sun, 9 Jul 2017 19:38:09 +0530
Bhavishya  wrote:
> Hello,
> 
> 1).I was going through the code of *python pickle* to search any
> optimization possibility.But the only thing that I found very alarming was
> again the import time(I tried with lazy-import but it didn't helped much.)
> 
> I found py3 to be ~45 times slower on* initial imports(very raw
> measure..using "time." ) *as compared to py2 on an usual example.

Can you explain how you measured exactly?

Regards

Antoine.


___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Pure pickle bechmark.

[INADA Naoki]

I don't know this is relating to your case.

When I saw Victor's report [1], I researched why Python 3 is slower than
Python 2 on unpickle_pure_python benchmark.

[1] https://mail.python.org/pipermail/speed/2017-February/000503.html


And I found Python 2 and 3 uses different version of pickle format.

Current Python 3 uses "framing" format.  While unpickling, `read(1)` is
very performance critical.  Python 2 uses `cStringIO.read` which is
implemented in C.
On the other hand, Python 3 uses `_Unframer.read` which is implemented
in Python.

Since this is not relating to "first import time", I don't know this
is what you want to optimize.
(Since _pickle is used for normal case, pure Python unpickle
performance is not a common
problem).

If you want to optimize it, _Unframer uses BytesIO internally and
performance critical
part may be able to call BytesIO.read directly instead of _Unframer.read.

Regards,
INADA Naoki  


On Sun, Jul 9, 2017 at 11:08 PM, Bhavishya  wrote:
> Hello,
>
> 1).I was going through the code of python pickle to search any optimization
> possibility.But the only thing that I found very alarming was again the
> import time(I tried with lazy-import but it didn't helped much.)
>
> I found py3 to be ~45 times slower on initial imports(very raw
> measure..using "time." ) as compared to py2 on an usual example.
>
> py3->
> ./python -c '
> favorite_color = { "lion": "yellow", "kitty": "red" }
> pickle.dump( favorite_color, open( "save.p", "wb" ) )'
> 0.009715557098388672(time taken to do initial imports...measured using
> time.time() )
>
> py2->
> ./python -c '
> favorite_color = { "lion": "yellow", "kitty": "red" }
> pickle.dump( favorite_color, open( "save.p", "wb" ) )'
> 0.000236034393311(time taken to do initial imports...measured using
> time.time() )
>
> Do you have any thought/ideas on improving this?
>
>
> Thank You.
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Pure pickle bechmark.

[Victor Stinner]

Wait. Are we talking about the C accelerator or the pure Python
implementation of pickle on Python 3?

Victor

Le 10 juil. 2017 01:19, "INADA Naoki"  a écrit :

> I don't know this is relating to your case.
>
> When I saw Victor's report [1], I researched why Python 3 is slower than
> Python 2 on unpickle_pure_python benchmark.
>
> [1] https://mail.python.org/pipermail/speed/2017-February/000503.html
>
>
> And I found Python 2 and 3 uses different version of pickle format.
>
> Current Python 3 uses "framing" format.  While unpickling, `read(1)` is
> very performance critical.  Python 2 uses `cStringIO.read` which is
> implemented in C.
> On the other hand, Python 3 uses `_Unframer.read` which is implemented
> in Python.
>
> Since this is not relating to "first import time", I don't know this
> is what you want to optimize.
> (Since _pickle is used for normal case, pure Python unpickle
> performance is not a common
> problem).
>
> If you want to optimize it, _Unframer uses BytesIO internally and
> performance critical
> part may be able to call BytesIO.read directly instead of _Unframer.read.
>
> Regards,
> INADA Naoki  
>
>
> On Sun, Jul 9, 2017 at 11:08 PM, Bhavishya 
> wrote:
> > Hello,
> >
> > 1).I was going through the code of python pickle to search any
> optimization
> > possibility.But the only thing that I found very alarming was again the
> > import time(I tried with lazy-import but it didn't helped much.)
> >
> > I found py3 to be ~45 times slower on initial imports(very raw
> > measure..using "time." ) as compared to py2 on an usual example.
> >
> > py3->
> > ./python -c '
> > favorite_color = { "lion": "yellow", "kitty": "red" }
> > pickle.dump( favorite_color, open( "save.p", "wb" ) )'
> > 0.009715557098388672(time taken to do initial imports...measured using
> > time.time() )
> >
> > py2->
> > ./python -c '
> > favorite_color = { "lion": "yellow", "kitty": "red" }
> > pickle.dump( favorite_color, open( "save.p", "wb" ) )'
> > 0.000236034393311(time taken to do initial imports...measured using
> > time.time() )
> >
> > Do you have any thought/ideas on improving this?
> >
> >
> > Thank You.
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: https://mail.python.org/mailman/options/python-dev/
> victor.stinner%40gmail.com
>
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Pure pickle bechmark.

[INADA Naoki]

I said about pure Python implementation (unpickle_pure_python),
because mail title is "Pure pickle bechmark".
INADA Naoki  


On Mon, Jul 10, 2017 at 8:36 AM, Victor Stinner
 wrote:
> Wait. Are we talking about the C accelerator or the pure Python
> implementation of pickle on Python 3?
>
> Victor
>
> Le 10 juil. 2017 01:19, "INADA Naoki"  a écrit :
>>
>> I don't know this is relating to your case.
>>
>> When I saw Victor's report [1], I researched why Python 3 is slower than
>> Python 2 on unpickle_pure_python benchmark.
>>
>> [1] https://mail.python.org/pipermail/speed/2017-February/000503.html
>>
>>
>> And I found Python 2 and 3 uses different version of pickle format.
>>
>> Current Python 3 uses "framing" format.  While unpickling, `read(1)` is
>> very performance critical.  Python 2 uses `cStringIO.read` which is
>> implemented in C.
>> On the other hand, Python 3 uses `_Unframer.read` which is implemented
>> in Python.
>>
>> Since this is not relating to "first import time", I don't know this
>> is what you want to optimize.
>> (Since _pickle is used for normal case, pure Python unpickle
>> performance is not a common
>> problem).
>>
>> If you want to optimize it, _Unframer uses BytesIO internally and
>> performance critical
>> part may be able to call BytesIO.read directly instead of _Unframer.read.
>>
>> Regards,
>> INADA Naoki  
>>
>>
>> On Sun, Jul 9, 2017 at 11:08 PM, Bhavishya 
>> wrote:
>> > Hello,
>> >
>> > 1).I was going through the code of python pickle to search any
>> > optimization
>> > possibility.But the only thing that I found very alarming was again the
>> > import time(I tried with lazy-import but it didn't helped much.)
>> >
>> > I found py3 to be ~45 times slower on initial imports(very raw
>> > measure..using "time." ) as compared to py2 on an usual example.
>> >
>> > py3->
>> > ./python -c '
>> > favorite_color = { "lion": "yellow", "kitty": "red" }
>> > pickle.dump( favorite_color, open( "save.p", "wb" ) )'
>> > 0.009715557098388672(time taken to do initial imports...measured using
>> > time.time() )
>> >
>> > py2->
>> > ./python -c '
>> > favorite_color = { "lion": "yellow", "kitty": "red" }
>> > pickle.dump( favorite_color, open( "save.p", "wb" ) )'
>> > 0.000236034393311(time taken to do initial imports...measured using
>> > time.time() )
>> >
>> > Do you have any thought/ideas on improving this?
>> >
>> >
>> > Thank You.
>> ___
>> Python-Dev mailing list
>> Python-Dev@python.org
>> https://mail.python.org/mailman/listinfo/python-dev
>> Unsubscribe:
>> https://mail.python.org/mailman/options/python-dev/victor.stinner%40gmail.com
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Pure pickle bechmark.

[Victor Stinner]

Please explain how to reproduce your benchmark. Maybe write a shell script?

Victor

Le 9 juil. 2017 17:49, "Bhavishya"  a écrit :

> Hello,
>
> 1).I was going through the code of *python pickle* to search any
> optimization possibility.But the only thing that I found very alarming was
> again the import time(I tried with lazy-import but it didn't helped much.)
>
> I found py3 to be ~45 times slower on* initial imports(very raw
> measure..using "time." ) *as compared to py2 on an usual example.
>
> py3->
> ./python -c '
> favorite_color = { "lion": "yellow", "kitty": "red" }
> pickle.dump( favorite_color, open( "save.p", "wb" ) )'
> 0.009715557098388672(time taken to do initial imports...measured using
> *time.time()* )
>
> py2->
> ./python -c '
> favorite_color = { "lion": "yellow", "kitty": "red" }
> pickle.dump( favorite_color, open( "save.p", "wb" ) )'
> 0.000236034393311(time taken to do initial imports...measured using
> *time.time()* )
>
> Do you have any thought/ideas on improving this?
>
>
> Thank You.
>
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: https://mail.python.org/mailman/options/python-dev/
> victor.stinner%40gmail.com
>
>
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Pure pickle bechmark.

[Victor Stinner]

(Oops, I didn't notice that we started to talk off the list, let's
discuss that in python-dev please.)

I don't see the point of optimizing "pickle/unpickle pure python"
benchmark on Python 3. This benchmark doesn't make sense on Python 3,
since I don't know anyone using the pure Python pickle. The C
accelerator is now used by default.

I already proposed to remove this benchmark:
https://mail.python.org/pipermail/speed/2017-April/000554.html

*but* Antoine Pitrou mentionned that the cloudpickle project uses it.

Maybe we should try to understand what's wrong with _pickle (C module)
for cloudpickle?

Victor

2017-07-10 2:10 GMT+02:00 Bhavishya :
> I was working on  the two regressed benchmarks (i.e.  pickle/unpickle
> pure-python), and as it was a case with other benchmarksthat performance
> is affected  by import ...I thought that could  be a case with pickle.py
> too. And thus tried adding the above patch to Lib/pickle.py to measure the
> initial import time.
>
> I haven't tried it for any practical use-case.
>
>
> On Mon, Jul 10, 2017 at 5:27 AM, Victor Stinner 
> wrote:
>>
>> Sorry, I don't understand the direct link between the import time of 4
>> modules and the pickle module. Can you please elaborate?
>>
>> What are you trying to optimize?
>>
>> What is your use case?
>>
>> Victor
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com