Alexandre Vassalotti added the comment:
Here's an updated patch. The new patch also cleans up tp_clear,
tp_traverse and tp_dealloc of io.BytesIO which used weakreflist incorrectly.
I also added support for test_memoryio for adding reference leak
regression tests. As you'll see, the
Alexandre Vassalotti added the comment:
> Why do you need all this? Isn't it enough to take a weakref and check
> the callback is triggered?
No, because you would need a weak reference to the instance's __dict__,
which is unavailable for io.StringIO.
Anyway, here's a sim
Alexandre Vassalotti added the comment:
I think the build configuration for _io on Windows is missing.
Modules/Setup.dist probably needs to be updated too.
--
___
Python tracker
<http://bugs.python.org/issue6
Alexandre Vassalotti added the comment:
> Not sure, I don't think any of these modules are built-in.
Ah true. Although there are built-in in py3k, you are right that it
doesn't make sense to make the trunk versions built-in.
--
___
P
Alexandre Vassalotti added the comment:
Jean Brouwers wrote:
> All tests passed after regenerating the expected results
The tests for profile are output test. If you regenerate them, they pass
for sure.
Merging cProfile/profile will require a lot more work than just renaming
the module.
Changes by Alexandre Vassalotti :
--
resolution: -> duplicate
status: open -> closed
superseder: -> GCC detection for runtime_library_dirs when ccache is used
___
Python tracker
<http://bugs.python.o
Alexandre Vassalotti added the comment:
Honestly, I don't think it is a big issue. MemoryErrors are rare and
typically cause the interpreter to shutdown.
By the way, do you think the static PyExc_RecursionErrorInst object is
affected by this bug?
--
nosy: +alexandre.vassa
Alexandre Vassalotti added the comment:
These tests are leaking on my build of r73393:
test_docxmlrpc leaked [0, 0, 85, 0, 0, -85, 0, 0, 0, 0] references, sum=0
test_popen leaked [0, 23, -23, 0, 23, 0, -23, 23, -23, 23] references,
sum=23
test_urllib leaked [4, 4, 4, 4, 6, 0, 0, 0, 2, 0
Alexandre Vassalotti added the comment:
Could simply merge the changes in r71185 to the PC/ and PCbuild/
directories?
--
___
Python tracker
<http://bugs.python.org/issue6
Alexandre Vassalotti added the comment:
Patch backported in r73399. Thanks!
--
resolution: -> fixed
status: open -> closed
___
Python tracker
<http://bugs.python.org/
Alexandre Vassalotti added the comment:
Could you provide a test case? The behaviour you are describing sounds
like a bug in cPickle.
--
nosy: +alexandre.vassalotti
___
Python tracker
<http://bugs.python.org/issue6
Alexandre Vassalotti added the comment:
Thanks for the test case. I will take a look.
--
assignee: georg.brandl -> alexandre.vassalotti
components: +Library (Lib) -Documentation, Extension Modules, Windows
___
Python tracker
<http://bugs.pyth
Alexandre Vassalotti added the comment:
Could you give me the output of this?
import cPickle
print repr(cPickle.dumps([float('+inf'), float('-inf'), float('nan')]))
print [float('+inf'), float('-inf'), float('nan')]
By the way,
Alexandre Vassalotti added the comment:
Here's a patch that implements the solution I described in msg85298.
Please give it a good review:
http://codereview.appspot.com/87072
--
Added file: http://bugs.python.org/file14369/portable_array_pickling
Alexandre Vassalotti added the comment:
Yesterday, I ported the _lsprof module to Python as an experiment. I
still have a few issues to iron out, but the core of the functionality
is there.
I am not sure yet how this experiment will fit in the profile/cProfile
merge. I wrote the port mainly
Changes by Alexandre Vassalotti :
Added file: http://bugs.python.org/file14377/pyprofile.py
___
Python tracker
<http://bugs.python.org/issue2919>
___
___
Python-bug
Alexandre Vassalotti added the comment:
That seems easy to fix by adding a dummy __reduce__ method to file. My
only worry is this could break file subclasses which may have ad-hoc
mechanisms implemented for pickling files.
--
keywords: +patch
nosy: +alexandre.vassalotti
stage
New submission from Alexandre Vassalotti :
The following patch tries to improve how xmlrpc.client handles strings.
In particular, it simplifies the decoding of strings by keeping them as
unicode str.
--
files: simplify_xmlrpc_string_decoding.diff
keywords: patch
messages: 89801
nosy
Alexandre Vassalotti added the comment:
I am closing this issue in favour of #3119, since Aaron's patch is
cleaner and slightly faster.
Thank you Daniel for the idea!
--
dependencies: -pickle.py is limited by python's call stack
resolution: -> duplicate
status: o
New submission from Alexandre Vassalotti :
Here's an example of the behaviour:
import io
def test(buf):
textio = io.TextIOWrapper(buf)
buf = io.BytesIO()
test(buf)
print(buf.closed) # This prints True currently
The problem here is TextIOWrapper closes its buffer when de
Alexandre Vassalotti added the comment:
I didn't test the changes extensively. I ran the test suite and the
changes seemed to be correct.
It is a bit difficult to provide a test case, since the patch shouldn't
change how the code currently behave. Nevertheless, here is a simp
Alexandre Vassalotti added the comment:
Oops, I forgot to update my client in my last message. The sample trace
run should make more sense now.
#!/usr/bin/python3.2
# xmlclient.py
import xmlrpc.client
server_proxy = xmlrpc.client.ServerProxy("http://localhost:8000&quo
Changes by Alexandre Vassalotti :
--
status: pending -> closed
___
Python tracker
<http://bugs.python.org/issue1761028>
___
___
Python-bugs-list mailing list
Un
Alexandre Vassalotti added the comment:
I am leaving this issue open for now.
I reconsidered whether we should add pickle support for methods and I
now think it would probably be a good idea. For example, the
multiprocessing module would benefit from a having built-in support for
method
Alexandre Vassalotti added the comment:
This has been fixed in 2.6 and 3.x.
Closing.
--
resolution: -> fixed
status: open -> closed
___
Python tracker
<http://bugs.python.org/
Changes by Alexandre Vassalotti :
--
___
Python tracker
<http://bugs.python.org/issue558238>
___
___
Python-bugs-list mailing list
Unsubscribe:
http://mail.pyth
Changes by Alexandre Vassalotti :
--
nosy: +alexandre.vassalotti
___
Python tracker
<http://bugs.python.org/issue6395>
___
___
Python-bugs-list mailing list
Unsub
Alexandre Vassalotti added the comment:
Could you elaborate on what is problem exactly? Please include any error
message you are getting. Also, what operating system are you using?
--
nosy: +alexandre.vassalotti
stage: -> test needed
___
Pyt
Alexandre Vassalotti added the comment:
I am against adding __add__ to dict, since merging dictionaries is not a
commutative operation.
If a short syntax is desired for merging dictionaries, the just define a
function. For example:
def merge_dicts(*args):
result = {}
for x in args
Alexandre Vassalotti added the comment:
How about checking the preconditions before calling asctime()? If the check
fails, then we can raise an exception without crashing.
--
___
Python tracker
<http://bugs.python.org/issue6
Alexandre Vassalotti added the comment:
Good idea!
You can remove the doctest behavior. I don't think it is useful. But if you
remove it, make sure you add an usage message when no argument is given.
--
___
Python tracker
<http://bugs.py
Alexandre Vassalotti added the comment:
The code for import_module_implementations seems a bit fragile. Prefixing the
module name with an underscore might not always yield the name of the optimized
implementation. An explicit dictionary to map the Python and C implementations
may be a
Alexandre Vassalotti added the comment:
It's a documentation enhancement request. The updated documentation in Python 3
for pickle added most of the requested details.
--
resolution: -> fixed
status: open -> closed
___
Python tra
Alexandre Vassalotti added the comment:
The new documentation for pickle in Python 3 fixes this.
I still believe we should outline the differences between pickle.py and
cPickle. But at this point, it's unlikely I will put the time to do it. Feel
free to open a new issue if you want th
Alexandre Vassalotti added the comment:
Thank you for the nice investigative work!
I will try my best to review this patch by next week.
--
___
Python tracker
<http://bugs.python.org/issue5
Alexandre Vassalotti added the comment:
> Do I understand correctly that the issue is that python
> Pickler class has dispatch attribute but C Pickler does
> not?
Yes.
> The add_dispatch_check-0.patch patch does not seem
> to add class attribute, it adds an instance attribut
Alexandre Vassalotti added the comment:
One easy fix for this would be to make InteractiveConsole use the string
"__main__" instead of "__console__". But other than that, I don't think we can
fix this within pickle.
--
Alexandre Vassalotti added the comment:
> This begs a question, however: why not use regular python bytecode in pickles?
Unlike pickle protocols, the bytecode is not required to be compatible across
Python versions. Furthermore, Python bytecode is designed has a general purpose
obj
Alexandre Vassalotti added the comment:
> It works for pickle/_pickle and heapq/_heapq, but won't work for io/_io/_pyio.
You can make the dictionary values as lists for the 'blocked' argument for
import_fresh_module(). That would work.
And, can you add
Alexandre Vassalotti added the comment:
Now, that 2.7 is out we won't able to commit this anymore. It is sad to abandon
a good patch like this.
--
resolution: -> wont fix
stage: patch review -> committed/rejected
status: open -> closed
Alexandre Vassalotti added the comment:
Closing this since Python 2.7 is out now.
--
nosy: +alexandre.vassalotti
resolution: -> wont fix
stage: needs patch -> committed/rejected
status: open -> closed
___
Python tracker
<http://bug
Alexandre Vassalotti added the comment:
Actually, you are right. This could be added as bug fix.
--
resolution: wont fix ->
stage: committed/rejected -> needs patch
status: closed -> open
___
Python tracker
<http://bugs.python.o
Alexandre Vassalotti added the comment:
I have fixed some style nits in your patch. It would be nice to have tests for
the different control paths in instantiate(). But, I realize that would be a
bit annoying.
Apart from that, the patch looks good.
--
Added file: http
Alexandre Vassalotti added the comment:
LGTM
--
___
Python tracker
<http://bugs.python.org/issue9268>
___
___
Python-bugs-list mailing list
Unsubscribe:
Alexandre Vassalotti added the comment:
The args argument is always a tuple created with Pdata_poptuple(). You can add
an explicit type check. If this check fails a RuntimeError should be raised,
because this would indicate a programming error in pickle
Alexandre Vassalotti added the comment:
Victor, have you tried using peek() instead of seek()? I mentioned this
previously in msg85780.
--
___
Python tracker
<http://bugs.python.org/issue3
Alexandre Vassalotti added the comment:
The patch look great. Can you update your patch to make it work against the
version of 2to3 in the sandbox (http://svn.python.org/sandbox/trunk/2to3)?
--
___
Python tracker
<http://bugs.python.org/issue5
New submission from Alexandre Vassalotti :
This is a big patch. Please review at http://codereview.appspot.com/1694050/show
This patch adds the most interesting optimizations from Unladen Swallow to
Python 3's pickle.
The core of the patch already been reviewed by Antoine and me
Alexandre Vassalotti added the comment:
The security issue mentioned previously has been known for years. And, it is
easy to protect against. See
http://docs.python.org/py3k/library/pickle.html#restricting-globals
Also I am against adding pickling support to code objects. Code objects have
Alexandre Vassalotti added the comment:
Patch committed in r83740. I will make the documentation update in a separate
commit.
Thanks!
--
resolution: -> accepted
stage: patch review -> committed/rejected
status: open -> closed
versions: -Python 2.6, Python 2.7, Python 3.
Alexandre Vassalotti added the comment:
Documentation added in r83741.
--
___
Python tracker
<http://bugs.python.org/issue5077>
___
___
Python-bugs-list mailin
Alexandre Vassalotti added the comment:
Although it is tempting to fix this, it feels to much like a feature. I am
closing as won't fix because we don't add new features to 2.x.
--
resolution: -> wont fix
stage: needs patch -> committed/rejected
status: open -&
Alexandre Vassalotti added the comment:
It is too late for 2.6.6 now that it is released.
--
resolution: -> wont fix
stage: -> committed/rejected
status: open -> closed
___
Python tracker
<http://bugs.python.o
Alexandre Vassalotti added the comment:
It is too late now for the 2.x version. And, the huge patch in issue 9410
includes an updated version of this patch for 3.x.
--
resolution: -> duplicate
stage: -> committed/rejected
status: open -> closed
superseder: -> Add Unlad
Alexandre Vassalotti added the comment:
OK I am convinced, the current behavior is fine. Let's close this one.
--
resolution: -> wont fix
stage: needs patch -> committed/rejected
status: open -> closed
___
Python tracker
<http:
Alexandre Vassalotti added the comment:
Ah, that's right Skip. You did fixed it in Unladen Swallow's trunk. I will take
a look at your solution.
http://code.google.com/p/unladen-swallow/source/diff?spec=svn1167&r=1038&format=side&path=/trunk/Modules/cPickle.c&old_pat
Alexandre Vassalotti added the comment:
Too late to make this change in 2.x. And the patch in issue 9410 includes the
optimization for 3.x.
--
resolution: -> duplicate
stage: patch review -> committed/rejected
status: open -> closed
superseder: -> Add Unladen Swallow
Alexandre Vassalotti added the comment:
I don't recall where I picked up the referred spec. Looking at PEP 307 and
pickle docs [1], it does look like that objects which provides the listitems
field of the reduce tuple should support extend() too.
It might best to try both i
Alexandre Vassalotti added the comment:
My point is I would prefer that we keep all optimizations to only the _pickle C
module and keep the Python implementation as simple as possible.
Also, I doubt the slight speedup shown by your microbenchmark will actually
result in any significant
New submission from Alexandre Vassalotti:
In pickle.py, load_appends is currently defined as
def load_appends(self):
stack = self.stack
mark = self.marker()
list = stack[mark - 1]
list.extend(stack[mark + 1:])
del stack[mark:]
However, according to
Alexandre Vassalotti added the comment:
In protocol 0, the persistent ID is restricted to alphanumeric strings because
of the problems that arise when the persistent ID contains newline characters.
_pickle likely should be changed to use the ASCII decoded. And perhaps, we
should check for
Alexandre Vassalotti added the comment:
I also wrote a patch for this. I took I slightly different approach though. I
fixed the C implementation to be more strict on the quoting. Currently, it
strips trailing non-printable characters, something pickle.py doesn't do.
I also cleaned u
Alexandre Vassalotti added the comment:
I was targeting head, not the release branches. It is fine to change the
exception there as we don't make any guarantee about the exceptions raised
during the unpickling process. It is easy enough to fix patch use ValueError
for the release branch.
Changes by Alexandre Vassalotti :
Removed file: http://bugs.python.org/file29824/fix_quoted_string_python3.patch
___
Python tracker
<http://bugs.python.org/issue17
Alexandre Vassalotti added the comment:
Here's a patch that fix the bug.
--
assignee: -> alexandre.vassalotti
keywords: +patch
stage: needs patch -> patch review
Added file: http://bugs.python.org/file29842/fix_bad_persid.patch
___
Pyt
Alexandre Vassalotti added the comment:
The mutating __getstate__ is very likely the problem here. I've attached a
small test case which shows the described behavior.
We could fix this by always making a copy of any mutable container we want to
iterate over to save its items. Perfor
Alexandre Vassalotti added the comment:
Alright alright! Here's a less bogus patch. :)
--
stage: needs patch -> patch review
Added file: http://bugs.python.org/file29846/fix_loads_appends.patch
___
Python tracker
<http://bugs.python.org
New submission from Alexandre Vassalotti:
Pickle fast mode is currently a deprecated feature of the Pickler class used to
disable its memoization mechanism. It was used mainly to create smaller output
by not emitting a PUT opcode for each object saved. Unfortunately, this mode
only worked
New submission from Alexandre Vassalotti:
As noted previously, TinyURL URL shortening API is rather slow and not always
available. Each requests takes between 0.5 and 1.5 seconds.
We should change it to use Google URL Shortener which returns a response within
10 milliseconds and is much more
Alexandre Vassalotti added the comment:
In addition, we could bring back the switch dispatch based on the first letter
of the type name. It does seem to speed up things as well but as much as the
type cache optimization.
--
nosy: +pitrou, serhiy.storchaka
title: Optimize pickling
Alexandre Vassalotti added the comment:
Here's a patch to fix the exception.
--
keywords: +patch
Added file: http://bugs.python.org/file29949/fix_array_err_msg.patch
___
Python tracker
<http://bugs.python.org/i
Changes by Alexandre Vassalotti :
--
resolution: -> fixed
stage: patch review -> committed/rejected
status: open -> closed
___
Python tracker
<http://bugs.python.or
Changes by Alexandre Vassalotti :
--
resolution: -> fixed
stage: patch review -> committed/rejected
status: open -> closed
___
Python tracker
<http://bugs.python.or
Alexandre Vassalotti added the comment:
Are you asking why do we need to call both PyMemoTable_Get and memo_get? Or,
why do we fetching the memo was moved to the save functions? For the former,
there is no real reason. The extra call could be removed though profiling
doesn't show this ca
New submission from Alexandre Vassalotti:
I have restarted the work on PEP 3154. Stefan Mihaila had begun an
implementation as part of the Google Summer of Code 2012. Unfortunately, he hit
multiple roadblocks which prevented him to finish his work by the end of the
summer. He previously shown
Alexandre Vassalotti added the comment:
I have started a new implementation of PEP 3154 since Stefan hasn't been active
on his. Moving the discussion to Issue #17810.
--
dependencies: -Unbinding of methods
resolution: -> out of date
stage: patch review -> committed/reje
Changes by Alexandre Vassalotti :
--
dependencies: +Unbinding of methods
___
Python tracker
<http://bugs.python.org/issue17810>
___
___
Python-bugs-list mailin
Alexandre Vassalotti added the comment:
I think it should be fixed though it is not a high-priority issue.
--
___
Python tracker
<http://bugs.python.org/issue16
Alexandre Vassalotti added the comment:
Antoine, I removed STACK_GLOBAL when I found performance issues with the
implementation. The changeset that added it had some unrelated changes that
made it harder to debug than necessary. I am planning to re-add it when I
worked out the kinks
New submission from Alexandre Vassalotti:
The changeset 2dd046be2c88 introduced _PyObject_CallMethodObjIdArgs. This API
should have been named _PyObject_CallMethodIdObjArgs since it is the variant of
_PyObject_CallMethodId which takes object arguments instead of building
arguments from a
New submission from Alexandre Vassalotti:
I have tried to clean up a bit of the implementation of the reduce protocol in
typeobject.c in preparation for PEP 3154's support of classes with __new__
using keyword-only arguments. I am not quite done yet with the refactorings,
but I
Changes by Alexandre Vassalotti :
--
dependencies: +Refactor reduce protocol implementation
___
Python tracker
<http://bugs.python.org/issue17810>
___
___
Pytho
Alexandre Vassalotti added the comment:
The latest framing patch looks pretty nice overall. One concern is we need to
make sure the C implementation call _Pickler_OpcodeBoundary often enough to
keep the frames around the sizes. For example, batch_save_list and
batch_save_dict can currently
Alexandre Vassalotti added the comment:
> I don't understand how that can happen. batch_list() and batch_dict()
> both call save() for each item, and save() calls
> _Pickler_OpcodeBoundary() at the end. Have I missed something?
Ah, you're right. I was thinking in terms of my
Changes by Alexandre Vassalotti :
--
dependencies: +Implement PEP 3154 (pickle protocol 4)
___
Python tracker
<http://bugs.python.org/issue4727>
___
___
Python-bug
Alexandre Vassalotti added the comment:
It is an integer overflow issue. It is easy to reproduce without numpy:
$ python2.7 -c "import cPickle; cPickle.dumps('\x00' * 2**31)"
Traceback (most recent call last):
File "", line 1, in
SystemError: error return withou
Changes by Alexandre Vassalotti :
--
dependencies: +Implement PEP 3154 (pickle protocol 4)
___
Python tracker
<http://bugs.python.org/issue9276>
___
___
Python-bug
Alexandre Vassalotti added the comment:
If you are pickling large objects, you're likely hitting issue #11872. We fixed
most 64-bit issues in Python 3, so upgrading might be solution if possible.
Since the particular bug you are hitting cannot be reproduced with your test
case, I am cl
Alexandre Vassalotti added the comment:
Without a test case, we cannot tell if this is a bug in pickle or not. Anyhow,
Floris's explanation is pretty much on the dot as why you might see this error.
--
nosy: +alexandre.vassalotti
resolution: -> works for me
stage: ->
Alexandre Vassalotti added the comment:
There is no guarantee the binary representation of pickled data will be same
between different runs. We try to make it mostly consistent when we can, but
there are cases, like this one, where we cannot ensure consistency without
hurting performance
Alexandre Vassalotti added the comment:
I fixed this while working on PEP 3154
[http://hg.python.org/features/pep-3154-alexandre/rev/eed9142d664f]. The
relevant piece is
@@ -420,7 +424,13 @@ class _Pickler:
write(REDUCE)
if obj is not None:
-self.memoize
Alexandre Vassalotti added the comment:
I am currently fleshing out an improved implementation for the reduce protocol
version 4. One thing I am curious about is whether we should keep the special
cases we currently have there for dict and list subclasses.
I recall Raymond expressed
Alexandre Vassalotti added the comment:
Those methods wouldn't be much more a maintenance burden than the special cases
already present in the implementation of __reduce__. These methods would only
need to be provided by classes that wishes to support efficient in-place
pickling provid
Alexandre Vassalotti added the comment:
The framing patch seems to have a significant negative effect on performance.
Report on Linux avassalotti 3.2.5-gg1130 #1 SMP Mon Feb 4 02:25:47 PST 2013
x86_64 x86_64
Total CPU cores: 12
### fastpickle ###
Min: 0.447194 -> 0.505841: 1.13x slower
Alexandre Vassalotti added the comment:
Do you have benchmark results to show the code with the patch is faster?
--
___
Python tracker
<http://bugs.python.org/issue17
Alexandre Vassalotti added the comment:
Antoine, can you share the code for your benchmarks which show performance
improvements when framing is enabled? I am seeing the same 10-15% slowdown even
when pickling stuff to pure Python objects:
### Without the patch
./python -m timeit -r 50 -s
Alexandre Vassalotti added the comment:
Our hands are pretty much tied here. The pickle bytearray as unicode hack is
likely the best we can do without pickling compatibility between Python 2 and
3. I can't think of a solution that could work here. For example.
1. Pickling bytearrays
Alexandre Vassalotti added the comment:
Thanks Stefan for the patch. It's very much appreciated. I will try to review
it soon.
Of the features you proposed, the twos I would like to take a look again is
implicit memoization and the BAIL_OUT opcode. For the implicit memoization
featur
Changes by Alexandre Vassalotti :
Removed file: http://bugs.python.org/file30229/pickle4+methods.patch
___
Python tracker
<http://bugs.python.org/issue17810>
___
___
Pytho
Alexandre Vassalotti added the comment:
Stefan, I took a quick look at your patch. There is a couple things that stands
out.
First, I think the implementation of BINGLOBAL and BINGLOBAL_BIG should be
moved to another patch. Adding a binary version opcode for GLOBAL is a separate
feature and
501 - 600 of 602 matches
Mail list logo
