Re: [Python-Dev] Mysterious Python pyc file corruption problems

2013-05-16 Thread Antoine Pitrou
On Thu, 16 May 2013 11:42:30 -0400 Barry Warsaw wrote: > On May 16, 2013, at 08:33 AM, Nick Coghlan wrote: > > >Personally, I would be suspicious of developmental web services doing > >auto-reloading while an installer is recompiling the world. I don't have > >enough context to be sure how plaus

Re: [Python-Dev] Why is nb_inplace_add copied to sq_inplace_concat?

2013-05-16 Thread Matt Newell
On Thursday, May 16, 2013 08:41:32 PM you wrote: > On Fri, May 17, 2013 at 1:38 PM, Nick Coghlan wrote: > > On Fri, May 17, 2013 at 9:17 AM, Matt Newell wrote: > >> I don't really understand what the fixup_slot_dispatchers function is > >> doing, but it does seem like there must be a bug either i

Re: [Python-Dev] Mysterious Python pyc file corruption problems

2013-05-16 Thread Tres Seaver
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 05/16/2013 06:59 PM, Nick Coghlan wrote: > 3.2 uses __pycache__, so it should only potentially conflict within > the same version. > > I haven't heard any rumblings about anything like this in Fedora or > RHEL, so my suspicions still lean toward

Re: [Python-Dev] Why is nb_inplace_add copied to sq_inplace_concat?

2013-05-16 Thread Nick Coghlan
On Fri, May 17, 2013 at 1:38 PM, Nick Coghlan wrote: > On Fri, May 17, 2013 at 9:17 AM, Matt Newell wrote: >> I don't really understand what the fixup_slot_dispatchers function is doing, >> but it does seem like there must be a bug either in what it's doing, or in >> PyNumber_InPlaceAdd's handlin

Re: [Python-Dev] Why is nb_inplace_add copied to sq_inplace_concat?

2013-05-16 Thread Nick Coghlan
On Fri, May 17, 2013 at 9:17 AM, Matt Newell wrote: > I don't really understand what the fixup_slot_dispatchers function is doing, > but it does seem like there must be a bug either in what it's doing, or in > PyNumber_InPlaceAdd's handling of a NotImplemented return value from > sq_inplace_concat

Re: [Python-Dev] [Python-checkins] cpython: fix compilation on Windows

2013-05-16 Thread Terry Reedy
On 5/16/2013 4:17 PM, victor.stinner wrote: summary: fix compilation on Windows That fixed my problem with compiling 3.4, 32 bit, Win 7. Thanks. But I cannot compile 3.3 python_d since May 6. In fact, there are more errors now than 8 hours ago. 7 things failed to build instead of 5 (3 is no

[Python-Dev] Why is nb_inplace_add copied to sq_inplace_concat?

2013-05-16 Thread Matt Newell
I have encountered what I believe to be a bug but I'm sure there is some reason things are done as they are and I am hoping someone can shed some light or confirm it is indeed a bug. As a bit of background I have a c++ class that I use sip to generate python bindings. The potential python bug

Re: [Python-Dev] Mysterious Python pyc file corruption problems

2013-05-16 Thread Nick Coghlan
On 17 May 2013 08:37, "Guido van Rossum" wrote: > > On Thu, May 16, 2013 at 3:27 PM, Greg Ewing wrote: > > Guido van Rossum wrote: > >> > >> This reminds me of the following bug, which can happen when two > >> processes are both writing the .pyc file and a third is reading it. > >> ... I think a

Re: [Python-Dev] Mysterious Python pyc file corruption problems

2013-05-16 Thread Guido van Rossum
On Thu, May 16, 2013 at 3:27 PM, Greg Ewing wrote: > Guido van Rossum wrote: >> >> This reminds me of the following bug, which can happen when two >> processes are both writing the .pyc file and a third is reading it. >> ... I think all the errors are >> >> actually explainable from this scenario

Re: [Python-Dev] Mysterious Python pyc file corruption problems

2013-05-16 Thread Greg Ewing
Guido van Rossum wrote: This reminds me of the following bug, which can happen when two processes are both writing the .pyc file and a third is reading it. ... I think all the errors are actually explainable from this scenario. The second writer will still carry on to write a valid .pyc file,

Re: [Python-Dev] Mysterious Python pyc file corruption problems

2013-05-16 Thread Thomas Wouters
On Thu, May 16, 2013 at 11:19 PM, Guido van Rossum wrote: > This reminds me of the following bug, which can happen when two > processes are both writing the .pyc file and a third is reading it. > First some background. > > When writing a .pyc file, we use the following strategy: > - open the file

Re: [Python-Dev] Mysterious Python pyc file corruption problems

2013-05-16 Thread Terry Jan Reedy
On 5/16/2013 5:30 PM, Brett Cannon wrote: On Thu, May 16, 2013 at 5:19 PM, Guido van Rossum wrote: This reminds me of the following bug, which can happen when two processes are both writing the .pyc file and a third is reading it. First some background. When writing a .pyc file, we use the fol

Re: [Python-Dev] Mysterious Python pyc file corruption problems

2013-05-16 Thread Brett Cannon
On Thu, May 16, 2013 at 5:40 PM, Guido van Rossum wrote: > I still suspect this might explain most of what Barry saw, if not all. Quite possible, especially since he is seeing more issues on 3.2 than 3.3. Just wanted to fill people in on how 3.3 onwards does things is all. -Brett > — > Sent fro

Re: [Python-Dev] Mysterious Python pyc file corruption problems

2013-05-16 Thread Guido van Rossum
I still suspect this might explain most of what Barry saw, if not all.  — Sent from Mailbox On Thu, May 16, 2013 at 2:36 PM, Brett Cannon wrote: > On Thu, May 16, 2013 at 5:19 PM, Guido van Rossum wrote: >> This reminds me of the following bug, which can happen when two >> processes are both wr

Re: [Python-Dev] Mysterious Python pyc file corruption problems

2013-05-16 Thread Brett Cannon
On Thu, May 16, 2013 at 5:19 PM, Guido van Rossum wrote: > This reminds me of the following bug, which can happen when two > processes are both writing the .pyc file and a third is reading it. > First some background. > > When writing a .pyc file, we use the following strategy: > - open the file

Re: [Python-Dev] Mysterious Python pyc file corruption problems

2013-05-16 Thread Guido van Rossum
This reminds me of the following bug, which can happen when two processes are both writing the .pyc file and a third is reading it. First some background. When writing a .pyc file, we use the following strategy: - open the file for writing - write a dummy header (four null bytes) - write the .py f

Re: [Python-Dev] Mysterious Python pyc file corruption problems

2013-05-16 Thread Terry Jan Reedy
On 5/16/2013 2:04 PM, Barry Warsaw wrote: No, it's all different kinds of machines, at different times, on different files. So far, there's no rhyme or reason to the corruptions that I can tell. If the corruption only happens on Ubuntu, that would constitute 'rhyme' ;-). I realize that askin

Re: [Python-Dev] Mysterious Python pyc file corruption problems

2013-05-16 Thread Gregory P. Smith
On Thu, May 16, 2013 at 11:04 AM, Barry Warsaw wrote: > On May 16, 2013, at 09:44 AM, Ethan Furman wrote: > > >Is it happening on the same machines? If so, perhaps a daemon to monitor > >those files and then scream and shout when one changes. Might help track > >down what's going on at the time

Re: [Python-Dev] Mysterious Python pyc file corruption problems

2013-05-16 Thread Barry Warsaw
On May 16, 2013, at 09:44 AM, Ethan Furman wrote: >Is it happening on the same machines? If so, perhaps a daemon to monitor >those files and then scream and shout when one changes. Might help track >down what's going on at the time. (Yeah, that does sound like saying >'inotify' but with more wo

Re: [Python-Dev] Mysterious Python pyc file corruption problems

2013-05-16 Thread Ethan Furman
On 05/16/2013 09:38 AM, Barry Warsaw wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA256 On May 16, 2013, at 06:22 PM, Christian Heimes wrote: Are you able to reproduce the issue? Perhaps you could use inotify to track down file activity. It shouldn't affect timing much and you can track if

Re: [Python-Dev] Mysterious Python pyc file corruption problems

2013-05-16 Thread Barry Warsaw
-BEGIN PGP SIGNED MESSAGE- Hash: SHA256 On May 16, 2013, at 06:22 PM, Christian Heimes wrote: >Are you able to reproduce the issue? Perhaps you could use inotify to >track down file activity. It shouldn't affect timing much and you can >track if more than one process it writing to the sam

Re: [Python-Dev] Mysterious Python pyc file corruption problems

2013-05-16 Thread Christian Heimes
-BEGIN PGP SIGNED MESSAGE- Hash: SHA256 Am 16.05.2013 17:40, schrieb Barry Warsaw: > We've since found a few cases where Python 3.3 pyc files are > probably corrupted, so that shoots down my theory about a race > condition on reading/writing pyc files, since 3.3 implements > atomic-rename

Re: [Python-Dev] Mysterious Python pyc file corruption problems

2013-05-16 Thread Barry Warsaw
On May 16, 2013, at 08:33 AM, Nick Coghlan wrote: >Personally, I would be suspicious of developmental web services doing >auto-reloading while an installer is recompiling the world. I don't have >enough context to be sure how plausible that is as a possible explanation, >though. It's possible tha

Re: [Python-Dev] Mysterious Python pyc file corruption problems

2013-05-16 Thread Barry Warsaw
-BEGIN PGP SIGNED MESSAGE- Hash: SHA256 On May 15, 2013, at 06:06 PM, Tres Seaver wrote: >On 05/15/2013 04:58 PM, Barry Warsaw wrote: >> This leads me to hypothesize that the bug is due to an as yet >> unidentified race condition during installation of Python source code >> on Ubuntu, whi

Re: [Python-Dev] [RELEASED] Python 3.2.5 and Python 3.3.2

2013-05-16 Thread Benjamin Peterson
2013/5/16 Serhiy Storchaka : > 16.05.13 08:20, Georg Brandl написав(ла): > >> On behalf of the Python development team, I am pleased to announce the >> releases of Python 3.2.5 and 3.3.2. >> >> The releases fix a few regressions in 3.2.4 and 3.3.1 in the zipfile, gzip >> and xml.sax modules. Detai

Re: [Python-Dev] [RELEASED] Python 3.2.5 and Python 3.3.2

2013-05-16 Thread Charles-François Natali
2013/5/16 Serhiy Storchaka : > 16.05.13 08:20, Georg Brandl написав(ла): >> >> On behalf of the Python development team, I am pleased to announce the >> releases of Python 3.2.5 and 3.3.2. >> >> The releases fix a few regressions in 3.2.4 and 3.3.1 in the zipfile, gzip >> and xml.sax modules. Deta

Re: [Python-Dev] [RELEASED] Python 2.7.5

2013-05-16 Thread Martin v. Löwis
Am 16.05.13 10:42, schrieb Ben Hoyt: > FYI, I tried this just now with Python 2.7.4 running, and the > installer nicely tells you that "some files that need to be updated > are currently in use ... the following applications are using files, > please close them and click Retry ... python.exe (Proc

Re: [Python-Dev] [RELEASED] Python 3.2.5 and Python 3.3.2

2013-05-16 Thread Serhiy Storchaka
16.05.13 08:20, Georg Brandl написав(ла): On behalf of the Python development team, I am pleased to announce the releases of Python 3.2.5 and 3.3.2. The releases fix a few regressions in 3.2.4 and 3.3.1 in the zipfile, gzip and xml.sax modules. Details can be found in the changelogs: It seems

Re: [Python-Dev] [RELEASED] Python 2.7.5

2013-05-16 Thread Ben Hoyt
>> Yes, I update all the time, but without python running. FYI, I tried this just now with Python 2.7.4 running, and the installer nicely tells you that "some files that need to be updated are currently in use ... the following applications are using files, please close them and click Retry ... py

Re: [Python-Dev] [RELEASED] Python 2.7.5

2013-05-16 Thread Ben Hoyt
This may not be a python-dev question exactly. But on Windows, is it > safe to update to 2.7.5 on top of 2.7.4 (at C:\Python27) using the .msi >> installer? In other words, will it update/add/remove all the files >> correctly? What if python.exe is running? >> > > Yes, I update all the time, but w

Re: [Python-Dev] [RELEASED] Python 2.7.5

2013-05-16 Thread Terry Jan Reedy
On 5/16/2013 1:18 AM, Ben Hoyt wrote: Thanks, Benjamin -- that's great! This may not be a python-dev question exactly. But on Windows, is it safe to update to 2.7.5 on top of 2.7.4 (at C:\Python27) using the .msi installer? In other words, will it update/add/remove all the files correctly? What