[issue11277] test_zlib.test_big_buffer crashes under BSD (Mac OS X and FreeBSD)

2011-04-15 Thread Steffen Daode Nurpmeso
Steffen Daode Nurpmeso added the comment: My last idea for today was to split the writes. This also works for the C version, but it does not for test_zlib.py. I attach the updated files. And for completeness: Adler-32 <7a54018b> CRC-32 <7f1be672> -- @test_13713_tmp Adler-

[issue11277] test_zlib.test_big_buffer crashes under BSD (Mac OS X and FreeBSD)

2011-04-16 Thread Steffen Daode Nurpmeso
Steffen Daode Nurpmeso added the comment: Yet another bug of Mac OS X: it sometimes creates messed up sparse regions: 14:00 ~/tmp/test $ ~/src/cpython/python.exe test_mmap.py .. 14:01 ~/tmp/test $ zsum32 py-mmap-testfile Adler-32 CRC-32 <78ebae7a> -- py-mmap-testfile 14:03 ~/tm

[issue11277] test_zlib.test_big_buffer crashes under BSD (Mac OS X and FreeBSD)

2011-04-16 Thread Steffen Daode Nurpmeso
Steffen Daode Nurpmeso added the comment: On Sat, Apr 16, 2011 at 02:55:29PM +, Nadeem Vawda wrote: > I don't think it is necessary to further investigate the behaviour of > Snow Leopard's mmap() - we know that it's broken, and we have a fix. > > At the moment,

[issue11700] mailbox.py proxy updates

2011-04-17 Thread Steffen Daode Nurpmeso
Steffen Daode Nurpmeso added the comment: On Sat, Apr 16, 2011 at 08:17:30PM +, R. David Murray wrote: > [...] rather odd considering that there is also a 'closed' > method that would fail similarly if close was ever called. Maybe someone got not enough feedback after she

[issue11277] test_zlib.test_big_buffer crashes under BSD (Mac OS X and FreeBSD)

2011-04-19 Thread Steffen Daode Nurpmeso
Steffen Daode Nurpmeso added the comment: Took some time, but here is a patch that makes mmap(2) work on Mac OS X. This also applies to #11779. Background: on OS X, fsync(2) seems to behave as fdatasync(2). To give people the possibility to do some kind of fync(2) nonetheless, a new fcntl(2

[issue11779] test_mmap timeout (1 hour) on "AMD64 Snow Leopard 3.x" buildbot

2011-04-19 Thread Steffen Daode Nurpmeso
Steffen Daode Nurpmeso added the comment: I think that will also be healed with the patch i've just posted to #11277. (However, though i can't ,reproduce`, it seems the Microkernel/IOKit interaction sometimes has a problem, because i *really* had messed up sparse files (about 1-2 per

[issue11277] test_zlib.test_big_buffer crashes under BSD (Mac OS X and FreeBSD)

2011-04-19 Thread Steffen Daode Nurpmeso
Changes by Steffen Daode Nurpmeso : Removed file: http://bugs.python.org/file20861/doc_lib_mmap.patch ___ Python tracker <http://bugs.python.org/issue11277> ___ ___ Pytho

[issue11277] test_zlib.test_big_buffer crashes under BSD (Mac OS X and FreeBSD)

2011-04-19 Thread Steffen Daode Nurpmeso
Changes by Steffen Daode Nurpmeso : Removed file: http://bugs.python.org/file21671/11277.1.diff ___ Python tracker <http://bugs.python.org/issue11277> ___ ___ Python-bug

[issue11277] test_zlib.test_big_buffer crashes under BSD (Mac OS X and FreeBSD)

2011-04-19 Thread Steffen Daode Nurpmeso
Changes by Steffen Daode Nurpmeso : Removed file: http://bugs.python.org/file21675/11277.2.diff ___ Python tracker <http://bugs.python.org/issue11277> ___ ___ Python-bug

[issue11277] test_zlib.test_big_buffer crashes under BSD (Mac OS X and FreeBSD)

2011-04-19 Thread Steffen Daode Nurpmeso
Steffen Daode Nurpmeso added the comment: (The working patch is http://bugs.python.org/file21715/11277.3.diff.) -- ___ Python tracker <http://bugs.python.org/issue11

[issue11277] test_zlib.test_big_buffer crashes under BSD (Mac OS X and FreeBSD)

2011-04-19 Thread Steffen Daode Nurpmeso
Changes by Steffen Daode Nurpmeso : Removed file: http://bugs.python.org/file21672/11277.mmap.c ___ Python tracker <http://bugs.python.org/issue11277> ___ ___ Python-bug

[issue11277] test_zlib.test_big_buffer crashes under BSD (Mac OS X and FreeBSD)

2011-04-19 Thread Steffen Daode Nurpmeso
Changes by Steffen Daode Nurpmeso : Removed file: http://bugs.python.org/file21676/11277.mmap-1.c ___ Python tracker <http://bugs.python.org/issue11277> ___ ___ Python-bug

[issue11277] test_zlib.test_big_buffer crashes under BSD (Mac OS X and FreeBSD)

2011-04-19 Thread Steffen Daode Nurpmeso
Changes by Steffen Daode Nurpmeso : Removed file: http://bugs.python.org/file21683/11277.mmap-2.c ___ Python tracker <http://bugs.python.org/issue11277> ___ ___ Python-bug

[issue11277] test_zlib.test_big_buffer crashes under BSD (Mac OS X and FreeBSD)

2011-04-19 Thread Steffen Daode Nurpmeso
Changes by Steffen Daode Nurpmeso : Removed file: http://bugs.python.org/file21684/11277.mmap-2.py ___ Python tracker <http://bugs.python.org/issue11277> ___ ___ Pytho

[issue11277] test_zlib.test_big_buffer crashes under BSD (Mac OS X and FreeBSD)

2011-04-19 Thread Steffen Daode Nurpmeso
Steffen Daode Nurpmeso added the comment: (Dropped the tests, too.) -- ___ Python tracker <http://bugs.python.org/issue11277> ___ ___ Python-bugs-list mailin

[issue11277] Crash with mmap and sparse files on Mac OS X

2011-04-19 Thread Steffen Daode Nurpmeso
Steffen Daode Nurpmeso added the comment: Updated 11277.4.diff also includes mmap.rst update. (Maybe os.fsync() and os.sync() should be modified to really do that fcntl, too? I'll think i'll open an issue with patch soon.) -- Added file: http://bugs.python.org/file21717/11

[issue11300] mmap() large file failures on Mac OS X docfix

2011-04-19 Thread Steffen Daode Nurpmeso
Steffen Daode Nurpmeso added the comment: Outdated due to found #11277 solution. -- resolution: -> fixed status: open -> closed ___ Python tracker <http://bugs.python.org/i

[issue11877] Mac OS X fsync() should really be fcntl(F_FULLFSYNC)

2011-04-19 Thread Steffen Daode Nurpmeso
New submission from Steffen Daode Nurpmeso : Issue 11277 revealed that Mac OS X fsync() does not always and gracefully synchronize file data to physical backing store. To gain real fsync(2) behaviour fcntl(2) must be called with the F_FULLFSYNC flag. -- components: Library (Lib) files

[issue11277] test_zlib.test_big_buffer crashes under BSD (Mac OS X and FreeBSD)

2011-04-19 Thread Steffen Daode Nurpmeso
Steffen Daode Nurpmeso added the comment: On Tue, Apr 19, 2011 at 10:34:11AM +, STINNER Victor wrote: > 11277.3.diff: this patch looks correct Unbelievable - you really fought yourself through this immense bunch of code in such a short time! :) -- title: Crash with mmap and spa

[issue11277] Crash with mmap and sparse files on Mac OS X

2011-04-19 Thread Steffen Daode Nurpmeso
Steffen Daode Nurpmeso added the comment: (My last reply-mail changed the title. Fixing.) -- title: test_zlib.test_big_buffer crashes under BSD (Mac OS X and FreeBSD) -> Crash with mmap and sparse files on Mac OS X ___ Python tracker &l

[issue11867] Make test_mailbox deterministic

2011-04-19 Thread Steffen Daode Nurpmeso
Steffen Daode Nurpmeso added the comment: Nice ping pong you play.. I, buildbot get: Using random seed 2215045 [1/1] test_mailbox test test_mailbox failed -- multiple errors occurred; run in verbose mode for details test test_mailbox failed -- Traceback (most recent call last): File "/

[issue11867] Make test_mailbox deterministic

2011-04-19 Thread Steffen Daode Nurpmeso
Steffen Daode Nurpmeso added the comment: Short glance into it, and after commenting out self.c_sock_close = self.c_sock_shutdown = True in the parent process it ends up as (maybe i can spend more time this evening): 15:36 ~/tmp $ python3 -E -Wd -m test -r -w -uall test_mailbox Using

[issue11867] Make test_mailbox deterministic

2011-04-20 Thread Steffen Daode Nurpmeso
Steffen Daode Nurpmeso added the comment: 'Was not allowed to look yesterday. If the child only closes and not self.c_sock_shutdown = True (shutdown is an ugly word for a child anyway) then (and i hope Apple did not modify the BSD network stack): 12:59 ~/tmp $ python3 -E -

[issue11877] Mac OS X fsync() should really be fcntl(F_FULLFSYNC)

2011-04-20 Thread Steffen Daode Nurpmeso
Steffen Daode Nurpmeso added the comment: > when one calls fsync, he expects [...] > Fixing this deficiency through Python's exposed fsync [...] I think so, too. http://pubs.opengroup.org/onlinepubs/009695399/functions/fsync.html even permits "null implementation"s etc.

[issue11877] Mac OS X fsync() should really be fcntl(F_FULLFSYNC)

2011-04-20 Thread Steffen Daode Nurpmeso
Steffen Daode Nurpmeso added the comment: On Wed, Apr 20, 2011 at 12:16:39PM +, Ronald Oussoren wrote: > This is the same behavior as fsync on OSX, and OSX also has > a second API that provides stronger guarantees. > With your patch it is no longer possible to call the C function &

[issue11877] Mac OS X fsync() should really be fcntl(F_FULLFSYNC)

2011-04-20 Thread Steffen Daode Nurpmeso
Steffen Daode Nurpmeso added the comment: On Wed, Apr 20, 2011 at 12:16:39PM +, Ronald Oussoren wrote: > This is the same behavior as fsync on OSX, and OSX also has > a second API that provides stronger guarantees. > With your patch it is no longer possible to call the C function &

[issue11877] Mac OS X fsync() should really be fcntl(F_FULLFSYNC)

2011-04-20 Thread Steffen Daode Nurpmeso
Steffen Daode Nurpmeso added the comment: 8-} The real message was: On Wed, Apr 20, 2011 at 12:16:39PM +, Antoine Pitrou wrote: > If Apple thinks [...] there's no reason for us Apple thinks (KernelProgramming.pdf, BPFileSystem.pdf): Kernel code must be nearly perfect.

[issue11877] Mac OS X fsync() should really be fcntl(F_FULLFSYNC)

2011-04-20 Thread Steffen Daode Nurpmeso
Steffen Daode Nurpmeso added the comment: Ronald Oussoren wrote: > Adding the F_FULLSYNC option to os.fsync would be fine though To show you that i'm not unteachable i'll attach a patch which does that. This approach can be heavily extended, then, e.g. by using sync_file_range(

[issue11877] Change os.fsync() to support physical backing store syncs

2011-04-21 Thread Steffen Daode Nurpmeso
Steffen Daode Nurpmeso added the comment: I'm convinced. And i've rewritten the patch. Now fsync(2) is always called first *regardless* of the new optional argument. The documentation is (a little bit) better now. And i've added support for NetBSD, AIX and Linux; though:

[issue11877] Change os.fsync() to support physical backing store syncs

2011-04-21 Thread Steffen Daode Nurpmeso
Changes by Steffen Daode Nurpmeso : Removed file: http://bugs.python.org/file21718/fsync.diff ___ Python tracker <http://bugs.python.org/issue11877> ___ ___ Python-bug

[issue11877] Change os.fsync() to support physical backing store syncs

2011-04-21 Thread Steffen Daode Nurpmeso
Steffen Daode Nurpmeso added the comment: P.S.: dito Linux and NetBSD. I've reused preprocessor tests we've discovered internally over the past years, but i cannot test this here. I could test Linux next Tuesd

[issue11877] Change os.fsync() to support physical backing store syncs

2011-04-21 Thread Steffen Daode Nurpmeso
Steffen Daode Nurpmeso added the comment: Ok, 11877.3.diff uses either-or. -- Added file: http://bugs.python.org/file21749/11877.3.diff ___ Python tracker <http://bugs.python.org/issue11877> ___diff --git

[issue11877] Change os.fsync() to support physical backing store syncs

2011-04-21 Thread Steffen Daode Nurpmeso
Changes by Steffen Daode Nurpmeso : Removed file: http://bugs.python.org/file21742/11877.optarg-1.diff ___ Python tracker <http://bugs.python.org/issue11877> ___ ___ Pytho

[issue11877] Change os.fsync() to support physical backing store syncs

2011-04-21 Thread Steffen Daode Nurpmeso
Changes by Steffen Daode Nurpmeso : Removed file: http://bugs.python.org/file21748/11877.2.diff ___ Python tracker <http://bugs.python.org/issue11877> ___ ___ Python-bug

[issue11877] Change os.fsync() to support physical backing store syncs

2011-04-21 Thread Steffen Daode Nurpmeso
Steffen Daode Nurpmeso added the comment: Charles-Francois Natali wrote: > I'm -10 on sync_file_range on Linux: > [...] last time I checked [...] I just looked at http://git.kernel.org/?p=linux/kernel/git/torvalds/linux-2.6.git;a=blob;f=fs/sync.c;h=c38ec163da6ccba00a0146c75606c

[issue11877] Change os.fsync() to support physical backing store syncs

2011-04-25 Thread Steffen Daode Nurpmeso
Steffen Daode Nurpmeso added the comment: I'll attach 11877.4.diff: - Docu change (i hope to be better) - Kept NetBSD after looking into http://cvsweb.netbsd.org/bsdweb.cgi/src/sys/kern/vfs_syscalls.c?rev=1.423&content-type=text/x-cvsweb-markup&only_with_tag=MAIN becaus

[issue11935] MMDF/MBOX mailbox need utime

2011-04-27 Thread Steffen Daode Nurpmeso
New submission from Steffen Daode Nurpmeso : According to the de-facto MBOX standard [1] and the MMDF description [2] mtime and atime are used to detect wether a mailbox has new mail: If the mtime on a nonempty mbox file is greater than the atime, the file has new mail. For [1] this is

[issue11277] Crash with mmap and sparse files on Mac OS X

2011-04-27 Thread Steffen Daode Nurpmeso
Steffen Daode Nurpmeso added the comment: What do you think - i think this issue can really be closed now. I'll attach a final 11277.5.diff which has a less irritated and thus better understandable comment than .4.diff. I'll also drop .3 and .4. A lot of noise again 8| --

[issue11277] Crash with mmap and sparse files on Mac OS X

2011-04-27 Thread Steffen Daode Nurpmeso
Changes by Steffen Daode Nurpmeso : Removed file: http://bugs.python.org/file21715/11277.3.diff ___ Python tracker <http://bugs.python.org/issue11277> ___ ___ Python-bug

[issue11277] Crash with mmap and sparse files on Mac OS X

2011-04-27 Thread Steffen Daode Nurpmeso
Changes by Steffen Daode Nurpmeso : Removed file: http://bugs.python.org/file21717/11277.4.diff ___ Python tracker <http://bugs.python.org/issue11277> ___ ___ Python-bug

[issue11935] MMDF/MBOX mailbox need utime

2011-05-02 Thread Steffen Daode Nurpmeso
Steffen Daode Nurpmeso added the comment: On Sun, 1 May 2011 00:15:11 +0200, R. David Murray wrote: > The problem with this patch is that it would also show 'new > mail' if what had in fact happened was that a message had been > *deleted* (see the comments at the beg

[issue11935] MMDF/MBOX mailbox need utime

2011-05-02 Thread Steffen Daode Nurpmeso
Steffen Daode Nurpmeso added the comment: I'll attach a patch with a clearer comment (entry-gate instead "new mail"), i.e. the comment now reflects what MUAs really do. -- Added file: http://bugs.python.org/file21850/11935.2.diff ___

[issue11935] MMDF/MBOX mailbox need utime

2011-05-02 Thread Steffen Daode Nurpmeso
Changes by Steffen Daode Nurpmeso : Removed file: http://bugs.python.org/file21795/mailbox.diff ___ Python tracker <http://bugs.python.org/issue11935> ___ ___ Python-bug

[issue11277] Crash with mmap and sparse files on Mac OS X

2011-05-02 Thread Steffen Daode Nurpmeso
Steffen Daode Nurpmeso added the comment: On Mon, 2 May 2011 01:22:41 +0200, STINNER Victor wrote: > @sdaoden: Can you try on Python 2.7? @haypo: Python 2.7 is absolute horror. But i tried and produced a (terrible - i don't know the test framework and that test_support stuff seems

[issue11277] Crash with mmap and sparse files on Mac OS X

2011-05-02 Thread Steffen Daode Nurpmeso
Changes by Steffen Daode Nurpmeso : Removed file: http://bugs.python.org/file21673/11277.zsum32.c ___ Python tracker <http://bugs.python.org/issue11277> ___ ___ Python-bug

[issue11935] MMDF/MBOX mailbox need utime

2011-05-02 Thread Steffen Daode Nurpmeso
Steffen Daode Nurpmeso added the comment: On Sun, 1 May 2011 00:15:11 +0200, R. David Murray wrote: > So actually fixing this is a bit more complicated. I like Pear OS X! I've just tried around a bit with the other of the Pear OS filesystems (claims to "support" two fsys:

[issue11935] MMDF/MBOX mailbox need utime

2011-05-02 Thread Steffen Daode Nurpmeso
Steffen Daode Nurpmeso added the comment: Sorry, the last message has been truncated, i've opened http://psf.upfronthosting.co.za/roundup/meta/issue397. Forget the first line, but for non-believers: PYP$ t=time.time(); os.utime('org.python', (t-2.42,t)); print(os.s

[issue10044] small int optimization

2011-05-03 Thread Steffen Daode Nurpmeso
Steffen Daode Nurpmeso added the comment: Now me. (http://gcc.gnu.org/onlinedocs/gcc/Arrays-and-pointers-implementation.html#Arrays-and-pointers-implementation) > When casting from pointer to integer and back again, the resulting > pointer must reference the same object as the original p

[issue11277] Crash with mmap and sparse files on Mac OS X

2011-05-03 Thread Steffen Daode Nurpmeso
Steffen Daode Nurpmeso added the comment: > Should we fix Python 2.7? > - backport issue #8651 > - use PY_SSIZE_T_CLEAN in zlibmodule.c I really thought about this over night. I'm a C programmer and thus: - Produce no bugs - If you've produced a bug, fix it at once - If

[issue11277] Crash with mmap and sparse files on Mac OS X

2011-05-03 Thread Steffen Daode Nurpmeso
Changes by Steffen Daode Nurpmeso : Removed file: http://bugs.python.org/file21855/11277-27.1.diff ___ Python tracker <http://bugs.python.org/issue11277> ___ ___ Pytho

[issue10276] zlib crc32/adler32 buffer length truncation (64-bit)

2011-05-03 Thread Steffen Daode Nurpmeso
Steffen Daode Nurpmeso added the comment: Ha! I always knew it! -- ___ Python tracker <http://bugs.python.org/issue10276> ___ ___ Python-bugs-list mailin

[issue11997] One typo in Doc/c-api/init.rst

2011-05-04 Thread Steffen Daode Nurpmeso
New submission from Steffen Daode Nurpmeso : Yes, i really found a typo. I'll send two patches, one with the typo fixed, and one with the typo fixed and one for which i've :setlocal tw=80:{gq} -- assignee: docs@python components: Documentation messages: 135107 nosy: docs@pytho

[issue11997] One typo in Doc/c-api/init.rst

2011-05-04 Thread Steffen Daode Nurpmeso
Steffen Daode Nurpmeso added the comment: 11997.1.diff only corrects the typo, 11997.2.diff does also reformat. (Note that all of init.rst is hard to read on a 80 column terminal.) -- keywords: +patch Added file: http://bugs.python.org/file21880/11997.1.diff Added file: http

[issue11999] sporadic failure in test_mailbox on FreeBSD

2011-05-04 Thread Steffen Daode Nurpmeso
Steffen Daode Nurpmeso added the comment: I think this relates #6896. Maybe a two second resolution should be tried? -- keywords: +patch nosy: +sdaoden Added file: http://bugs.python.org/file21884/11999.1.diff ___ Python tracker <h

[issue11277] Crash with mmap and sparse files on Mac OS X

2011-05-04 Thread Steffen Daode Nurpmeso
Steffen Daode Nurpmeso added the comment: > error: [Errno 12] Cannot allocate memory @haypo: Well i told you i have no idea. These bots are 32 bit? I'll attach 11277-27.3.diff which does @skipUnless(not 32 bit). Note i'll test against >_4G - does this work (on 32 bit and in

[issue11277] Crash with mmap and sparse files on Mac OS X

2011-05-04 Thread Steffen Daode Nurpmeso
Steffen Daode Nurpmeso added the comment: @haypo: Oh. Not: if sys.maxsize > _4G: # (64 bits system) crc32() and adler32() stores the buffer size into an # int, the maximum filesize is INT_MAX (0x7FFF) filesize = 0x7FFF crc_res = 0x70941

[issue11935] MMDF/MBOX mailbox need utime

2011-05-05 Thread Steffen Daode Nurpmeso
Steffen Daode Nurpmeso added the comment: On Thu, 5 May 2011 03:52:29 +0200, R. David Murray wrote: > [..] the shell [..] I believe it just looks at the mtime/atime. /* check_mail () is useful for more than just checking mail. Since it has the paranoids dream ability of telling you w

[issue11935] MMDF/MBOX mailbox need utime

2011-05-05 Thread Steffen Daode Nurpmeso
Steffen Daode Nurpmeso added the comment: On Thu, 5 May 2011 03:52:29 +0200, R. David Murray wrote: > [..] the shell [..] I believe it just looks at the mtime/atime. Pretty good, huh? Mr. Mojo says: Prowd to be a part of this number. Successful hills are here to s

[issue11277] Crash with mmap and sparse files on Mac OS X

2011-05-05 Thread Steffen Daode Nurpmeso
Steffen Daode Nurpmeso added the comment: @haypo: trouble, trouble on the dev-list, as i've seen currently. Sorry, sorry. (Cannot subscribe, my DynIP's are often blacklisted ;) Of course my comments were completely wrong, as Ethan has pointed out correctly. This is all s**t. These

[issue12000] SSL certificate verification failed if no dNSName entry in subjectAltName

2011-05-05 Thread Steffen Daode Nurpmeso
Steffen Daode Nurpmeso added the comment: P.S.: if you're really right ('have those RFC's, but didn't read them yet), you could also open an issue for Mercurial at http://mercurial.selenic.com/bts - i think those guys do the very same. Thanks, Steffen! -

[issue11935] MMDF/MBOX mailbox need utime

2011-05-05 Thread Steffen Daode Nurpmeso
Steffen Daode Nurpmeso added the comment: On Thu, 5 May 2011 13:42:29 +0200, R. David Murray wrote: > what does mutt do in the case you are talking about? 16 -rwxr-s--- 1 steffen mail 14832 23 Jan 19:13 usr/bin/mutt_bitlock set bitlock_program="~/usr/bin/mutt_bitlock -p

[issue11935] MMDF/MBOX mailbox need utime

2011-05-05 Thread Steffen Daode Nurpmeso
Steffen Daode Nurpmeso added the comment: > The problem report in question was submitted by one of the > Debian maintainers. Yeah, a documentainer at least. I've used Debian (Woody i think that was 3.1). Actually great because Lehmanns, Heidelberg, Germany did not include the

[issue11935] MMDF/MBOX mailbox need utime

2011-05-05 Thread Steffen Daode Nurpmeso
Steffen Daode Nurpmeso added the comment: After half an hour of shallow inspection. mutt really modifies mailbox files in place (mbox_sync_mailbox()) after creating (all the new tail in) a temporary file. Then seek()/write()/truncate() etc.. It however has mutt_dotlock(1) and it does block

[issue11277] Crash with mmap and sparse files on Mac OS X

2011-05-05 Thread Steffen Daode Nurpmeso
Steffen Daode Nurpmeso added the comment: In fact i like my idea of using iterations. I have some time tomorrow, so if nobody complains until then, i write diffs for the tests of 3.x and 2.7 with these updates: - Two different target sizes: 1. 0x + x (7) 2. 0x7FFF + x (7

[issue7359] mailbox cannot modify mailboxes in system mail spool

2011-05-05 Thread Steffen Daode Nurpmeso
Steffen Daode Nurpmeso added the comment: Issue #11935 becomes gigantic and may even swamp this one over! -- nosy: +sdaoden ___ Python tracker <http://bugs.python.org/issue7

[issue11935] MMDF/MBOX mailbox need utime

2011-05-05 Thread Steffen Daode Nurpmeso
Steffen Daode Nurpmeso added the comment: On Thu, 5 May 2011 19:04:16 +0200, R. David Murray wrote: > "prepare new tail" means all of the text from the first modified > line to the end? (As opposed to "just the new mail"?) mailbox > does locking. I see no re

[issue11999] sporadic failure in test_mailbox on FreeBSD

2011-05-06 Thread Steffen Daode Nurpmeso
Steffen Daode Nurpmeso added the comment: On Fri, 6 May 2011 04:44:00 +0200, R. David Murray wrote: > [.] the mtime only has a resolution of one second. You always say that! But i'm pretty sure from somewhen long ago that there are filesystems which have a two second time resolut

[issue11999] sporadic failure in test_mailbox on FreeBSD

2011-05-06 Thread Steffen Daode Nurpmeso
Steffen Daode Nurpmeso added the comment: > I also added an additional delta in case the file system clock > is skewing relative to the system clock. In fact this idea could even be made public e.g. like this class ClockDrifter: def add_snapshot(self, exttime, loctim

[issue11999] sporadic failure in test_mailbox on FreeBSD

2011-05-06 Thread Steffen Daode Nurpmeso
Steffen Daode Nurpmeso added the comment: I like all kind of single-screw solutions. And i feel a bit uncomfortable with your usual tightness. (0.1 seconds - i mean, come on) -- ___ Python tracker <http://bugs.python.org/issue11

[issue11999] sporadic failure in test_mailbox on FreeBSD

2011-05-06 Thread Steffen Daode Nurpmeso
Steffen Daode Nurpmeso added the comment: > how many people are going to use maildir on FAT? Dunno. But it's maybe the lowest common denominator of mountable readwrite filesystems. Except for that MacBook i always had a shared FAT partition on my pri

[issue11935] MMDF/MBOX mailbox need utime

2011-05-06 Thread Steffen Daode Nurpmeso
Steffen Daode Nurpmeso added the comment: @david: note i got stuck on updating my patch for mailbox.py and switched to do test_mmap.py instead, so that i don't know wether i will be able to finish it today. Is it really true that mailbox.py even writes mailboxes without locking in case

[issue11999] sporadic failure in test_mailbox on FreeBSD

2011-05-06 Thread Steffen Daode Nurpmeso
Steffen Daode Nurpmeso added the comment: I agree that it's fun to see pieces of code interacting like gears in a transmission, but often it gets ugly due to the noise from the outside which requires to add ugly fuzziness or honour stupid design decisions. Maybe an environment variable

[issue11277] Crash with mmap and sparse files on Mac OS X

2011-05-06 Thread Steffen Daode Nurpmeso
Steffen Daode Nurpmeso added the comment: On Fri, 6 May 2011 02:54:07 +0200, Nadeem Vawda wrote: > I think so. [.] > it turns out that the OS X sparsefile crash is also covered by > LargeMmapTests.test_large_offset() in test_mmap [!!!]. [.] So i followed your suggestion and d

[issue11277] Crash with mmap and sparse files on Mac OS X

2011-05-06 Thread Steffen Daode Nurpmeso
Changes by Steffen Daode Nurpmeso : Removed file: http://bugs.python.org/file21869/11277-27.2.diff ___ Python tracker <http://bugs.python.org/issue11277> ___ ___ Pytho

[issue11277] Crash with mmap and sparse files on Mac OS X

2011-05-06 Thread Steffen Daode Nurpmeso
Changes by Steffen Daode Nurpmeso : Removed file: http://bugs.python.org/file21885/11277-27.3.diff ___ Python tracker <http://bugs.python.org/issue11277> ___ ___ Pytho

[issue11277] Crash with mmap and sparse files on Mac OS X

2011-05-07 Thread Steffen Daode Nurpmeso
Steffen Daode Nurpmeso added the comment: @Nadeem: note that the committed versions of the tests would not show up the Mac OS X mmap() bug AFAIK, because there is an intermediate .close() of the file to be mmapped. The OS X bug is that the VMS/VFS interaction fails to provide a valid memory

[issue11277] Crash with mmap and sparse files on Mac OS X

2011-05-07 Thread Steffen Daode Nurpmeso
Steffen Daode Nurpmeso added the comment: (Of course this may also be intentional, say. But then i would vote against it :), because it's better the tests bring out errors than end-user apps.) -- ___ Python tracker <http://bugs.py

[issue11999] sporadic failure in test_mailbox on FreeBSD

2011-05-07 Thread Steffen Daode Nurpmeso
Steffen Daode Nurpmeso added the comment: That's really a good one. (In my eyes.) -- title: sporadic failure in test_mailbox -> sporadic failure in test_mailbox on FreeBSD ___ Python tracker <http://bugs.python.org

[issue11877] Change os.fsync() to support physical backing store syncs

2011-05-07 Thread Steffen Daode Nurpmeso
Steffen Daode Nurpmeso added the comment: On Sat, 7 May 2011 14:20:51 +0200, Charles-François Natali wrote: > I really can't see a good reason for using it here (and > anywhere, see http://c-faq.com/decl/auto.html). You're right. > Why are you using the "auto" st

[issue11877] Change os.fsync() to support physical backing store syncs

2011-05-07 Thread Steffen Daode Nurpmeso
Steffen Daode Nurpmeso added the comment: On Sat, 7 May 2011 14:20:51 +0200, Charles-François Natali wrote: > # ifdef __APPLE__ > res = fcntl(fd, F_FULLFSYNC); > # endif > # ifdef __NetBSD__ > res = fsync_range(fd, FFILESYNC|FDISKSYNC, 0, 0); > # endif >

[issue11999] sporadic failure in test_mailbox

2011-05-07 Thread Steffen Daode Nurpmeso
Steffen Daode Nurpmeso added the comment: This tracker - sorry! (I surely will try to write and offer a patch for the tracker, but first i need to understand that mailbox.py jungle for #11935, next week.) -- title: sporadic failure in test_mailbox on FreeBSD -> sporadic failure

[issue11877] Change os.fsync() to support physical backing store syncs

2011-05-07 Thread Steffen Daode Nurpmeso
Steffen Daode Nurpmeso added the comment: 11877.5.diff incorporates all changes suggested by Charles-François except for the 'auto' keyword, which is extremely important and which would need to be invented if it would not yet exist. I'm dropping the old stuff. And i think th

[issue11877] Change os.fsync() to support physical backing store syncs

2011-05-07 Thread Steffen Daode Nurpmeso
Changes by Steffen Daode Nurpmeso : Removed file: http://bugs.python.org/file21749/11877.3.diff ___ Python tracker <http://bugs.python.org/issue11877> ___ ___ Python-bug

[issue11877] Change os.fsync() to support physical backing store syncs

2011-05-07 Thread Steffen Daode Nurpmeso
Changes by Steffen Daode Nurpmeso : Removed file: http://bugs.python.org/file21771/11877.4.diff ___ Python tracker <http://bugs.python.org/issue11877> ___ ___ Python-bug

[issue11877] Change os.fsync() to support physical backing store syncs

2011-05-09 Thread Steffen Daode Nurpmeso
Steffen Daode Nurpmeso added the comment: Ronald Oussoren wrote (2011-05-08 10:33+0200): > Steffen, I don't understand your comment about "auto". Declaring > variables as "auto" is not necessary in C code and not used > anywhere else in Python's sourc

[issue11877] Change os.fsync() to support physical backing store syncs

2011-05-10 Thread Steffen Daode Nurpmeso
Steffen Daode Nurpmeso added the comment: I don't agree with you and i don't believe it is implemented like that. But it seems i am the only one on this issue who sees it like that. Thus i apply 11877.6.diff. > Declaring variables as "auto" is not necessary in C code

[issue11935] MMDF/MBOX mailbox need utime

2011-05-11 Thread Steffen Daode Nurpmeso
Steffen Daode Nurpmeso added the comment: For the record: On Mac OS X 10.6.7, ,HFS, case sensitive` updates st_atime by itself *once only*. It does so ~0.75 seconds after os.utime() (+) was called. A time.sleep(0.8) can be used to detect this automatic update reliably (about 50 tests with

[issue11877] Change os.fsync() to support physical backing store syncs

2011-05-11 Thread Steffen Daode Nurpmeso
Steffen Daode Nurpmeso added the comment: Ouch, ouch, ouch!! I'll have to send 11877.7.diff which extends 11877.6.diff. This is necessary because using fcntl(2) with F_FULLFSYNC may fail with ENOTTY (inapprobiate ioctl for device) in situations where a normal fsync(2) succeeds

[issue11877] Change os.fsync() to support physical backing store syncs

2011-05-11 Thread Steffen Daode Nurpmeso
Changes by Steffen Daode Nurpmeso : Removed file: http://bugs.python.org/file21924/11877.5.diff ___ Python tracker <http://bugs.python.org/issue11877> ___ ___ Python-bug

[issue11877] Change os.fsync() to support physical backing store syncs

2011-05-11 Thread Steffen Daode Nurpmeso
Changes by Steffen Daode Nurpmeso : Removed file: http://bugs.python.org/file21953/11877.6.diff ___ Python tracker <http://bugs.python.org/issue11877> ___ ___ Python-bug

[issue11877] Change os.fsync() to support physical backing store syncs

2011-05-12 Thread Steffen Daode Nurpmeso
Steffen Daode Nurpmeso added the comment: Just adding more notes on that by reactivating one of haypo's links from #8604. (And: maybe some Linux documentation should be updated?) From Theodore Ts'o, http://www.linuxfoundation.org/news-media/blogs/browse/2009/03/don’t-fear-fsync:

[issue12060] Python doesn't support real time signals

2011-05-12 Thread Steffen Daode Nurpmeso
Steffen Daode Nurpmeso added the comment: Dunno. > The patch is not completely safe. Yeah it will not work without atomic ops. Unfortunately the C standart seems to go into a direction noone understands - as if a atomic_compare_and_swap() would not suffice! Do you know any machine langu

[issue12060] Python doesn't support real time signals

2011-05-12 Thread Steffen Daode Nurpmeso
Steffen Daode Nurpmeso added the comment: > On my Linux box, Python 3.3 says that signal.NSIG is equal to 65 > which looks correct. On FreeBSD NSIG only counts "old" signals (32, one 32 bit mask), SIGRTMIN is 65 and SIGRTMAX is 126. Our internal old signal.h states

[issue11877] Change os.fsync() to support physical backing store syncs

2011-05-12 Thread Steffen Daode Nurpmeso
Steffen Daode Nurpmeso added the comment: [.] > OSError: [Errno 22] Invalid argument Sorry, i didn't know that. Mac OS X (2.5 and 2.6 Apple shipped): 21:43 ~/tmp $ python2.5 -c 'import os; os.fsync(1)'; echo $? 0 21:43 ~/tmp $ python2.6 -c 'import os; os.fsync(1)&

[issue11877] Change os.fsync() to support physical backing store syncs

2011-05-12 Thread Steffen Daode Nurpmeso
Steffen Daode Nurpmeso added the comment: > So I think you should stick with the previous version (well, if the > full sync fails on other FDs, then it's another story, but in that > case it should just be dropped altogether if it's not reliable...). Strong stuff. *This* i

[issue11877] Change os.fsync() to support physical backing store syncs

2011-05-12 Thread Steffen Daode Nurpmeso
Changes by Steffen Daode Nurpmeso : Removed file: http://bugs.python.org/file21973/11877.7.diff ___ Python tracker <http://bugs.python.org/issue11877> ___ ___ Python-bug

[issue6721] Locks in python standard library should be sanitized on fork

2011-05-12 Thread Steffen Daode Nurpmeso
Steffen Daode Nurpmeso added the comment: @Nir Aides: *thanks* for this link: http://groups.google.com/group/comp.programming.threads/msg/3a43122820983fde You made my day! -- nosy: +sdaoden ___ Python tracker <http://bugs.python.org/issue6

[issue6721] Locks in python standard library should be sanitized on fork

2011-05-13 Thread Steffen Daode Nurpmeso
Steffen Daode Nurpmeso added the comment: @ Charles-François Natali wrote (2011-05-13 13:24+0200): > I happily posted a reinit patch I must say in advance that we have implemented our own thread support 2003-2005 and i'm thus lucky not to need to use anything else ever since. So.

[issue11877] Change os.fsync() to support physical backing store syncs

2011-05-15 Thread Steffen Daode Nurpmeso
Steffen Daode Nurpmeso added the comment: > Finally, depending on the workload, it could have a significant > performance impact. Oh yes (first replaces os.fsync() with an in-python wrapper): 18:12 ~/tmp $ ll mail ls: mail: No such file or directory 18:12 ~/tmp $ ll X-MAIL 312 -rw-r--

[issue6721] Locks in python standard library should be sanitized on fork

2011-05-15 Thread Steffen Daode Nurpmeso
Steffen Daode Nurpmeso added the comment: @ Charles-François Natali wrote (2011-05-15 01:14+0200): > So if we really wanted to be safe, the only solution would be to > forbid fork() in a multi-threaded program. > Since it's not really a reasonable option But now - why this? T

<    1   2   3   4   >