[issue7632] dtoa.c: oversize b in quorem, and a menagerie of other bugs

Mark Dickinson added the comment: Thanks, Stefan. Applied in r77589 (trunk), r77590 (py3k), r77591 (release31-maint) with one small change: I moved the freelist and p5s declarations inside the #ifndef Py_USING_MEMORY_DEBUGGER conditionals. The leak itself was fixed in revisions r77578 throu

[issue7632] dtoa.c: oversize b in quorem, and a menagerie of other bugs

Stefan Krah added the comment: With the latest dtoa.c, your non-caching pow5mult and a quick hack for Balloc and Bfree I get zero (dtoa.c related) Valgrind errors. So the attached memory_debugger.diff is pretty much all what's needed for Valgrind. -- Added file: http://bugs.python.or

[issue7632] dtoa.c: oversize b in quorem, and a menagerie of other bugs

Mark Dickinson added the comment: Stefan, I'm not particularly familiar with Valgrind: can you tell me what would need to be done? Is a non-caching version of pow5mult all that's required? Here's the patch that I'm using to detect leaks at the moment. (It includes a slow pow5mult version.

[issue7632] dtoa.c: oversize b in quorem, and a menagerie of other bugs

Stefan Krah added the comment: Mark, thanks for the explanation! - You can generate suppressions for the Misc/valgrind-python.supp file, but you have to know exactly which errors can be ignored. Going through the Valgrind output again, it looks like most of it is about what you already menti

[issue7632] dtoa.c: oversize b in quorem, and a menagerie of other bugs

Mark Dickinson added the comment: Upgrading to release blocker again: the memory leak should be fixed for 2.7 (and more immediately, for 3.1.2). -- priority: normal -> release blocker ___ Python tracker _

[issue7632] dtoa.c: oversize b in quorem, and a menagerie of other bugs

Mark Dickinson added the comment: Stefan, thanks for that! I'm not entirely sure how to make use of it, though. Is there a way to tell Valgrind that some leaks are expected? The main problem with leak detection is that dtoa.c deliberately keeps hold of any malloc'ed chunks less than a certa

[issue7632] dtoa.c: oversize b in quorem, and a menagerie of other bugs

Stefan Krah added the comment: This is what Valgrind complains about: ==4750== 3,456 (1,440 direct, 2,016 indirect) bytes in 30 blocks are definitely lost in loss record 3,302 of 3,430 ==4750==at 0x4C2412C: malloc (vg_replace_malloc.c:195) ==4750==by 0x41B7B5: PyMem_Malloc (object.c:17

[issue7632] dtoa.c: oversize b in quorem, and a menagerie of other bugs

Mark Dickinson added the comment: Okay, so there's a memory leak for overflowing values: if an overflow is detected in the main correction loop of _Py_dg_strtod, then 'references' to bd0, bd, bb, bs and delta aren't released. There may be other leaks; I'm trying to come up with a good way t

[issue7632] dtoa.c: oversize b in quorem, and a menagerie of other bugs

Mark Dickinson added the comment: One of the buildbots just produced a MemoryError from test_strtod: http://www.python.org/dev/buildbot/all/builders/i386%20Ubuntu%203.x/builds/411 It looks as though there's a memory leak somewhere in dtoa.c. It's a bit difficult to tell, though, since the me

[issue7632] dtoa.c: oversize b in quorem, and a menagerie of other bugs

Mark Dickinson added the comment: Fixes merged to py3k and release31-maint in r77535 and r77537. -- priority: release blocker -> normal ___ Python tracker ___ ___

[issue7632] dtoa.c: oversize b in quorem, and a menagerie of other bugs

Mark Dickinson added the comment: I've applied a minimal fix for bugs 5 and 7 in r77530 (trunk). (I wasn't able to produce any strings that trigger bug 7, so it may not technically be a bug.) I'm continuing to review, comment, and clean up the remainder of the _Py_dg_strtod. -- ___

[issue7632] dtoa.c: oversize b in quorem, and a menagerie of other bugs

Mark Dickinson added the comment: Applied the bug 8 patch in r77519 (thanks Eric for reviewing!). For safety, I'll leave this as a release blocker until fixes have been merged to py3k and release31-maint. I've uploaded a fix for bugs 5 and 7 to Rietveld: http://codereview.appspot.com/186182

[issue7632] dtoa.c: oversize b in quorem, and a menagerie of other bugs

Eric Smith added the comment: I have a few minor comments posted on Rietveld, but nothing that would keep you from checking this in. -- ___ Python tracker ___ __

[issue7632] dtoa.c: oversize b in quorem, and a menagerie of other bugs

Eric Smith added the comment: It looks correct to me, assuming this comment is correct: /* scan back until we hit a nonzero digit. significant digit 'i' is s0[i] if i < nd0, s0[i+1] if i >= nd0. */ I didn't verify the comment itself. -- _

[issue7632] dtoa.c: oversize b in quorem, and a menagerie of other bugs

Mark Dickinson added the comment: issue7632_bug8.patch uploaded to Rietveld: http://codereview.appspot.com/186168 -- ___ Python tracker ___ _

[issue7632] dtoa.c: oversize b in quorem, and a menagerie of other bugs

Mark Dickinson added the comment: Here's a patch for the release blocker. Eric, would you be interested in double checking the logic for this patch? Tim: No, I have to admit I didn't forsee quite this number of bugs. :) -- title: dtoa.c: oversize b in quorem -> dtoa.c: oversize b i

[issue7632] dtoa.c: oversize b in quorem

Mark Dickinson added the comment: Upgrading to release blocker. It'll almost certainly be fixed before the weekend is out. (And I will, of course, report it upstream.) -- priority: high -> release blocker ___ Python tracker

[issue7632] dtoa.c: oversize b in quorem

Tim Peters added the comment: Mark, I agree that last one should be a release blocker -- it's truly dreadful. BTW, did you guess in advance just how many bugs there could be in this kind of code? I did ;-) -- ___ Python tracker

[issue7632] dtoa.c: oversize b in quorem

Mark Dickinson added the comment: I was considering downgrading this to 'normal'. Then I found Bug 8, and it's a biggie: >>> 10.900012345678912345678912345 10.0 Now I'm thinking it should be upgraded to release blocker instead. The cause is in the _Py_strtod block that starts: '

[issue7632] dtoa.c: oversize b in quorem

Mark Dickinson added the comment: Fixes and tests so far merged to py3k in r77494, release31-maint in r77496. -- ___ Python tracker ___ __

[issue7632] dtoa.c: oversize b in quorem

Mark Dickinson added the comment: Tests committed in r77493. -- ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe:

[issue7632] dtoa.c: oversize b in quorem

Mark Dickinson added the comment: Bug 4 fixed in r77492. This just leaves bugs 5 and 7; I have a fix for these in the works. -- ___ Python tracker ___

[issue7632] dtoa.c: oversize b in quorem

Mark Dickinson added the comment: Bug 6 is indeed a bug: an example incorrectly-rounded string is: '104308485241983990666713401708072175773165034278685682646111762292409330928739751702404658197872319129036519947435319418387839758990478549475866730759458448959810120243879921356170645

[issue7632] dtoa.c: oversize b in quorem

Mark Dickinson added the comment: Just so I don't forget, there are a couple more places in the dtoa.c that look suspicious and need to be checked; I haven't tried to generate failures for them yet. Since we're up to bug 5, I'll number these 6 and 7: (6) at the end of bigcomp, when applying

[issue7632] dtoa.c: oversize b in quorem

Mark Dickinson added the comment: Merged fixes so far, and a couple of other cleanups, to py3k in r77452, and release31-maint in r77453. -- ___ Python tracker ___ __

[issue7632] dtoa.c: oversize b in quorem

Mark Dickinson added the comment: Second bug fixed in r77451 (trunk), using a fix from David Gay, modified slightly for correctness. -- ___ Python tracker ___ __

[issue7632] dtoa.c: oversize b in quorem

Mark Dickinson added the comment: Fixed the crash that Stefan originally reported in r77450. That revision also removes the 'possibly incorrect code in bigcomp that probably never actually gets called'. -- ___ Python tracker

[issue7632] dtoa.c: oversize b in quorem

Mark Dickinson added the comment: Progress report: I've had a response, and fix, from David Gay for the first 2 bugs (Stefan's original bug and the incorrect subnormal result); I'm still arguing with him about a 3rd one (not reported here; there's some possibly incorrect code in bigcomp tha

[issue7632] dtoa.c: oversize b in quorem

Mark Dickinson added the comment: > Showing once again that a proof of FP code correctness is about as > compelling as a proof of God's ontological status ;-) Clearly we need a 1000-page Isabelle/HOL-style machine-checked formal proof, rather than a ten-page TeX proof. Any takers? All of the

[issue7632] dtoa.c: oversize b in quorem

Tim Peters added the comment: Showing once again that a proof of FP code correctness is about as compelling as a proof of God's ontological status ;-) Still, have to express surprised admiration for 4487665465554760717039532578546e-47! That one's not even close to being a "h

[issue7632] dtoa.c: oversize b in quorem

Mark Dickinson added the comment: One more incorrectly rounded result, this time for a normal number: AssertionError: Incorrectly rounded str->float conversion for 4487665465554760717039532578546e-47: expected 0x1.0p+0, got 0x1.fp-1 -- ___

[issue7632] dtoa.c: oversize b in quorem

Mark Dickinson added the comment: Here's the (rather crude) testing program that turned up these errors. -- Added file: http://bugs.python.org/file15797/test_dtoa.py ___ Python tracker _

[issue7632] dtoa.c: oversize b in quorem

Mark Dickinson added the comment: Second patch, adding a fix for the rounding bug to the first patch. -- Added file: http://bugs.python.org/file15796/issue7632_v2.patch ___ Python tracker __

[issue7632] dtoa.c: oversize b in quorem

Mark Dickinson added the comment: Okay, I think I've found the cause of the second rounding bug above: at the end of the bigcomp function there's a correction block that looks like ... else if (dd < 0) { if (!dsign) /* does not happen for round-near */ retlow1:

[issue7632] dtoa.c: oversize b in quorem

Mark Dickinson added the comment: Randomised testing quickly turned up another troublesome string for str -> float conversion: s = "94393431193180696942841837085033647913224148539854e-358" This one's actually giving incorrectly rounded results (the horror!) in a non-debug build of trunk, and

[issue7632] dtoa.c: oversize b in quorem

Mark Dickinson added the comment: Here's a patch that seems to fix the problem; I'll wait a while to see if I get a response from David Gay before applying this. Also, if we've got to the stage of modifying the algorithmic part of the original dtoa.c, we should really make sure that we've go

[issue7632] dtoa.c: oversize b in quorem

Mark Dickinson added the comment: The bug is present in the current version of dtoa.c from http://www.netlib.org/fp, so I'll report it upstream. As far as I can tell, though, it's benign, in the sense that if the check is disabled then nothing bad happens, and the correct result is eventuall

[issue7632] dtoa.c: oversize b in quorem

Eric Smith added the comment: I'm testing on a Fedora Core 6 i386 box and an Intel Mac 32-bit 10.5 box. I only see this on debug builds. I've tested trunk, py3k, release31-maint, and release26-maint (just for giggles). The error shows up in debug builds of trunk, py3k, and release31-maint on

[issue7632] dtoa.c: oversize b in quorem

Mark Dickinson added the comment: I can reproduce this on OS X 10.6 (64-bit), both in py3k and trunk debug builds. In non-debug builds it appears to return the correct result (0.0), so the oversize b appears to have no ill effects. So this may just be an overeager assert; it may be a sympt

[issue7632] dtoa.c: oversize b in quorem

Changes by Mark Dickinson : -- components: +Interpreter Core stage: -> needs patch type: -> crash ___ Python tracker ___ ___ Python-b

[issue7632] dtoa.c: oversize b in quorem

Mark Dickinson added the comment: Nice catch! I'll take a look. We should find out whether this is something that happens with Gay's original code, or whether it was introduced in the process of adapting that code for Python. -- assignee: -> mark.dickinson priority: -> critical ve

[issue7632] dtoa.c: oversize b in quorem

New submission from Stefan Krah : In a debug build: Python 3.2a0 (py3k:76671M, Dec 22 2009, 19:41:08) [GCC 4.1.3 20080623 (prerelease) (Ubuntu 4.1.2-23ubuntu3)] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> s = "2183167012312112312312.23538020374420446192e