[issue15993] Windows: 3.3.0-rc2.msi: test_buffer fails

Steve Dower added the comment: Eh, why bother. I don't remember if the fix is in for 3.5.0b3, but I'll vouch that the compiler build with the fix does exist and will be used for 3.5, so this should just be closed (again). -- stage: -> resolved status: open -> closed _

[issue15993] Windows: 3.3.0-rc2.msi: test_buffer fails

Larry Hastings added the comment: So, the purpose in marking this as a "release blocker" is so that we can hold up the release while we wait for Microsoft to release a new compiler? If our approach to fixing this is to get the compiler fixed, I can live with marking this as "critical", but no

[issue15993] Windows: 3.3.0-rc2.msi: test_buffer fails

Mark Lawrence added the comment: Is this now fixed in VS? I don't believe I can test myself as I've only got express/community editions. -- components: +Windows nosy: +BreamoreBoy ___ Python tracker _

[issue15993] Windows: 3.3.0-rc2.msi: test_buffer fails

Changes by Stefan Krah : -- nosy: -skrah ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.o

[issue15993] Windows: 3.3.0-rc2.msi: test_buffer fails

Changes by STINNER Victor : -- nosy: +haypo ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python

[issue15993] Windows: 3.3.0-rc2.msi: test_buffer fails

Changes by STINNER Victor : -- nosy: +tim.golden, zach.ware -haypo ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscrib

[issue15993] Windows: 3.3.0-rc2.msi: test_buffer fails

Martin v. Löwis added the comment: Thanks a lot for this investigation; I'm glad you are working on this. -- ___ Python tracker ___ __

[issue15993] Windows: 3.3.0-rc2.msi: test_buffer fails

Steve Dower added the comment: This has been confirmed as a bug in VC14 (and earlier) and there'll be a fix going in soon. For those interested, here's a brief rundown of the root cause: * the switch in build_filter_spec() switches on a 64-bit value * one case is 0x4001 and the res

[issue15993] Windows: 3.3.0-rc2.msi: test_buffer fails

Steve Dower added the comment: > I'd be fine to reconsider if a previously-demonstrated bug is now > demonstrated-fixed. However, if the actual bug persists, optimization > should be disabled for all code, not just for the code that allows to > demonstrate the bug. I'm okay with that. I thou

[issue15993] Windows: 3.3.0-rc2.msi: test_buffer fails

Stefan Krah added the comment: > The two issues were unrelated - the 'invalid filter ID' > (4611686018427387905 == 0x4000_0001) is the correct > value but the wrong branch in the switch was taken, leading > to the error message. Unfortunately I don't have a Visual Studio setup right now.

[issue15993] Windows: 3.3.0-rc2.msi: test_buffer fails

Changes by Stefan Krah : Added file: http://bugs.python.org/file35681/ull_vctest.diff ___ Python tracker ___ ___ Python-bugs-list mailing list

[issue15993] Windows: 3.3.0-rc2.msi: test_buffer fails

Martin v. Löwis added the comment: I'd be fine to reconsider if a previously-demonstrated bug is now demonstrated-fixed. However, if the actual bug persists, optimization should be disabled for all code, not just for the code that allows to demonstrate the bug. This principle should indeed bee

[issue15993] Windows: 3.3.0-rc2.msi: test_buffer fails

Steve Dower added the comment: > Isn't PyLong_FromUnsignedLongLong() still involved through spec_add_field()? The two issues were unrelated - the 'invalid filter ID' (4611686018427387905 == 0x4000_0001) is the correct value but the wrong branch in the switch was taken, leading to the e

[issue15993] Windows: 3.3.0-rc2.msi: test_buffer fails

Martin v. Löwis added the comment: Please don't. If the compiler is demonstrated to generate bad code in one case, we should *not* exclude that code from optimization, but not use optimization at all. How many other places will there be which also cause bad code being generated that just happe

[issue15993] Windows: 3.3.0-rc2.msi: test_buffer fails

Stefan Krah added the comment: Isn't PyLong_FromUnsignedLongLong() still involved through spec_add_field()? -- ___ Python tracker ___

[issue15993] Windows: 3.3.0-rc2.msi: test_buffer fails

Steve Dower added the comment: It's actually bad code generation for the switch statement in build_filter_spec() in _lzmamodule.c. I've filed a bug, so hopefully it will be fixed, but if not then it should be easy enough to exclude that function (or even the whole module - _lzmamodule.c doesn

[issue15993] Windows: 3.3.0-rc2.msi: test_buffer fails

Steve Dower added the comment: test_memoryview_assign seems to be okay, but the two test_lzma tests still fail with the same message. Both pass without PGO. I'll get in touch with the PGO team and try and get it fixed. I haven't checked, but it looks consistent with Stefan's analysis of the di

[issue15993] Windows: 3.3.0-rc2.msi: test_buffer fails

Stefan Krah added the comment: "I ran a quick test with profile-guided optimization (PGO, pronounced "pogo"), which has supposedly been improved since VC9, and +saw a very unscientific 20% speed improvement on pybench.py and 10% size reduction in python35.dll. I'm not sure what we used +to get

[issue15993] Windows: 3.3.0-rc2.msi: test_buffer fails

Stefan Krah added the comment: Hmm, the bug is in the 32-bit build. The 64-bit build is fine. Or do you mean MMX registers? -- ___ Python tracker ___

[issue15993] Windows: 3.3.0-rc2.msi: test_buffer fails

Martin v. Löwis added the comment: It's interesting that the compiler uses ecx:edx to represent a __int64 quantity when 64-bit registers are available... -- ___ Python tracker _

[issue15993] Windows: 3.3.0-rc2.msi: test_buffer fails

Stefan Krah added the comment: > Given all the problems, I'll stop using PGO on all branches and > as the compiler apparently generates bad code. That is probably the best solution. The problem in memoryview.c:pack_single() is that Visual Studio optimizes the memcpy() to mov instructions, but

[issue15993] Windows: 3.3.0-rc2.msi: test_buffer fails

Changes by Martin v. Löwis : -- resolution: -> fixed status: open -> closed ___ Python tracker ___ ___ Python-bugs-list mailing list

[issue15993] Windows: 3.3.0-rc2.msi: test_buffer fails

Martin v. Löwis added the comment: Unfortunately (?) I now cannot produce this myself, anymore, either. Given all the problems, I'll stop using PGO on all branches and targets, as the compiler apparently generates bad code. Anybody curious to investigate the issue further who is able to reprod

[issue15993] Windows: 3.3.0-rc2.msi: test_buffer fails

Stefan Krah added the comment: Sorry, the line numbers are messed up. They should be: Objects/memoryobject.c:1569 Objects/memoryobject.c:1776 -- ___ Python tracker ___ _

[issue15993] Windows: 3.3.0-rc2.msi: test_buffer fails

Stefan Krah added the comment: System: Windows 7 64-bit Build (unpatched): PCBuild\Win32-pgo\python.exe, trained with profile.bat In the unpatched version, I stepped through this test case in the debugger: import array a = array.array('Q', [1]) m = memoryview(a) m[0] = 1 At Objects/memoryobj

[issue15993] Windows: 3.3.0-rc2.msi: test_buffer fails

Stefan Krah added the comment: It's a bit late here already, but unless I'm missing something I think this is an optimizer bug. I'm attaching a workaround for memoryview.c and _lzmamodule.c. -- keywords: +patch Added file: http://bugs.python.org/file27253/issue15993.diff ___

[issue15993] Windows: 3.3.0-rc2.msi: test_buffer fails

Martin v. Löwis added the comment: I'm using Ultimate, but I think Professional should provide you with all required tools. -- ___ Python tracker ___ ___

[issue15993] Windows: 3.3.0-rc2.msi: test_buffer fails

Stefan Krah added the comment: Martin v. Löwis wrote: > For the record, the released binary is not just a PGO build, but has > been trained with the attached training script. Thanks. Now I can reproduce the issue with a source build. -- ___ Python t

[issue15993] Windows: 3.3.0-rc2.msi: test_buffer fails

Stefan Krah added the comment: STINNER Victor wrote: > Which version should I try? Ultimate? Premium? Professional? Try Ultimate, it's AFAIK the only version these days that supports PGO. -- ___ Python tracker __

[issue15993] Windows: 3.3.0-rc2.msi: test_buffer fails

STINNER Victor added the comment: > You need the professional version of Visual Studio 10. > The express version doesn't support 64 bits. I only have the > Express version. Ah yes, I now remember my issue with VS10 Express: when I set the project to 64 bits, I get such error popup: http://www.h

[issue15993] Windows: 3.3.0-rc2.msi: test_buffer fails

Changes by Martin v. Löwis : Added file: http://bugs.python.org/file27252/profiletests.txt ___ Python tracker ___ ___ Python-bugs-list mailing

[issue15993] Windows: 3.3.0-rc2.msi: test_buffer fails

Martin v. Löwis added the comment: For the record, the released binary is not just a PGO build, but has been trained with the attached training script. -- Added file: http://bugs.python.org/file27251/profile.bat ___ Python tracker

[issue15993] Windows: 3.3.0-rc2.msi: test_buffer fails

Martin v. Löwis added the comment: Declaring this a release blocker is technically difficult. If it is a release blocker, further releases cannot be done until it is resolved. Since it is an issue with the binary only, the only possible way to resolve this is with a release. So declaring this

[issue15993] Windows: 3.3.0-rc2.msi: test_buffer fails

STINNER Victor added the comment: I fail to reproduce the issue on Windows 7 (Version 6.0.1, number 7601, Service Pack 1): --- Microsoft Windows [version 6.1.7601] Copyright (c) 2009 Microsoft Corporation. Tous droits réservés. C:\Users\haypo>cd \python33 C:\Python33>python.exe Python 3.3.0rc2

[issue15993] Windows: 3.3.0-rc2.msi: test_buffer fails

STINNER Victor added the comment: Issue #15995 was marked as a duplicate of this issue. Copy of its initial message (msg170853): This is similar to #15993: With the installed Python from the rc2-msi test_lzma fails. I cannot reproduce the failure with python.exe (PGO) compiled from source: ==

[issue15993] Windows: 3.3.0-rc2.msi: test_buffer fails

Changes by Stefan Krah : -- nosy: +nadeem.vawda ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.pyt

[issue15993] Windows: 3.3.0-rc2.msi: test_buffer fails

Stefan Krah added the comment: The high and low words of the 64-bit value are switched: >>> a = array.array('Q', [1]) >>> m = memoryview(a) >>> m[0]= 2**32+5 >>> m[0] 21474836481 >>> struct.unpack_from('8s', m, 0) (b'\x01\x00\x00\x00\x05\x00\x00\x00',) Can anyone reproduce this in a source bui

[issue15993] Windows: 3.3.0-rc2.msi: test_buffer fails

Stefan Krah added the comment: Both lzma and memoryview use PyLong_AsUnsignedLongLong() in the affected code paths. I get this with the msi installed python.exe: >>> import array >>> a = array.array('Q', [1,2,3,4]) >>> m = memoryview(a) >>> m[0] = 4 >>> m[0] 17179869184 >>> And the correct res

[issue15993] Windows: 3.3.0-rc2.msi: test_buffer fails

Changes by STINNER Victor : -- nosy: +haypo ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.

[issue15993] Windows: 3.3.0-rc2.msi: test_buffer fails

New submission from Stefan Krah: I've installed 3.3.0-rc2 on Windows-7 64-bit using the msi installer. I'm getting these failures in test_buffer, but I can *not* reproduce them when I build Win-32/pgo/python.exe from source: ==