[issue45435] delete misleading faq entry about atomic operations
Jeff Allen added the comment: I'm interested in Thomas' reasons, but here are some of mine (as far as I understand things): 1. It is specific to one interpreter implemented in C, equipped with a GIL, and on certain assumptions about the byte code interpreter and the implementation of built-ins, that may not hold long-term. 2. In x = L[i], the index and assignment are distinct actions (in today's byte code), allowing L or i to change before x is assigned. This applies to multiple other of the examples. 3. A compiler (even a CPU) is free to re-order operations and cache values in unguessable ways, on the assumption of a single thread. 4. Code written on these principals is fragile. It only takes the replacement of a built-in with sub-class redefining __getitem__ (to support some worthy aim elsewhere in the code) to invalidate it. 5. sort() is not atomic if an element is of a type that overrides comparison in Python. (Nor is modifying a dictionary if __hash__ or __eq__ are redefined.) If you want retain the question, with a better answer, the last sentence is good: "When in doubt, use a mutex!", accompanied by "Always be in doubt." -- nosy: +jeff.allen ___ Python tracker <https://bugs.python.org/issue45435> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue45435] delete misleading faq entry about atomic operations
Jeff Allen added the comment: Thomas wrote: > it's as part of this discussion in > https://mail.python.org/archives/list/python-...@python.org/thread/ABR2L6BENNA6UPSPKV474HCS4LWT26GY/#IAOCDDCJ653NBED3G2J2YBWD7HHPFHT6 > and others in #python-dev That's where I noticed it, but it seemed the wrong place to explore this way. Steven is right, I'm over-stating the case. And although valid that this is CPython specific, it's well sign-posted and I'm just being thin-skinned. Serhiy writes: > sort() is atomic, even if GIL is released during executing custom __lt__. It > is guaranteed that no operations on the list in other threads can affect the > result of sort(). The strategy noted here: https://github.com/python/cpython/blob/2d21612f0dd84bf6d0ce35bcfcc9f0e1a41c202d/Objects/listobject.c#L2261-L2265 does guarantee that, which I hadn't noticed. What if during the release of the GIL, another thread appends to L? In my simple experiment I get a ValueError and the modifications are lost. I think that is not thread-safe. Serhiy also writes: > I do not understand what non-atomic you see in x = L[i]. The value of x is > determined by values of L and i at the start of the operation. GIL is not > released during indexing L, and if it is released between indexing and > assignment, it does not affect the result. and Steven: > Does that matter though? I think that's a distinction that makes no difference. > We know that another thread could change the L or the i before the assignment, if they are global. But once the L[i] lookup has occurred, it doesn't matter if they change. It's not going to affect what value gets bound to the x. Fair enough. Atomicity is a bit slippery, I find. It depends where the critical region starts. Thinking again, it's not the assignment that's the issue ... L is pushed i is pushed __getitem__ is called x is popped It is possible, if i and L are accessible to another thread and change after L is pushed, that x is given a value composed from an i and an L that never existed concurrently in the view of the other thread. Straining at gnats here, but atomicity is a strong claim. And on the point about re-ordering and CPUs, I can't imagine re-ordering that effectively changes the order of byte codes. But do CPython threads run in separate CPUs, or is that only when we have multiple interpreters? If so, and L were in a hot memory location (either the variable or its content), this could be inconsistent between threads. Sorry, I don't know the memory coherence CPython has: I know I couldn't rely on it in Java. I'm just arguing that the section gives advice that is *nearly* always right, which is a horrible thing to debug. I'll stop stirring. -- ___ Python tracker <https://bugs.python.org/issue45435> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue39639] Remove Suite node from AST
Jeff Allen added the comment: Jython uses the reference grammar and ASDL as a way to ensure it is Python we approximate, not some subtly different language. The presence of Suite here gives rise to a class (https://github.com/jythontools/jython/blob/v2.7.2b3/src/org/python/antlr/ast/Suite.java) and we actually use instances of it in the compiler (https://github.com/jythontools/jython/blob/v2.7.2b3/src/org/python/compiler/CodeCompiler.java#L2389). It is a bit of a wart, to have a Jython-specific type here: somewhat defeating the object of using the same source. I expect there was a good reason: perhaps there was no better way to express the commonality between Interactive and Module. It was all before my involvement. I would try to avoid needing it in Jython 3, and if we can't, it doesn't look hard to manage the variation our copy. It's not like we copy these files mechanically from from CPython during a build. +1 on removing it. -- nosy: +jeff.allen ___ Python tracker <https://bugs.python.org/issue39639> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue31630] math.tan has poor accuracy near pi/2 on OpenBSD
Change by Jeff Allen : -- nosy: +jeff.allen ___ Python tracker <https://bugs.python.org/issue31630> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue20155] Regression test test_httpservers fails, hangs on Windows
Jeff Allen added the comment: Disabling the AV/firewall did not stop the symptoms when I was investigating originally. In order to get the unmodified test to pass, I had to stop the BFE (base filtering engine), which I think may have been given new rules or behaviours as a result of installing the AV solution ... or maybe it was a Windows upgrade that did it. I did wonder if this might be a moving target, as the test deliberately includes server abuse, while the products want to stop that. If I try test_httpservers.py as amended (http://hg.python.org/cpython/file/ffdd2d0b0049/Lib/test/test_httpservers.py) on my machine with CPython 3.4.1, I do not get the error Terry reports. (test_urlquote_decoding_in_cgi_check fails but it should.) -- ___ Python tracker <http://bugs.python.org/issue20155> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue20155] Regression test test_httpservers fails, hangs on Windows
New submission from Jeff Allen: When I run: start python -m test.test_httpservers test_request_line_trimming reports ERROR, and the test hangs at test_version_none. If I run a copy of the test in which the latter test is skipped with @unittest.skipIf(sys.platform == "win32", "..."), the error report is: == ERROR: test_request_line_trimming (__main__.BaseHTTPServerTestCase) -- Traceback (most recent call last): File "test_httpservers.py", line 122, in test_request_line_trimming res = self.con.getresponse() File "C:\Python33\lib\http\client.py", line 1131, in getresponse response.begin() File "C:\Python33\lib\http\client.py", line 354, in begin version, status, reason = self._read_status() File "C:\Python33\lib\http\client.py", line 316, in _read_status line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1") File "C:\Python33\lib\socket.py", line 297, in readinto return self._sock.recv_into(b) ConnectionResetError: [WinError 10054] An existing connection was forcibly closed by the remote host This is essentially the same for Python 2.7.6 and for the current development tip of Jython 2.7b1+, which is actually where the problem first manifested. My machine is running 64-bit Windows 7 SP1, recently re-installed to a new, empty disk. Careful testing, elaborating the failing tests, shows that what is sent in PUT and GET operations is not quite what is received. Something tampers with the connection between the client and the server. (Identical traffic where the verbs are not PUT and GET arrives as sent.) Something fiddles with the forward message, for example "correcting" the spurious \n in test_request_line_trimming to a full \r\n, and holding back the payload of a PUT even when it was in the first packet. On the reverse path, it appears to act on the error response itself by closing the connection, without passing it to the client. Disabling the firewall (Windows Firewall and a commercial one), with the network cable unplugged, makes no difference. Nor does stopping anti-virus, anti-phishing, parental controls, etc.. However, stopping the Windows Basic Filtering Engine (BFE), makes the regression test run without error. Stopping the BFE takes out several dependent services, including Windows Firewall, but it seems likely the BFE itself is the culprit. Although the cause lies in the platform, not in Python, it seems to me still an "issue" for Python that the tests fail on a common platform practically out of the box. I'll work on this in the context of the Jython test and report back here. -- components: Library (Lib), Windows messages: 207497 nosy: jeff.allen priority: normal severity: normal status: open title: Regression test test_httpservers fails, hangs on Windows versions: Python 2.7, Python 3.3 ___ Python tracker <http://bugs.python.org/issue20155> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue20155] Regression test test_httpservers fails, hangs on Windows
Jeff Allen added the comment: Thanks for adding to the evidence here. As discussed above, disabling the security product (which is Bitdefender) on my PC didn't stop the problem for me, and I'm reluctant to uninstall. I narrowed it to the Windows Base Filtering Engine, but perhaps the behaviour of the BFE is extended by installing BD. If so, you could say this is not a Python problem, it is caused by BD "normalising" the HTTP. Or BD could say it is caused by expecting a defined result from abnormal HTTP. I took the view it were best fixed at our end. I found I could test the same thing (AFAICT), but modify the tests so they don't get interfered with. http://bugs.jython.org/issue2109 http://hg.python.org/jython/rev/6441fcfd940b Would a patch made from this be applicable to CPython? -- ___ Python tracker <http://bugs.python.org/issue20155> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue20155] Regression test test_httpservers fails, hangs on Windows
Jeff Allen added the comment: Actual patch for your convenience. I'm not set up to build CPython from source, so I've tested this with my installed CPython 2.7.6, and it's clean. [As for keeping the tests in sync, yes that's our aim. Jython's Lib contains only the customised versions, and everything else comes from a copy of CPython's in lib-python/2.7. I'm always looking for a chance to delete one (i.e. use the common file).] -- keywords: +patch nosy: +fwierzbicki Added file: http://bugs.python.org/file34202/issue20155_py.patch ___ Python tracker <http://bugs.python.org/issue20155> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue20155] Regression test test_httpservers fails, hangs on Windows
Jeff Allen added the comment: I worked out that the essence of the test is to insert an extra \n at the end of a GET request line. The request is syntactically invalid for HTTP. The \n\r\n appears like two blank lines, implying no headers, but the headers then follow where no data should be. The server is supposed to respond with status 501, because it does not, in fact, define a GET operation. To find the replacement test plausible you have to accept that, with a server that doesn't define GET, the verb may as well be XYZBOGUS. Since the security filter doesn't understand that verb either (unlike GET), it doesn't interfere in the test. -- ___ Python tracker <http://bugs.python.org/issue20155> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue29463] Add `docstring` attribute to AST nodes
Jeff Allen added the comment: Just terminology ... strictly speaking what you've done here is "add a *field* to the nodes Module, FunctionDef and ClassDef", rather than add an *attribute* -- that is, when one is consistent with the terms used in the ast module (https://docs.python.org/3/library/ast.html#node-classes) or Wang (https://docs.python.org/devguide/compiler.html#wang97). -- nosy: +jeff.allen ___ Python tracker <http://bugs.python.org/issue29463> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue27427] Add new math module tests
Jeff Allen added the comment: It would be nice to see this considered alongside #26040. -- nosy: +jeff.allen ___ Python tracker <http://bugs.python.org/issue27427> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue26040] Improve coverage and rigour of test.test_math
Jeff Allen added the comment: Mark: Thanks for validating the additional cases so carefully. If you still want to apply it in stages then I suppose the change to the comparison logic could go first (untested idea), although that's also where I could most easily have made a mistake. -- ___ Python tracker <https://bugs.python.org/issue26040> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue26040] Improve coverage and rigour of test.test_math
Jeff Allen added the comment: Mark: Thanks for doing my homework. Points 1 and 3 I can readily agree with. I must take another look at to_ulps() with your patch on locally. I used the approach I did because I thought it was incorrect in exactly those corners where you prefer it. I'll take a closer look. -- ___ Python tracker <https://bugs.python.org/issue26040> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue26040] Improve coverage and rigour of test.test_math
Jeff Allen added the comment:
Ah, cunning: I can make sense of it in hex.
>>> hex(to_ulps(expected))
'0x3ff0'
>>> hex(to_ulps(got))
'0x3fec'
>>> hex( to_ulps(got) - to_ulps(expected) )
'-0x4'
... and what you've done with ulp then follows.
In my version a format like "{:d} ulps" was a bad idea when the error was a
gross one, but your to_ulps is only piece-wise linear -- large differences are
compressed.
I'm pleased my work has mostly survived: here's hoping the house build-bots
agree. erfc() is perhaps the last worry, but math & cmath pass on my machine.
--
___
Python tracker
<https://bugs.python.org/issue26040>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue22121] IDLE should start with HOME as the initial working directory
Jeff Allen added the comment: I'm also interested in a smooth experience for beginners. I have a factual observation with respect to Terry's comment: '''Windows icons have a Shortcut tab with a Start-in field. We should like to put %USERPROFILE% there, but this does not work -- msg253393.''' ... I note that several menu shortcuts have "Start in" set to %HOMEDRIVE%%HOMEPATH%. Examples are notepad, Internet Explorer and the command prompt. (This is on Win7x64.) What we want seems to be a normal thing to do, and achieved by some, but perhaps by a post installation script. Alternatively, once a .py file exists where you want to work, right-click "Edit with IDLE" provides the CWD we'd like best. Idea: add a New >> Python File context menu item. Encourage users to create a new file that way, then open it, and everything from there is smooth. (New issue if liked.) -- nosy: +jeff.allen ___ Python tracker <http://bugs.python.org/issue22121> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue26040] Improve coverage and rigour of test.test_math
Jeff Allen added the comment: Here is a patch that improves coverage and addresses the uneven accuracy. Required accuracy is now specified in ulps. Mostly, I have choses 1 ulp, since this passed for me on an x86 architecture (and also ARM), but this may be too ambitious. I have also responded to the comment relating to erfc: # XXX Would be better to weaken this test only # for large x, instead of for all x." I found I could not contribute the code I used to generate the additional test cases in Tools/scripts without failing test_tools. (It complained of a missing dependency. The generator uses mpmath.) -- keywords: +patch Added file: http://bugs.python.org/file42166/iss26040.patch ___ Python tracker <http://bugs.python.org/issue26040> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue26040] Improve coverage and rigour of test.test_math
Changes by Jeff Allen : Added file: http://bugs.python.org/file42190/stat_math.py ___ Python tracker <http://bugs.python.org/issue26040> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue26040] Improve coverage and rigour of test.test_math
Jeff Allen added the comment: Thanks for the prompt acknowledgement and for accepting this to review. I have updated the coverage & tolerance demo program. Usage in the comments (in v3). I have also added the program I used to generate the extra test cases (needs mpmath -- easier to get working than mpf in the original Windows/Jython environment). -- ___ Python tracker <http://bugs.python.org/issue26040> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue26040] Improve coverage and rigour of test.test_math
Changes by Jeff Allen : Removed file: http://bugs.python.org/file41526/stat_math.py ___ Python tracker <http://bugs.python.org/issue26040> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue26040] Improve coverage and rigour of test.test_math
Changes by Jeff Allen : Added file: http://bugs.python.org/file42192/stat_math.py ___ Python tracker <http://bugs.python.org/issue26040> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue26040] Improve coverage and rigour of test.test_math
Changes by Jeff Allen : Removed file: http://bugs.python.org/file42190/stat_math.py ___ Python tracker <http://bugs.python.org/issue26040> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue26040] Improve coverage and rigour of test.test_math
Changes by Jeff Allen : Added file: http://bugs.python.org/file42191/extra_cmath_testcases.py ___ Python tracker <http://bugs.python.org/issue26040> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
