[Cython] Broken tests
Hi, could we please make it a requirement to run the complete test suite after a branch merge? Or at least taking a close look at Jenkins after pushing a merge? And then fixing the bugs it finds? Stefan ___ cython-devel mailing list cython-devel@python.org http://mail.python.org/mailman/listinfo/cython-devel
Re: [Cython] Broken tests
2011/5/27 Stefan Behnel : > Hi, > > could we please make it a requirement to run the complete test suite after a > branch merge? Or at least taking a close look at Jenkins after pushing a > merge? And then fixing the bugs it finds? > > Stefan > There are failing tests in _control_flow branch. I guess it's better to fix them after merge. -- vitja. ___ cython-devel mailing list cython-devel@python.org http://mail.python.org/mailman/listinfo/cython-devel
Re: [Cython] Broken tests
On 05/27/2011 02:41 PM, Vitja Makarov wrote: 2011/5/27 Stefan Behnel: Hi, could we please make it a requirement to run the complete test suite after a branch merge? Or at least taking a close look at Jenkins after pushing a merge? And then fixing the bugs it finds? Stefan There are failing tests in _control_flow branch. I guess it's better to fix them after merge. Wouldn't it be better to rebase _control_flow on master, then fix tests, then merge it into master? (Replace "rebase" with "merge in master" if you wish, I'm going by the _ prefix here.) Dag Sverre ___ cython-devel mailing list cython-devel@python.org http://mail.python.org/mailman/listinfo/cython-devel
Re: [Cython] Broken tests
2011/5/27 Dag Sverre Seljebotn : > On 05/27/2011 02:41 PM, Vitja Makarov wrote: >> >> 2011/5/27 Stefan Behnel: >>> >>> Hi, >>> >>> could we please make it a requirement to run the complete test suite >>> after a >>> branch merge? Or at least taking a close look at Jenkins after pushing a >>> merge? And then fixing the bugs it finds? >>> >>> Stefan >>> >> >> There are failing tests in _control_flow branch. >> I guess it's better to fix them after merge. > > Wouldn't it be better to rebase _control_flow on master, then fix tests, > then merge it into master? > > (Replace "rebase" with "merge in master" if you wish, I'm going by the _ > prefix here.) > Sure. But I think that I need some help to make decision which tests should be fixed and which removed. -- vitja. ___ cython-devel mailing list cython-devel@python.org http://mail.python.org/mailman/listinfo/cython-devel
Re: [Cython] [cython] Openmp (#30)
[regarding the code for analysing uninitialised variables in the OpenMP branch] dagss, 27.05.2011 10:32: I suggest this: a) Vitja and Stefan notifies Mark of any issues that stops control flow analysis from going into trunk (outright collisions etc.). Those have prioritiy over fused types work etc (although I guess Mark don't have much GSoC time the next month). b) Come July, when control flow analysis is presumably in master, Mark takes a pass and removes any duplicated logic. Thoughts? I don't think there are any collisions, it's just that Vitja is currently removing the old visitor based attempts of doing variable initialisation analysis. Getting yet another one in while getting rid of the old ones is a rather unnecessary duplication of effort. I suggest we rebase Vitja's branch when the current test issues are solved, then we'll see if there are any issues with the parallel looping code. I'm in favour of merging the control flow branch ASAP, simply because it's such a great feature. I already disabled two tests in Mark's code that relied on uninitialised variables being None. There are likely others (way older ones) that were written with such misfeatures in mind. Stefan ___ cython-devel mailing list cython-devel@python.org http://mail.python.org/mailman/listinfo/cython-devel
Re: [Cython] [cython] Openmp (#30)
On 05/27/2011 03:54 PM, Stefan Behnel wrote: [regarding the code for analysing uninitialised variables in the OpenMP branch] dagss, 27.05.2011 10:32: I suggest this: a) Vitja and Stefan notifies Mark of any issues that stops control flow analysis from going into trunk (outright collisions etc.). Those have prioritiy over fused types work etc (although I guess Mark don't have much GSoC time the next month). b) Come July, when control flow analysis is presumably in master, Mark takes a pass and removes any duplicated logic. Thoughts? I don't think there are any collisions, it's just that Vitja is currently removing the old visitor based attempts of doing variable initialisation analysis. Getting yet another one in while getting rid of the old ones is a rather unnecessary duplication of effort. Well, what's done is done. We can try to be smarter in the future. Dag Sverre ___ cython-devel mailing list cython-devel@python.org http://mail.python.org/mailman/listinfo/cython-devel
[Cython] Control Flow
I've recently fixed some issues: - closure variables were not tracked - scoped expression variables were initialized to None So, I should fix broken tests, here are some of them: nogil (should be fixed) purecdef (upstream) cfunc_directive_in_pyclass (upstream) bufaccess_noassignT444 (should be removed?) bufaccess (should be fixed) ass2longlong (should be fixed) builtinfuncs (should be fixed) extcmethcall (should be fixed) extcoerce (fixed or removed?) e_extweakref (should be fixed) -- vitja. ___ cython-devel mailing list cython-devel@python.org http://mail.python.org/mailman/listinfo/cython-devel
Re: [Cython] Control Flow
On 05/27/2011 04:27 PM, Vitja Makarov wrote: I've recently fixed some issues: - closure variables were not tracked - scoped expression variables were initialized to None So, I should fix broken tests, here are some of them: nogil (should be fixed) purecdef (upstream) cfunc_directive_in_pyclass (upstream) bufaccess_noassignT444 (should be removed?) One could make it an error testcase instead, trapping the syntax error you now raise. Depends on how confident you are on new control flow tests and whether buffers are tested there. bufaccess (should be fixed) Yeah, feel free to just remove nousage() and printbuf(). ass2longlong (should be fixed) builtinfuncs (should be fixed) extcmethcall (should be fixed) extcoerce (fixed or removed?) e_extweakref (should be fixed) Dag ___ cython-devel mailing list cython-devel@python.org http://mail.python.org/mailman/listinfo/cython-devel
Re: [Cython] [cython] Openmp (#30)
On 27 May 2011 16:15, Dag Sverre Seljebotn wrote: > On 05/27/2011 03:54 PM, Stefan Behnel wrote: >> >> [regarding the code for analysing uninitialised variables in the OpenMP >> branch] >> >> dagss, 27.05.2011 10:32: >>> >>> I suggest this: >>> >>> a) Vitja and Stefan notifies Mark of any issues that stops control flow >>> analysis from going into trunk (outright collisions etc.). Those have >>> prioritiy over fused types work etc (although I guess Mark don't have >>> much GSoC time the next month). >>> >>> b) Come July, when control flow analysis is presumably in master, Mark >>> takes a pass and removes any duplicated logic. >>> >>> Thoughts? >> >> I don't think there are any collisions, it's just that Vitja is >> currently removing the old visitor based attempts of doing variable >> initialisation analysis. Getting yet another one in while getting rid of >> the old ones is a rather unnecessary duplication of effort. > > Well, what's done is done. We can try to be smarter in the future. > > Dag Sverre > ___ > cython-devel mailing list > cython-devel@python.org > http://mail.python.org/mailman/listinfo/cython-devel > Fortunately, the effort was rather small as it was a very simplistic way to detect uninitialized variables. I don't mind wasting 30 lines for a stopgap solution. And the good thing about it is that when the control flow branch gets merged the error tests are already written. ___ cython-devel mailing list cython-devel@python.org http://mail.python.org/mailman/listinfo/cython-devel
[Cython] Bug in ord(Py_UNICODE) optimization
Hi, ord() applied to a Py_UNICODE typed variable seems to yield a 1 character unicode string rather than an integer. The following doctest fails when added to unicodefunction.pyx: def Py_UNICODE_ord(unicode s): """ >>> Py_UNICODE_ord(u' ') 32 """ cdef Py_UNICODE u u = s[0] return ord(u) Thanks, John ___ cython-devel mailing list cython-devel@python.org http://mail.python.org/mailman/listinfo/cython-devel
[Cython] Redundant Cython exception message strings
Hi, I recently stumbled over a tradeoff question with AttributeError, and now found the same situation for UnboundLocalError in Vitja's control flow branch. So here it is. When we raise an exception several times in different parts of the code with a message that only differs slightly each time (usually something like "'NoneType' has no attribute X", or "local variable X referenced before assignment"), we have three choices to handle this: 1) Optimise for speed: create a Python string object at module initialisation time and call PyErr_SetObject(exc_type, msg_str_obj). 2) Current way: let CPython create the string object when raising the exception and just call PyErr_SetString(exc_type, "complete message"). 3) Trade speed for size and allow the C compiler to reduce the storage redundancy: write only the message template and the names as C char* constants by calling PyErr_Format(exc_type, "message template %s", "X"). Assuming that exceptions should be exceptional, I'm leaning towards 3). This would allow the C compiler to collapse multiple usages of the same C string into one data constant, thus reducing a bit of redundancy in the shared library size and the memory footprint. However, it would (slightly?) slow down the exception raising due to the additional string formatting, even when compared to the need to build a Python string object that it shares with 2). While 1) would obviously be the fastest way to raise an exception (no memory allocation, only refcounting), I think it's not worth it for exceptions as it increases both the runtime memory overhead and the module startup time. Thoughts? Stefan ___ cython-devel mailing list cython-devel@python.org http://mail.python.org/mailman/listinfo/cython-devel
Re: [Cython] Redundant Cython exception message strings
On Fri, May 27, 2011 at 3:32 PM, Stefan Behnel wrote: > Hi, > > I recently stumbled over a tradeoff question with AttributeError, and now > found the same situation for UnboundLocalError in Vitja's control flow > branch. So here it is. > > When we raise an exception several times in different parts of the code with > a message that only differs slightly each time (usually something like > "'NoneType' has no attribute X", or "local variable X referenced before > assignment"), we have three choices to handle this: > > 1) Optimise for speed: create a Python string object at module > initialisation time and call PyErr_SetObject(exc_type, msg_str_obj). > > 2) Current way: let CPython create the string object when raising the > exception and just call PyErr_SetString(exc_type, "complete message"). > > 3) Trade speed for size and allow the C compiler to reduce the storage > redundancy: write only the message template and the names as C char* > constants by calling PyErr_Format(exc_type, "message template %s", "X"). > > Assuming that exceptions should be exceptional, I'm leaning towards 3). This > would allow the C compiler to collapse multiple usages of the same C string > into one data constant, thus reducing a bit of redundancy in the shared > library size and the memory footprint. However, it would (slightly?) slow > down the exception raising due to the additional string formatting, even > when compared to the need to build a Python string object that it shares > with 2). While 1) would obviously be the fastest way to raise an exception > (no memory allocation, only refcounting), I think it's not worth it for > exceptions as it increases both the runtime memory overhead and the module > startup time. > > Thoughts? Any back-of-the-envelope calculations on how much the savings would be? I think I'm leaning towards 3 as well, certainly not option 1. - Robert ___ cython-devel mailing list cython-devel@python.org http://mail.python.org/mailman/listinfo/cython-devel
Re: [Cython] Broken tests
On Fri, May 27, 2011 at 6:16 AM, Vitja Makarov wrote: > 2011/5/27 Dag Sverre Seljebotn : >> On 05/27/2011 02:41 PM, Vitja Makarov wrote: >>> >>> 2011/5/27 Stefan Behnel: Hi, could we please make it a requirement to run the complete test suite after a branch merge? Or at least taking a close look at Jenkins after pushing a merge? And then fixing the bugs it finds? Stefan >>> >>> There are failing tests in _control_flow branch. >>> I guess it's better to fix them after merge. >> >> Wouldn't it be better to rebase _control_flow on master, then fix tests, >> then merge it into master? +1 >> (Replace "rebase" with "merge in master" if you wish, I'm going by the _ >> prefix here.) >> > > Sure. But I think that I need some help to make decision which tests > should be fixed and which removed. Do you have any specific tests that are unclear? In any case, I think this email was primarily aimed at me, as there was a conflict with the with-gil branch and the pure cdef function branch that I didn't catch before I merged. (I had planned to take a look at the results later, but my day got too crazy...) +1 to the policy of running the entire testsuite on at least one Python before pushing a merge. - Robert ___ cython-devel mailing list cython-devel@python.org http://mail.python.org/mailman/listinfo/cython-devel
Re: [Cython] Bug in ord(Py_UNICODE) optimization
John Ehresman, 27.05.2011 21:55: ord() applied to a Py_UNICODE typed variable seems to yield a 1 character unicode string rather than an integer. The following doctest fails when added to unicodefunction.pyx: def Py_UNICODE_ord(unicode s): """ >>> Py_UNICODE_ord(u' ') 32 """ cdef Py_UNICODE u u = s[0] return ord(u) ord() is actually unpacked in this case (see Optimise.py), but the result type is incorrect. It's Py_UNICODE (which coerces back to a unicode string), whereas it should be int (thus coercing to a Python integer object). I pushed a fix, and also the obvious optimisation for calling ord() on single character string literals. Thanks for the report. Stefan ___ cython-devel mailing list cython-devel@python.org http://mail.python.org/mailman/listinfo/cython-devel
Re: [Cython] Control Flow
On Fri, May 27, 2011 at 7:27 AM, Vitja Makarov wrote: > I've recently fixed some issues: > > - closure variables were not tracked > - scoped expression variables were initialized to None > > So, I should fix broken tests, here are some of them: > > nogil (should be fixed) > purecdef (upstream) > cfunc_directive_in_pyclass (upstream) > bufaccess_noassignT444 (should be removed?) > bufaccess (should be fixed) > ass2longlong (should be fixed) > builtinfuncs (should be fixed) > extcmethcall (should be fixed) > extcoerce (fixed or removed?) Unless you see this code being exercised elsewhere (a run test would be better her anyways) it's an easy fix that should be done. > e_extweakref (should be fixed) > > -- > vitja. > ___ > cython-devel mailing list > cython-devel@python.org > http://mail.python.org/mailman/listinfo/cython-devel > ___ cython-devel mailing list cython-devel@python.org http://mail.python.org/mailman/listinfo/cython-devel
