Re: [Cython] OS X 10.7 Lion: GCC __builtin_expect unrecognized inside OpenMP blocks
mark florisson, 16.03.2012 04:00: > $ cpp > #if defined(__APPLE__) || defined(__OSX__) > FOO > #endif > ^D Ah, hackers. Isn't that just lovely? Stefan ___ cython-devel mailing list cython-devel@python.org http://mail.python.org/mailman/listinfo/cython-devel
[Cython] Exception instantiation
Hi, I was made aware of a difference between the ways Cython and CPython raise exceptions. In CPython, the exception is always instantiated immediately, even if you raise a plain type, i.e. raise TypeError will actually raise a TypeError() instance. In Cython, the exception is only stored, not instantiated. The difference can show when the instantiation fails for some reason or has other side effects. Depending on *when* and *if* the exception is actually instantiated, these effects may show at unexpected times during execution, or may be not occur at all in some corner cases (e.g. when the exception is shadowed by another one in Py2 or caught by an except block in Cython that contains only C code). I think this should change. We may consider leaving it as it is for known exception types that are created without arguments, but otherwise, and especially for user provided exceptions, we should take the safer route of always instantiating them, like CPython does. That implies a certain performance hit in some rather rare cases (of which an explicitly raised StopIteration is the most visible one, if not exempted), but under normal conditions where an exception is regularly caught at some point, the overhead is the same and it would have less surprising corner cases. Stefan ___ cython-devel mailing list cython-devel@python.org http://mail.python.org/mailman/listinfo/cython-devel
Re: [Cython] Exception instantiation
On 21 March 2012 11:03, Stefan Behnel wrote: > Hi, > I think this should change. We may consider leaving it as it is for known > exception types that are created without arguments, +1 > but otherwise, and > especially for user provided exceptions, we should take the safer route of > always instantiating them, like CPython does. +1 -- Lisandro Dalcin --- CIMEC (INTEC/CONICET-UNL) Predio CONICET-Santa Fe Colectora RN 168 Km 472, Paraje El Pozo 3000 Santa Fe, Argentina Tel: +54-342-4511594 (ext 1011) Tel/Fax: +54-342-4511169 ___ cython-devel mailing list cython-devel@python.org http://mail.python.org/mailman/listinfo/cython-devel
Re: [Cython] OS X 10.7 Lion: GCC __builtin_expect unrecognized inside OpenMP blocks
Could you try the new fix from my branch? If it doesn't do the trick then I will need to know whether the macros are passing the preprocessor. On 15 March 2012 13:21, Lisandro Dalcin wrote: > On 14 March 2012 08:58, mark florisson wrote: >> On 9 March 2012 14:22, Lisandro Dalcin wrote: >> >> Could you give this fix a try? >> https://github.com/markflorisson88/cython/commit/2bffde15edc66c7416716051959e3b0cf1d6b41b >> > > I'm still getting the linking error. Your fix does not resolve the issue :-( > > > -- > Lisandro Dalcin > --- > CIMEC (INTEC/CONICET-UNL) > Predio CONICET-Santa Fe > Colectora RN 168 Km 472, Paraje El Pozo > 3000 Santa Fe, Argentina > Tel: +54-342-4511594 (ext 1011) > Tel/Fax: +54-342-4511169 > ___ > cython-devel mailing list > cython-devel@python.org > http://mail.python.org/mailman/listinfo/cython-devel ___ cython-devel mailing list cython-devel@python.org http://mail.python.org/mailman/listinfo/cython-devel
Re: [Cython] some pull requests for 0.16?
On 20 March 2012 18:51, Stefan Behnel wrote: > mark florisson, 20.03.2012 17:40: >> On 18 March 2012 11:58, Stefan Behnel wrote: >>> I put up two new pull requests on github: >>> >>> Implementation of PEP 380 (yield from): >>> https://github.com/cython/cython/pull/96 >>> >>> Rewrite of dict iteration: >>> https://github.com/cython/cython/pull/95 >>> >>> Given that the release of 0.16 has currently slowed down a bit, and given >>> that these are really nice features, could someone (and especially Mark, as >>> the responsible release manager) take a look at them and give an opinion >>> regarding their suitability for 0.16? >> >> Sorry about the slowdown, my BT internet connection at home got closed >> off, so working on a release is rather hard, hopefully that will be >> fixed within the next few days. Anyway, during the sprints Dag worked >> on the numpy attributes accesses to generate non-deprecated numpy >> code, I'm not sure how far along that is and if we can get that in as >> well, or have it wait until 0.16.1. > > I wouldn't mind if my changes waited for 0.16.1 as well, as long as that's > not too long from now. There will certainly be things to fix after the > release anyway. > > >> I worked on enhancing fused types runtime dispatching (e.g. select the >> right specialization at runtime from the argument types), especially >> on matching buffers. It's also a bit of a rewrite as the complexity >> grew quite a bit and ndarray and fused types were broken in subtle >> ways (I would do the pull request now if I hadn't left the code at >> home :). > > I took a tiny look at that when I wrote a test case and was surprised to > see that it imports "sys" inside of the generated function. Looks like > there's a bit left to optimise. > > I faintly remember a blog post (or more than one) by PJ Eby when he wrote a > "generic functions" module. It uses type dispatching as well, and he wrote > about the ways he found to tune that. Might be an interesting read. > Thanks, I'll look into that. Maybe the modules could be bound as locals through default arguments. >> Finally, there is a serious bug in the reversed(range()) optimization >> that Mike Graham discovered when working on the range() optimization >> with a runtime step, which is basically wrong for any step that is not >> 1 or -1 (it simply swaps the bounds with an offset by 1, but it >> actually has to figure out what the actual last value was in the range >> and use that as the start instead). I personally consider this a >> blocker, so I'll try contacting Mike and see if he is still >> (interested in) working on that. > > Interesting. Yes, I can well imagine that being wrong. These things are > exceedingly tricky. > > However, it's easy to disable the optimisation for a specific case - likely > easier than fixing it. > > Also, I don't consider this a blocker. It's not like it works in the > current release, so it's not a regression compared to that. > Yes, it's not a regression, but I find it hard to release something that is so broken, and I feel it kind of represents a lack of care which may make people believe Cython can't be taken seriously. >> As for the two pull requests, both are quite large, but the dict >> iteration rewrite is more like an enhancement whereas the 'yield from' >> is really a new (somewhat big) feature. I would personally prefer to >> merge only enhancements and bug fixes, but not new features. > > Funny. I would have proposed the opposite, because I consider the "yield > from" implementation (as a new language feature) safer than the dict > iteration (which impacts existing code). As I said, I wouldn't mind > postponing both to 0.16.1, but I think it would be better to get them out > and have people use them, so that we can also fix the bugs earlier. > A good point. My reasoning was that we want less new functionality per release, and keeping 'yield from' out of 0.16 makes us push out the next release earlier as we will have pending functionality we want to get out there. If the dict iteration is tested well enough (and it looked to me like that), then hopefully code in sage or lxml will break if there are leftover bugs, or our next beta might catch them (or some). If you do want it in now, then that's fine with me as well, you have a point. >> If you >> feel confident enough that the dict iteration pull request is ready to >> be merged, then please do so. > > I'll take another look over it and decide. Thanks. > > >> We can do an 0.16.1 release with the >> 'yield from', the 'lnotab' line number mapping from code objects to >> Cython source > > Note that that's not useful yet - it requires changes to the code that > *uses* the code objects as well, to a) actually reuse the code objects (iff > C line numbers are disabled in traces) and b) write the proper relative > line into the byte code offset field. I just put it up to have a place > where we can discuss further changes. > > >> and probably the boundscheck and wraparound >>
[Cython] Jenkins down
Hi, it seems like the sage.math server is broken again, at least it's lacking mounts when I try to log in. I'll restart Jenkins as soon as it's back working. Stefan ___ cython-devel mailing list cython-devel@python.org http://mail.python.org/mailman/listinfo/cython-devel
Re: [Cython] Jenkins down
On 21 March 2012 14:58, Stefan Behnel wrote: > Hi, > > it seems like the sage.math server is broken again, at least it's lacking > mounts when I try to log in. I'll restart Jenkins as soon as it's back > working. > > Stefan > ___ > cython-devel mailing list > cython-devel@python.org > http://mail.python.org/mailman/listinfo/cython-devel Indeed, it's rather worrying and is actually a bit of a slowdown for development. Dag and I discussed getting some funding from numfocus to use Jenkins in a cloud environment (along possibly with other projects, like sage, lxml, numpy, scipy, depending on interest of respective projects). At PyCon we saw one such company that provides these services, which is shiningpanda.com (which is down right now, which isn't very promising :). ___ cython-devel mailing list cython-devel@python.org http://mail.python.org/mailman/listinfo/cython-devel
Re: [Cython] Jenkins down
On 21 March 2012 15:07, mark florisson wrote: > On 21 March 2012 14:58, Stefan Behnel wrote: >> Hi, >> >> it seems like the sage.math server is broken again, at least it's lacking >> mounts when I try to log in. I'll restart Jenkins as soon as it's back >> working. >> >> Stefan >> ___ >> cython-devel mailing list >> cython-devel@python.org >> http://mail.python.org/mailman/listinfo/cython-devel > > Indeed, it's rather worrying and is actually a bit of a slowdown for > development. Dag and I discussed getting some funding from numfocus to > use Jenkins in a cloud environment (along possibly with other > projects, like sage, lxml, numpy, scipy, depending on interest of > respective projects). At PyCon we saw one such company that provides > these services, which is shiningpanda.com (which is down right now, > which isn't very promising :). Or maybe that was my flaky wifi connection :p. Anyway, there is also TeamCity from Jetbrains http://www.jetbrains.com/teamcity/, which is supposed to be free for small teams , and it can be used together with Amazon's cloud services: http://www.jetbrains.com/teamcity/features/amazon_ec2.html . ___ cython-devel mailing list cython-devel@python.org http://mail.python.org/mailman/listinfo/cython-devel
Re: [Cython] Jenkins down
mark florisson, 21.03.2012 15:12: > On 21 March 2012 15:07, mark florisson wrote: >> On 21 March 2012 14:58, Stefan Behnel wrote: >>> it seems like the sage.math server is broken again, at least it's lacking >>> mounts when I try to log in. I'll restart Jenkins as soon as it's back >>> working. >> >> Indeed, it's rather worrying and is actually a bit of a slowdown for >> development. Sure. However, we get to use it for free, and we've had a good gain from that for a long time now. It's just bad timing that this coincides with our release preparations. >> Dag and I discussed getting some funding from numfocus to >> use Jenkins in a cloud environment (along possibly with other >> projects, like sage, lxml, numpy, scipy, depending on interest of >> respective projects). At PyCon we saw one such company that provides >> these services, which is shiningpanda.com Their free OSS plan is one hour per day, which is way too short for us. Not sure how much we'd want to pay for their dedicated plan, and no idea what performance we'd get for the money. For our current setup, you can count way more than one hour of processing time for each single push and for each day's run of the CPython triggered integration tests. And then there's our set of separate developer branches on top of that, plus the fact that we can run builds in parallel to get faster feedback. Seriously - thanks William! > Anyway, there is also > TeamCity from Jetbrains http://www.jetbrains.com/teamcity/, which is > supposed to be free for small teams , and it can be used together with > Amazon's cloud services: > http://www.jetbrains.com/teamcity/features/amazon_ec2.html . I'm quite happy with Jenkins, though. Stefan ___ cython-devel mailing list cython-devel@python.org http://mail.python.org/mailman/listinfo/cython-devel
Re: [Cython] Jenkins down
On Wed, Mar 21, 2012 at 11:58 AM, Stefan Behnel wrote: > mark florisson, 21.03.2012 15:12: >> On 21 March 2012 15:07, mark florisson wrote: >>> On 21 March 2012 14:58, Stefan Behnel wrote: it seems like the sage.math server is broken again, at least it's lacking mounts when I try to log in. I'll restart Jenkins as soon as it's back working. >>> >>> Indeed, it's rather worrying and is actually a bit of a slowdown for >>> development. > > Sure. However, we get to use it for free, and we've had a good gain from > that for a long time now. It's just bad timing that this coincides with our > release preparations. > > >>> Dag and I discussed getting some funding from numfocus to >>> use Jenkins in a cloud environment (along possibly with other >>> projects, like sage, lxml, numpy, scipy, depending on interest of >>> respective projects). At PyCon we saw one such company that provides >>> these services, which is shiningpanda.com > > Their free OSS plan is one hour per day, which is way too short for us. Not > sure how much we'd want to pay for their dedicated plan, and no idea what > performance we'd get for the money. > > For our current setup, you can count way more than one hour of processing > time for each single push and for each day's run of the CPython triggered > integration tests. And then there's our set of separate developer branches > on top of that, plus the fact that we can run builds in parallel to get > faster feedback. > > Seriously - thanks William! And thanks to the National Science Foundation for buying the machines. If anybody reading this can volunteer some "sysadmin time and expertise" that could help improve the stability of sage.math. The latest crash was purely linux/software related. > > >> Anyway, there is also >> TeamCity from Jetbrains http://www.jetbrains.com/teamcity/, which is >> supposed to be free for small teams , and it can be used together with >> Amazon's cloud services: >> http://www.jetbrains.com/teamcity/features/amazon_ec2.html . > > I'm quite happy with Jenkins, though. > > Stefan > ___ > cython-devel mailing list > cython-devel@python.org > http://mail.python.org/mailman/listinfo/cython-devel -- William Stein Professor of Mathematics University of Washington http://wstein.org ___ cython-devel mailing list cython-devel@python.org http://mail.python.org/mailman/listinfo/cython-devel
Re: [Cython] Jenkins down
On 21 March 2012 15:58, Stefan Behnel wrote: > mark florisson, 21.03.2012 15:12: >> On 21 March 2012 15:07, mark florisson wrote: >>> On 21 March 2012 14:58, Stefan Behnel wrote: it seems like the sage.math server is broken again, at least it's lacking mounts when I try to log in. I'll restart Jenkins as soon as it's back working. >>> >>> Indeed, it's rather worrying and is actually a bit of a slowdown for >>> development. > > Sure. However, we get to use it for free, and we've had a good gain from > that for a long time now. It's just bad timing that this coincides with our > release preparations. > > >>> Dag and I discussed getting some funding from numfocus to >>> use Jenkins in a cloud environment (along possibly with other >>> projects, like sage, lxml, numpy, scipy, depending on interest of >>> respective projects). At PyCon we saw one such company that provides >>> these services, which is shiningpanda.com > > Their free OSS plan is one hour per day, which is way too short for us. Not > sure how much we'd want to pay for their dedicated plan, and no idea what > performance we'd get for the money. > > For our current setup, you can count way more than one hour of processing > time for each single push and for each day's run of the CPython triggered > integration tests. And then there's our set of separate developer branches > on top of that, plus the fact that we can run builds in parallel to get > faster feedback. > > Seriously - thanks William! > Yeah I also like it quite a bit, and I am grateful for these servers as well. It's just that they have been down quite often lately. For $240 a month you get one build queue for an unlimited number of projects, but indeed, I don't know the performance. Just wanted to bring it up in case there is any interest. At least with amazon I think you can run all tests in parallel as it can just fire up more VMs. >> Anyway, there is also >> TeamCity from Jetbrains http://www.jetbrains.com/teamcity/, which is >> supposed to be free for small teams , and it can be used together with >> Amazon's cloud services: >> http://www.jetbrains.com/teamcity/features/amazon_ec2.html . > > I'm quite happy with Jenkins, though. > > Stefan > ___ > cython-devel mailing list > cython-devel@python.org > http://mail.python.org/mailman/listinfo/cython-devel ___ cython-devel mailing list cython-devel@python.org http://mail.python.org/mailman/listinfo/cython-devel
Re: [Cython] Exception instantiation
On Wed, Mar 21, 2012 at 1:03 AM, Stefan Behnel wrote: > Hi, > > I was made aware of a difference between the ways Cython and CPython raise > exceptions. In CPython, the exception is always instantiated immediately, > even if you raise a plain type, i.e. > > raise TypeError > > will actually raise a TypeError() instance. In Cython, the exception is > only stored, not instantiated. The difference can show when the > instantiation fails for some reason or has other side effects. Depending on > *when* and *if* the exception is actually instantiated, these effects may > show at unexpected times during execution, or may be not occur at all in > some corner cases (e.g. when the exception is shadowed by another one in > Py2 or caught by an except block in Cython that contains only C code). > > I think this should change. We may consider leaving it as it is for known > exception types that are created without arguments, but otherwise, and > especially for user provided exceptions, we should take the safer route of > always instantiating them, like CPython does. That implies a certain > performance hit in some rather rare cases (of which an explicitly raised > StopIteration is the most visible one, if not exempted), but under normal > conditions where an exception is regularly caught at some point, the > overhead is the same and it would have less surprising corner cases. +1. One could consider making an exception for the no-argument builtin (side-effect-free) ones, but this would just be an optimization, and we're typically already in an expensive, exceptional state at this point. - Robert ___ cython-devel mailing list cython-devel@python.org http://mail.python.org/mailman/listinfo/cython-devel
Re: [Cython] Jenkins down
On Wed, Mar 21, 2012 at 8:58 AM, Stefan Behnel wrote: > mark florisson, 21.03.2012 15:12: >> On 21 March 2012 15:07, mark florisson wrote: >>> On 21 March 2012 14:58, Stefan Behnel wrote: it seems like the sage.math server is broken again, at least it's lacking mounts when I try to log in. I'll restart Jenkins as soon as it's back working. >>> >>> Indeed, it's rather worrying and is actually a bit of a slowdown for >>> development. > > Sure. However, we get to use it for free, and we've had a good gain from > that for a long time now. It's just bad timing that this coincides with our > release preparations. I'd say that since I've started using it sage.math has averaged close to 99% uptime, it's just bad timing at the moment. >>> Dag and I discussed getting some funding from numfocus to >>> use Jenkins in a cloud environment (along possibly with other >>> projects, like sage, lxml, numpy, scipy, depending on interest of >>> respective projects). At PyCon we saw one such company that provides >>> these services, which is shiningpanda.com > > Their free OSS plan is one hour per day, which is way too short for us. Not > sure how much we'd want to pay for their dedicated plan, and no idea what > performance we'd get for the money. > > For our current setup, you can count way more than one hour of processing > time for each single push and for each day's run of the CPython triggered > integration tests. And then there's our set of separate developer branches > on top of that, plus the fact that we can run builds in parallel to get > faster feedback. > > Seriously - thanks William! +1, this has made a huge difference in our development process. A system powerful enough to rebuild and retest Sage as well as running Cython's own self-tests is certainly a worthwhile sue of this hardware from Sage's point of view as well. Unfortunately, I have neither the skills nor time to do the kinds of sys-adminy things that sage.math needs. - Robert ___ cython-devel mailing list cython-devel@python.org http://mail.python.org/mailman/listinfo/cython-devel
Re: [Cython] some pull requests for 0.16?
On Wed, Mar 21, 2012 at 6:47 AM, mark florisson wrote: > On 20 March 2012 18:51, Stefan Behnel wrote: >> mark florisson, 20.03.2012 17:40: >>> On 18 March 2012 11:58, Stefan Behnel wrote: I put up two new pull requests on github: Implementation of PEP 380 (yield from): https://github.com/cython/cython/pull/96 Rewrite of dict iteration: https://github.com/cython/cython/pull/95 Given that the release of 0.16 has currently slowed down a bit, and given that these are really nice features, could someone (and especially Mark, as the responsible release manager) take a look at them and give an opinion regarding their suitability for 0.16? >>> >>> Sorry about the slowdown, my BT internet connection at home got closed >>> off, so working on a release is rather hard, hopefully that will be >>> fixed within the next few days. Thanks for the status update. Is there a separate release branch, or are you working off the github master? Is there a list of issues/blockers somewhere? >>> Anyway, during the sprints Dag worked >>> on the numpy attributes accesses to generate non-deprecated numpy >>> code, I'm not sure how far along that is and if we can get that in as >>> well, or have it wait until 0.16.1. >> >> I wouldn't mind if my changes waited for 0.16.1 as well, as long as that's >> not too long from now. There will certainly be things to fix after the >> release anyway. >> >> >>> I worked on enhancing fused types runtime dispatching (e.g. select the >>> right specialization at runtime from the argument types), especially >>> on matching buffers. It's also a bit of a rewrite as the complexity >>> grew quite a bit and ndarray and fused types were broken in subtle >>> ways (I would do the pull request now if I hadn't left the code at >>> home :). >> >> I took a tiny look at that when I wrote a test case and was surprised to >> see that it imports "sys" inside of the generated function. Looks like >> there's a bit left to optimise. >> >> I faintly remember a blog post (or more than one) by PJ Eby when he wrote a >> "generic functions" module. It uses type dispatching as well, and he wrote >> about the ways he found to tune that. Might be an interesting read. >> > > Thanks, I'll look into that. Maybe the modules could be bound as > locals through default arguments. > >>> Finally, there is a serious bug in the reversed(range()) optimization >>> that Mike Graham discovered when working on the range() optimization >>> with a runtime step, which is basically wrong for any step that is not >>> 1 or -1 (it simply swaps the bounds with an offset by 1, but it >>> actually has to figure out what the actual last value was in the range >>> and use that as the start instead). I personally consider this a >>> blocker, so I'll try contacting Mike and see if he is still >>> (interested in) working on that. >> >> Interesting. Yes, I can well imagine that being wrong. These things are >> exceedingly tricky. >> >> However, it's easy to disable the optimisation for a specific case - likely >> easier than fixing it. >> >> Also, I don't consider this a blocker. It's not like it works in the >> current release, so it's not a regression compared to that. >> > > Yes, it's not a regression, but I find it hard to release something > that is so broken, and I feel it kind of represents a lack of care > which may make people believe Cython can't be taken seriously. I'm usually in the "not a regression" camp as well, but this is pretty fundamental. Sounds like an easy fix, or at least an easy-to-disable optimization. >>> As for the two pull requests, both are quite large, but the dict >>> iteration rewrite is more like an enhancement whereas the 'yield from' >>> is really a new (somewhat big) feature. I would personally prefer to >>> merge only enhancements and bug fixes, but not new features. >> >> Funny. I would have proposed the opposite, because I consider the "yield >> from" implementation (as a new language feature) safer than the dict >> iteration (which impacts existing code). As I said, I wouldn't mind >> postponing both to 0.16.1, but I think it would be better to get them out >> and have people use them, so that we can also fix the bugs earlier. >> > > A good point. My reasoning was that we want less new functionality per > release, and keeping 'yield from' out of 0.16 makes us push out the > next release earlier as we will have pending functionality we want to > get out there. If the dict iteration is tested well enough (and it > looked to me like that), then hopefully code in sage or lxml will > break if there are leftover bugs, or our next beta might catch them > (or some). If you do want it in now, then that's fine with me as well, > you have a point. Has this been pushed into the devel branch for sufficient testing there (e.g. on Sage)? Personally, I'd like to see the release get out sooner rather than later. - Robert ___ cyth
Re: [Cython] [cython-users] GSoC 2012
On Sat, Mar 10, 2012 at 10:44 PM, mark florisson wrote: > On 8 March 2012 14:27, Stefan Behnel wrote: >> Hi, >> >> I noticed that people start rushing for the next season on Python's GSoC >> mailing lists. Do we have any interested developers here, or general ideas >> about suitable topics? I would expect that we'll do as in the last years >> and participate under Python's umbrella. Also, we'd like to see patches from anyone interesting in being a GSoC student, as this will be a requirement as in past years. > I will likely be submitting a proposal for the OpenCL support CEP. OpenCL would be an interesting experiment, but I think still has limited utility. Dag and I were talking the other day about the challenge of generating the best possible code for evaluating array expressions (think inlined memoryview arithmetic) taking into account memory layout, blocking, etc. which Fortran does really well which could be an interesting direction. > As for other proposals, anyone can come up with something themselves or > choose a suitable CEP. Some ideas: Numbering items for clarity: 1. > - fused cdef classes (probably not as an entire gsoc project) 2. > - profile guided optimizations (using python's profilers and/or a > custom profiler that collects data such as types etc, which can be > used to specialize variables inside loops (or entire functions) with a > fallback to normal mode in case the type changes) 3. > - cython library to contain common functionality (although that might > be a bit boring and rather involved) 4. > - better type inference, that would be enabled by default and again > handle thing like reassignments of variables and fallbacks to the > default object type. With entry caching Cython could build a database > of types ((extension) classes, functions, variables) used in the > modules and functions that are compiled (also def functions), and > infer the types used and specialize on those. Maybe a switch should be > added to cython to handle circular dependencies, or maybe with the > distutils preprocessing it can run all the type inference first and > keep track of unresolved entries, and try to fill those in after > building the database. For bonus points the user can be allowed to > write plugins to aid the process. 5. > - llvm based JIT :), i.e. have Cython instrument the generated code to > record information and use that to create specializations at runtime > (probably far out for a gsoc) What I would most like to see is the common component in 2 and 4, i.e. the ability to generate optimized code for imperfectly-inferred types, with transparent fallback to generic code if conditions are not met at runtime (during as well as at entering the optimized code path). Too many times we have to decide between being safe for all cases or fast for the common case. That being said, there are few people I'd trust with such an ambitious project, and you're on that short list :). - Robert ___ cython-devel mailing list cython-devel@python.org http://mail.python.org/mailman/listinfo/cython-devel
Re: [Cython] [cython-users] GSoC 2012
On 03/21/2012 01:56 PM, Robert Bradshaw wrote: On Sat, Mar 10, 2012 at 10:44 PM, mark florisson wrote: On 8 March 2012 14:27, Stefan Behnel wrote: Hi, I noticed that people start rushing for the next season on Python's GSoC mailing lists. Do we have any interested developers here, or general ideas about suitable topics? I would expect that we'll do as in the last years and participate under Python's umbrella. Also, we'd like to see patches from anyone interesting in being a GSoC student, as this will be a requirement as in past years. I will likely be submitting a proposal for the OpenCL support CEP. OpenCL would be an interesting experiment, but I think still has limited utility. Dag and I were talking the other day about the challenge of generating the best possible code for evaluating array expressions (think inlined memoryview arithmetic) taking into account memory layout, blocking, etc. which Fortran does really well which could be an interesting direction. Yes, a lot of water has run in the river since March 8 here. Anyone interested in reading up on current ideas on what Mark is thinking for his GSoC proposal should read up on the numpy-discussion thread "Looking for people interested in helping with Python compiler to LLVM". Dag As for other proposals, anyone can come up with something themselves or choose a suitable CEP. Some ideas: Numbering items for clarity: 1. - fused cdef classes (probably not as an entire gsoc project) 2. - profile guided optimizations (using python's profilers and/or a custom profiler that collects data such as types etc, which can be used to specialize variables inside loops (or entire functions) with a fallback to normal mode in case the type changes) 3. - cython library to contain common functionality (although that might be a bit boring and rather involved) 4. - better type inference, that would be enabled by default and again handle thing like reassignments of variables and fallbacks to the default object type. With entry caching Cython could build a database of types ((extension) classes, functions, variables) used in the modules and functions that are compiled (also def functions), and infer the types used and specialize on those. Maybe a switch should be added to cython to handle circular dependencies, or maybe with the distutils preprocessing it can run all the type inference first and keep track of unresolved entries, and try to fill those in after building the database. For bonus points the user can be allowed to write plugins to aid the process. 5. - llvm based JIT :), i.e. have Cython instrument the generated code to record information and use that to create specializations at runtime (probably far out for a gsoc) What I would most like to see is the common component in 2 and 4, i.e. the ability to generate optimized code for imperfectly-inferred types, with transparent fallback to generic code if conditions are not met at runtime (during as well as at entering the optimized code path). Too many times we have to decide between being safe for all cases or fast for the common case. That being said, there are few people I'd trust with such an ambitious project, and you're on that short list :). - Robert ___ cython-devel mailing list cython-devel@python.org http://mail.python.org/mailman/listinfo/cython-devel ___ cython-devel mailing list cython-devel@python.org http://mail.python.org/mailman/listinfo/cython-devel
Re: [Cython] [cython-users] GSoC 2012
On Wed, Mar 21, 2012 at 2:11 PM, Dag Sverre Seljebotn wrote: > On 03/21/2012 01:56 PM, Robert Bradshaw wrote: >> >> On Sat, Mar 10, 2012 at 10:44 PM, mark florisson >> wrote: >>> >>> On 8 March 2012 14:27, Stefan Behnel wrote: Hi, I noticed that people start rushing for the next season on Python's GSoC mailing lists. Do we have any interested developers here, or general ideas about suitable topics? I would expect that we'll do as in the last years and participate under Python's umbrella. >> >> >> Also, we'd like to see patches from anyone interesting in being a GSoC >> student, as this will be a requirement as in past years. >> >>> I will likely be submitting a proposal for the OpenCL support CEP. >> >> >> OpenCL would be an interesting experiment, but I think still has >> limited utility. Dag and I were talking the other day about the >> challenge of generating the best possible code for evaluating array >> expressions (think inlined memoryview arithmetic) taking into account >> memory layout, blocking, etc. which Fortran does really well which >> could be an interesting direction. > > > Yes, a lot of water has run in the river since March 8 here. Anyone > interested in reading up on current ideas on what Mark is thinking for his > GSoC proposal should read up on the numpy-discussion thread "Looking for > people interested in helping with Python compiler to LLVM". Thanks for the pointer. Clearly I'm not subscribed to all the relevant lists (and, admittedly, just finally starting to catch up on the ones I am subscribed to). - Robert ___ cython-devel mailing list cython-devel@python.org http://mail.python.org/mailman/listinfo/cython-devel
