Re: [Cython] Shared Cython runtime

2013-04-09 Thread Stefan Behnel
Nikita Nemkin, 10.04.2013 08:44: > On Wed, 10 Apr 2013 11:24:33 +0600, Stefan Behnel wrote: >> The problem is not which value to use. The problem is that the shared >> library pathname is not known at init time, except inside of the loader, >> and it doesn't tell us about it. In the specific case o

Re: [Cython] Shared Cython runtime

2013-04-09 Thread Nikita Nemkin
On Wed, 10 Apr 2013 11:24:33 +0600, Stefan Behnel wrote: Nikita Nemkin, 10.04.2013 06:22: The stubs don't have to be shared libraries, pure python would work just fine (and reduce the complexity). Sure. That's actually the most common way to do it. Most C modules in CPython's stdlib have a

Re: [Cython] Shared Cython runtime

2013-04-09 Thread Robert Bradshaw
On Tue, Apr 9, 2013 at 10:24 PM, Stefan Behnel wrote: > Nikita Nemkin, 10.04.2013 06:22: >> Robert Bradshaw, 09.04.2013 20:16: > Yep. We could even create stub .so files for the included ones that > did nothing but import the big one to avoid having to worry about > importing things in

Re: [Cython] Shared Cython runtime

2013-04-09 Thread Stefan Behnel
Nikita Nemkin, 10.04.2013 06:22: > Robert Bradshaw, 09.04.2013 20:16: Yep. We could even create stub .so files for the included ones that did nothing but import the big one to avoid having to worry about importing things in the right order. > > The stubs don't have to be shared libr

Re: [Cython] Shared Cython runtime

2013-04-09 Thread Nikita Nemkin
Robert Bradshaw, 09.04.2013 20:16: Yep. We could even create stub .so files for the included ones that did nothing but import the big one to avoid having to worry about importing things in the right order. The stubs don't have to be shared libraries, pure python would work just fine (and reduce

Re: [Cython] Shared Cython runtime (was: Upcoming cython/numpy breakage with stride checking)

2013-04-09 Thread Stefan Behnel
Robert Bradshaw, 09.04.2013 21:00: > On Tue, Apr 9, 2013 at 7:22 AM, Nathaniel Smith wrote: >> On Tue, Apr 9, 2013 at 3:15 PM, Stefan Behnel wrote: >>> Ok, got it now. That solves the distribution problem, assuming that all >>> installed runtimes with a given version are equivalent. Basically, we'd

Re: [Cython] Shared Cython runtime (was: Upcoming cython/numpy breakage with stride checking)

2013-04-09 Thread Pauli Virtanen
09.04.2013 22:00, Robert Bradshaw kirjoitti: [clip] > I am curious what percentage of the final .so file is boilerplate. A > huge chunk of the .c code often is, but that consists largely of > specialize macros and inline functions that (ideally) optimize away to > nearly nothing. FWIW, switching f

Re: [Cython] Shared Cython runtime (was: Upcoming cython/numpy breakage with stride checking)

2013-04-09 Thread Robert Bradshaw
On Tue, Apr 9, 2013 at 7:22 AM, Nathaniel Smith wrote: > On Tue, Apr 9, 2013 at 3:15 PM, Stefan Behnel wrote: >> Ok, got it now. That solves the distribution problem, assuming that all >> installed runtimes with a given version are equivalent. Basically, we'd >> move the code out and then cimport

Re: [Cython] Shared Cython runtime

2013-04-09 Thread Robert Bradshaw
On Tue, Apr 9, 2013 at 11:35 AM, Stefan Behnel wrote: > Robert Bradshaw, 09.04.2013 20:16: >> On Tue, Apr 9, 2013 at 8:21 AM, Stefan Behnel wrote: >>> Stefan Behnel, 09.04.2013 17:12: mark florisson, 09.04.2013 16:51: > On 9 April 2013 15:47, Stefan Behnel wrote: > >> mark floris

Re: [Cython] Constant pointers not working

2013-04-09 Thread Robert Bradshaw
On Mon, Apr 8, 2013 at 4:56 AM, Stefan Behnel wrote: > Volker Mische, 08.04.2013 13:43: >> On 04/02/2013 07:20 PM, Volker Mische wrote: >>> On 03/22/2013 02:47 PM, Volker Mische wrote: Hi all, I was excited to see that 'const' is finally supported, but constant pointers are not

Re: [Cython] Shared Cython runtime

2013-04-09 Thread Stefan Behnel
Robert Bradshaw, 09.04.2013 20:16: > On Tue, Apr 9, 2013 at 8:21 AM, Stefan Behnel wrote: >> Stefan Behnel, 09.04.2013 17:12: >>> mark florisson, 09.04.2013 16:51: On 9 April 2013 15:47, Stefan Behnel wrote: > mark florisson, 09.04.2013 16:32: >> On 9 April 2013 14:55, Nikita Nem

Re: [Cython] Shared Cython runtime

2013-04-09 Thread Robert Bradshaw
On Tue, Apr 9, 2013 at 8:21 AM, Stefan Behnel wrote: > Stefan Behnel, 09.04.2013 17:12: >> mark florisson, 09.04.2013 16:51: >>> On 9 April 2013 15:47, Stefan Behnel wrote: >>> mark florisson, 09.04.2013 16:32: > On 9 April 2013 14:55, Nikita Nemkin wrote: >> One alternative for code

Re: [Cython] Shared Cython runtime

2013-04-09 Thread Stefan Behnel
Nikita Nemkin, 09.04.2013 18:30: > On Tue, 09 Apr 2013 21:21:25 +0600, Stefan Behnel wrote: >> Oh, and even simpler, you can just stick multiple Cython modules with their >> module init functions into one big shared library and have the "main" init >> function of that module call the others. That w

Re: [Cython] Shared Cython runtime

2013-04-09 Thread Nikita Nemkin
On Tue, 09 Apr 2013 21:21:25 +0600, Stefan Behnel wrote: Oh, and even simpler, you can just stick multiple Cython modules with their module init functions into one big shared library and have the "main" init function of that module call the others. That will stick them into sys.modules as

Re: [Cython] Shared Cython runtime

2013-04-09 Thread Stefan Behnel
Stefan Behnel, 09.04.2013 17:12: > mark florisson, 09.04.2013 16:51: >> On 9 April 2013 15:47, Stefan Behnel wrote: >> >>> mark florisson, 09.04.2013 16:32: On 9 April 2013 14:55, Nikita Nemkin wrote: > One alternative for code reuse in large Cython projects > could be packaging multip

Re: [Cython] Shared Cython runtime

2013-04-09 Thread Stefan Behnel
mark florisson, 09.04.2013 16:51: > On 9 April 2013 15:47, Stefan Behnel wrote: > >> mark florisson, 09.04.2013 16:32: >>> On 9 April 2013 14:55, Nikita Nemkin wrote: One alternative for code reuse in large Cython projects could be packaging multiple modules into one shared library. >>>

Re: [Cython] Shared Cython runtime (was: Upcoming cython/numpy breakage with stride checking)

2013-04-09 Thread mark florisson
On 9 April 2013 15:45, Nathaniel Smith wrote: > On Tue, Apr 9, 2013 at 3:32 PM, mark florisson > wrote: > > On 9 April 2013 14:55, Nikita Nemkin wrote: > >> One alternative for code reuse in large Cython projects > >> could be packaging multiple modules into one shared library. > > > > We have

Re: [Cython] Shared Cython runtime (was: Upcoming cython/numpy breakage with stride checking)

2013-04-09 Thread mark florisson
On 9 April 2013 15:47, Stefan Behnel wrote: > mark florisson, 09.04.2013 16:32: > > On 9 April 2013 14:55, Nikita Nemkin wrote: > >> One alternative for code reuse in large Cython projects > >> could be packaging multiple modules into one shared library. > > > > We have 'include'! :) Seriously th

Re: [Cython] Shared Cython runtime (was: Upcoming cython/numpy breakage with stride checking)

2013-04-09 Thread Stefan Behnel
mark florisson, 09.04.2013 16:32: > On 9 April 2013 14:55, Nikita Nemkin wrote: >> One alternative for code reuse in large Cython projects >> could be packaging multiple modules into one shared library. > > We have 'include'! :) Seriously though, that wouldn't work well with the > import mechanism

Re: [Cython] Shared Cython runtime (was: Upcoming cython/numpy breakage with stride checking)

2013-04-09 Thread Nathaniel Smith
On Tue, Apr 9, 2013 at 3:32 PM, mark florisson wrote: > On 9 April 2013 14:55, Nikita Nemkin wrote: >> One alternative for code reuse in large Cython projects >> could be packaging multiple modules into one shared library. > > We have 'include'! :) Seriously though, that wouldn't work well with t

Re: [Cython] Shared Cython runtime (was: Upcoming cython/numpy breakage with stride checking)

2013-04-09 Thread mark florisson
On 9 April 2013 14:55, Nikita Nemkin wrote: > On Tue, 09 Apr 2013 19:33:20 +0600, mark florisson < > markflorisso...@gmail.com> wrote: > >> On 9 April 2013 14:24, Stefan Behnel wrote: >> >> mark florisson, 09.04.2013 15:13: >>> > On 9 April 2013 14:11, Stefan Behnel wrote: >>> >> Nathaniel Smit

Re: [Cython] Shared Cython runtime (was: Upcoming cython/numpy breakage with stride checking)

2013-04-09 Thread Nathaniel Smith
On Tue, Apr 9, 2013 at 3:15 PM, Stefan Behnel wrote: > Ok, got it now. That solves the distribution problem, assuming that all > installed runtimes with a given version are equivalent. Basically, we'd > move the code out and then cimport the stuff back that we need, which would > then let the firs

Re: [Cython] Shared Cython runtime (was: Upcoming cython/numpy breakage with stride checking)

2013-04-09 Thread Stefan Behnel
Nathaniel Smith, 09.04.2013 15:46: > On Tue, Apr 9, 2013 at 2:11 PM, Stefan Behnel wrote: >> Nathaniel Smith, 09.04.2013 15:00: >>> On 9 Apr 2013 13:50, "Stefan Behnel" wrote: Nathaniel Smith, 09.04.2013 14:25: There's also the problem of dependency hell and getting rid of old modules >>>

Re: [Cython] Shared Cython runtime (was: Upcoming cython/numpy breakage with stride checking)

2013-04-09 Thread mark florisson
On 9 April 2013 14:46, Nathaniel Smith wrote: > On Tue, Apr 9, 2013 at 2:11 PM, Stefan Behnel wrote: > > Nathaniel Smith, 09.04.2013 15:00: > >> On 9 Apr 2013 13:50, "Stefan Behnel" wrote: > >>> Nathaniel Smith, 09.04.2013 14:25: > >>> There's also the problem of dependency hell and getting rid

Re: [Cython] Shared Cython runtime (was: Upcoming cython/numpy breakage with stride checking)

2013-04-09 Thread Nikita Nemkin
On Tue, 09 Apr 2013 19:33:20 +0600, mark florisson wrote: On 9 April 2013 14:24, Stefan Behnel wrote: mark florisson, 09.04.2013 15:13: > On 9 April 2013 14:11, Stefan Behnel wrote: >> Nathaniel Smith, 09.04.2013 15:00: >>> On 9 Apr 2013 13:50, "Stefan Behnel" wrote: There's also the pr

Re: [Cython] Shared Cython runtime (was: Upcoming cython/numpy breakage with stride checking)

2013-04-09 Thread Nathaniel Smith
On Tue, Apr 9, 2013 at 2:11 PM, Stefan Behnel wrote: > Nathaniel Smith, 09.04.2013 15:00: >> On 9 Apr 2013 13:50, "Stefan Behnel" wrote: >>> Nathaniel Smith, 09.04.2013 14:25: >>> There's also the problem of dependency hell and getting rid of old modules >>> once they are no longer used on the use

Re: [Cython] Shared Cython runtime (was: Upcoming cython/numpy breakage with stride checking)

2013-04-09 Thread mark florisson
On 9 April 2013 14:35, mark florisson wrote: > > > On 9 April 2013 14:33, mark florisson wrote: > >> >> >> On 9 April 2013 14:24, Stefan Behnel wrote: >> >>> mark florisson, 09.04.2013 15:13: >>> > On 9 April 2013 14:11, Stefan Behnel wrote: >>> >> Nathaniel Smith, 09.04.2013 15:00: >>> >>> On

Re: [Cython] Shared Cython runtime (was: Upcoming cython/numpy breakage with stride checking)

2013-04-09 Thread mark florisson
On 9 April 2013 14:33, mark florisson wrote: > > > On 9 April 2013 14:24, Stefan Behnel wrote: > >> mark florisson, 09.04.2013 15:13: >> > On 9 April 2013 14:11, Stefan Behnel wrote: >> >> Nathaniel Smith, 09.04.2013 15:00: >> >>> On 9 Apr 2013 13:50, "Stefan Behnel" wrote: >> There's also

Re: [Cython] Shared Cython runtime (was: Upcoming cython/numpy breakage with stride checking)

2013-04-09 Thread mark florisson
On 9 April 2013 14:24, Stefan Behnel wrote: > mark florisson, 09.04.2013 15:13: > > On 9 April 2013 14:11, Stefan Behnel wrote: > >> Nathaniel Smith, 09.04.2013 15:00: > >>> On 9 Apr 2013 13:50, "Stefan Behnel" wrote: > There's also the problem of dependency hell and getting rid of old > >>>

Re: [Cython] Shared Cython runtime (was: Upcoming cython/numpy breakage with stride checking)

2013-04-09 Thread Stefan Behnel
mark florisson, 09.04.2013 15:13: > On 9 April 2013 14:11, Stefan Behnel wrote: >> Nathaniel Smith, 09.04.2013 15:00: >>> On 9 Apr 2013 13:50, "Stefan Behnel" wrote: There's also the problem of dependency hell and getting rid of old modules once they are no longer used on the user si

Re: [Cython] Shared Cython runtime (was: Upcoming cython/numpy breakage with stride checking)

2013-04-09 Thread mark florisson
On 9 April 2013 14:11, Stefan Behnel wrote: > Nathaniel Smith, 09.04.2013 15:00: > > On 9 Apr 2013 13:50, "Stefan Behnel" wrote: > >> Nathaniel Smith, 09.04.2013 14:25: > >> There's also the problem of dependency hell and getting rid of old > modules > >> once they are no longer used on the user

Re: [Cython] Shared Cython runtime (was: Upcoming cython/numpy breakage with stride checking)

2013-04-09 Thread Stefan Behnel
Nathaniel Smith, 09.04.2013 15:00: > On 9 Apr 2013 13:50, "Stefan Behnel" wrote: >> Nathaniel Smith, 09.04.2013 14:25: >> There's also the problem of dependency hell and getting rid of old modules >> once they are no longer used on the user side. And also, how to get them >> there in the first plac

Re: [Cython] Shared Cython runtime (was: Upcoming cython/numpy breakage with stride checking)

2013-04-09 Thread Nathaniel Smith
On 9 Apr 2013 13:50, "Stefan Behnel" wrote: > > Nathaniel Smith, 09.04.2013 14:25: > > On 9 Apr 2013 13:05, "mark florisson" wrote: > >> However, the memoryview class is duplicated in every cython module, which > >> means a memoryview object from another module will fail this check. This is > >> a

Re: [Cython] Shared Cython runtime (was: Upcoming cython/numpy breakage with stride checking)

2013-04-09 Thread Stefan Behnel
Nathaniel Smith, 09.04.2013 14:25: > On 9 Apr 2013 13:05, "mark florisson" wrote: >> However, the memoryview class is duplicated in every cython module, which >> means a memoryview object from another module will fail this check. This is >> a general problem in Cython that could be worked around fo

[Cython] Shared Cython runtime (was: Upcoming cython/numpy breakage with stride checking)

2013-04-09 Thread Nathaniel Smith
On 9 Apr 2013 13:05, "mark florisson" wrote: > However, the memoryview class is duplicated in every cython module, which means a memoryview object from another module will fail this check. This is a general problem in Cython that could be worked around for memoryviews, but in general the lack of a

Re: [Cython] Upcoming cython/numpy breakage with stride checking

2013-04-09 Thread mark florisson
On 9 April 2013 13:09, mark florisson wrote: > > > > On 8 April 2013 13:04, Sebastian Berg wrote: > >> On Mon, 2013-04-08 at 12:31 +0200, Dag Sverre Seljebotn wrote: >> > On 04/08/2013 09:59 AM, Sebastian Berg wrote: >> > > On Mon, 2013-04-08 at 08:42 +0200, Dag Sverre Seljebotn wrote: >> > >> O

Re: [Cython] Upcoming cython/numpy breakage with stride checking

2013-04-09 Thread mark florisson
On 8 April 2013 13:04, Sebastian Berg wrote: > On Mon, 2013-04-08 at 12:31 +0200, Dag Sverre Seljebotn wrote: > > On 04/08/2013 09:59 AM, Sebastian Berg wrote: > > > On Mon, 2013-04-08 at 08:42 +0200, Dag Sverre Seljebotn wrote: > > >> On 04/06/2013 04:19 PM, Nathaniel Smith wrote: > > >>> Hi all

Re: [Cython] Upcoming cython/numpy breakage with stride checking

2013-04-09 Thread mark florisson
On 9 April 2013 02:04, Dave Hirschfeld wrote: > Dag Sverre Seljebotn writes: > > > > > cdef np.ndarray[double, mode='fortran'] arr > > > > that relies on PEP 3118 contiguous-flags and I did no checking myself. > > Lots of Cython code does this instead of memoryviews (I still write my > > own cod