Re: [Cython] CEP1000: Native dispatch through callables

2012-04-17 Thread Greg Ewing
Dag Sverre Seljebotn wrote: if you use Python bytes in a dict in sys.modules['_nativecall'], the bytes objects could be deallocated before callables containing the interned string -- unless you Py_INCREF once too many, I don't understand that. Is there some reason that refcounting of the inte

Re: [Cython] CEP1000: Native dispatch through callables

2012-04-17 Thread Nathaniel Smith
On Tue, Apr 17, 2012 at 8:07 PM, Dag Sverre Seljebotn wrote: > > > Nathaniel Smith wrote: > >>On Tue, Apr 17, 2012 at 3:34 PM, Dag Sverre Seljebotn >> wrote: >>> On 04/17/2012 04:20 PM, Nathaniel Smith wrote: Since you've set this up... I have a suggestion for something that >>may be wo

Re: [Cython] CEP1000: Native dispatch through callables

2012-04-17 Thread Dag Sverre Seljebotn
Nathaniel Smith wrote: >On Tue, Apr 17, 2012 at 3:34 PM, Dag Sverre Seljebotn > wrote: >> On 04/17/2012 04:20 PM, Nathaniel Smith wrote: >>> Since you've set this up... I have a suggestion for something that >may >>> be worth trying, though I've hesitated to propose it seriously. And >>> that i

Re: [Cython] CEP1000: Native dispatch through callables

2012-04-17 Thread Stefan Behnel
Dag Sverre Seljebotn, 17.04.2012 15:20: > On 04/17/2012 03:16 PM, Stefan Behnel wrote: >> Dag Sverre Seljebotn, 17.04.2012 14:53: >>> Is bytes a vararg object or does it wrap a char*? >> >> The data is stored internally in all CPython versions. Note that access to >> it may not be efficient in othe

Re: [Cython] CEP1000: Native dispatch through callables

2012-04-17 Thread Nathaniel Smith
On Tue, Apr 17, 2012 at 3:34 PM, Dag Sverre Seljebotn wrote: > On 04/17/2012 04:20 PM, Nathaniel Smith wrote: >> Since you've set this up... I have a suggestion for something that may >> be worth trying, though I've hesitated to propose it seriously. And >> that is, an API where instead of scannin

Re: [Cython] CEP1000: Native dispatch through callables

2012-04-17 Thread Dag Sverre Seljebotn
On 04/17/2012 04:20 PM, Nathaniel Smith wrote: On Tue, Apr 17, 2012 at 1:24 PM, Dag Sverre Seljebotn wrote: OK, here's the benchmark code I've written: https://github.com/dagss/cep1000 This is great! Assumptions etc.: - (Very) warm cache case is tested - I compile and link libmycall

Re: [Cython] CEP1000: Native dispatch through callables

2012-04-17 Thread Nathaniel Smith
On Tue, Apr 17, 2012 at 1:24 PM, Dag Sverre Seljebotn wrote: > OK, here's the benchmark code I've written: > > https://github.com/dagss/cep1000 This is great! > Assumptions etc.: > >  - (Very) warm cache case is tested > >  - I compile and link libmycallable.so, libmycaller.so and ./bench; with

Re: [Cython] CEP1000: Native dispatch through callables

2012-04-17 Thread Dag Sverre Seljebotn
On 04/17/2012 03:16 PM, Stefan Behnel wrote: Dag Sverre Seljebotn, 17.04.2012 14:53: Is bytes a vararg object or does it wrap a char*? The data is stored internally in all CPython versions. Note that access to it may not be efficient in other Python implementations, but at least PyPy would als

Re: [Cython] CEP1000: Native dispatch through callables

2012-04-17 Thread Stefan Behnel
Dag Sverre Seljebotn, 17.04.2012 14:53: > Is bytes a vararg object or does it wrap a char*? The data is stored internally in all CPython versions. Note that access to it may not be efficient in other Python implementations, but at least PyPy would also have a problem with providing a non-reference

Re: [Cython] CEP1000: Native dispatch through callables

2012-04-17 Thread Nathaniel Smith
On Tue, Apr 17, 2012 at 1:53 PM, Dag Sverre Seljebotn wrote: > On 04/17/2012 02:40 PM, Nathaniel Smith wrote: >> >> On Tue, Apr 17, 2012 at 12:54 PM, Dag Sverre Seljebotn >>  wrote: >>> >>> I don't believe doing interning right without a common dependency .so is >>> all >>> that easy. I'd love to

Re: [Cython] CEP1000: Native dispatch through callables

2012-04-17 Thread Stefan Behnel
Dag Sverre Seljebotn, 17.04.2012 14:53: > On 04/17/2012 02:40 PM, Nathaniel Smith wrote: >> On Tue, Apr 17, 2012 at 12:54 PM, Dag Sverre Seljebotn wrote: >>> I don't believe doing interning right without a common dependency .so is >>> all >>> that easy. I'd love to see a concrete spec for it (e.g.,

Re: [Cython] CEP1000: Native dispatch through callables

2012-04-17 Thread Dag Sverre Seljebotn
On 04/17/2012 02:40 PM, Nathaniel Smith wrote: On Tue, Apr 17, 2012 at 12:54 PM, Dag Sverre Seljebotn wrote: I don't believe doing interning right without a common dependency .so is all that easy. I'd love to see a concrete spec for it (e.g., if you use Python bytes in a dict in sys.modules['_

Re: [Cython] CEP1000: Native dispatch through callables

2012-04-17 Thread Dag Sverre Seljebotn
On 04/17/2012 02:36 PM, Dag Sverre Seljebotn wrote: On 04/17/2012 02:24 PM, Dag Sverre Seljebotn wrote: On 04/13/2012 12:11 AM, Dag Sverre Seljebotn wrote: Travis Oliphant recently raised the issue on the NumPy list of what mechanisms to use to box native functions produced by his Numba so that

Re: [Cython] CEP1000: Native dispatch through callables

2012-04-17 Thread Nathaniel Smith
On Tue, Apr 17, 2012 at 12:54 PM, Dag Sverre Seljebotn wrote: > I don't believe doing interning right without a common dependency .so is all > that easy. I'd love to see a concrete spec for it (e.g., if you use Python > bytes in a dict in sys.modules['_nativecall'], the bytes objects could be > de

Re: [Cython] CEP1000: Native dispatch through callables

2012-04-17 Thread Dag Sverre Seljebotn
On 04/17/2012 02:24 PM, Dag Sverre Seljebotn wrote: On 04/13/2012 12:11 AM, Dag Sverre Seljebotn wrote: Travis Oliphant recently raised the issue on the NumPy list of what mechanisms to use to box native functions produced by his Numba so that SciPy functions can call it, e.g. (I'm making the nu

Re: [Cython] CEP1000: Native dispatch through callables

2012-04-17 Thread Dag Sverre Seljebotn
On 04/13/2012 12:11 AM, Dag Sverre Seljebotn wrote: Travis Oliphant recently raised the issue on the NumPy list of what mechanisms to use to box native functions produced by his Numba so that SciPy functions can call it, e.g. (I'm making the numba part up): @numba # Compiles function using LLVM

Re: [Cython] CEP1000: Native dispatch through callables

2012-04-17 Thread Dag Sverre Seljebotn
On 04/15/2012 10:48 AM, Nathaniel Smith wrote: On Sun, Apr 15, 2012 at 9:15 AM, Dag Sverre Seljebotn wrote: Do you really think it complicates the spec? SHA-1 is pretty standard, and Python ships with hashlib (the hashing part isn't performance critical). I prefer hashing to string-interning