Re: [Cython] Cannot cythonize subclasses of setuptools.extension._Extension

2016-04-16 Thread Erik Bray
On Apr 14, 2016 21:07, "Manuel Nuno Melo" 
wrote:
>
> Our need to control cythonization comes from the fact that we implement
cython as a lazy and optional dependency. Lazy in the sense that we delay
as much as possible cythonization so that setuptools or pip have time to
install cython, if needed. Optional because we distribute both .pyx and
cythonized .c files, and decide on which to use based on user flags.
>
> We, therefore, need an Extension class that only cythonizes if we decide
to.
>
> Thanks for the feedback,
> Manel

I have a solution for exactly this in astropy_helpers which makes it
possible, for example, to pull in Cython via setup_requires, but only if
it's needed.

Remind me on Monday and I can point out how it works.

> On Apr 14, 2016 8:17 PM, "Matthew Brett"  wrote:
> >
> > On Thu, Apr 14, 2016 at 6:08 AM, Erik Bray 
wrote:
> > > On Wed, Apr 13, 2016 at 9:35 PM, Manuel Nuno Melo
> > >  wrote:
> > >> Hello devs,
> > >>
> > >> I'm developing the setup.py for a scientific package, MDAnalysis
(see PR
> > >> #799). We depend on distutils and setuptool. Namely, we use
> > >> setuptools.extension.Extension class for our extensions.
> > >>
> > >> Some older versions of setuptools (<18.0) do filename cythonization
> > >> themselves upon initialization of the Extension object.
> > >>
> > >> Because we want to control name cythonization ourselves I try to
directly
> > >> use distutils.extension.Extension, which has none of setuptools'
> > >> cythonization. However, this doesn't work because setuptools patches
> > >> distutils, so that distutils.extension.Extension effectively becomes
> > >> setuptools.extension.Extension.
> > >
> > > I'm wondering what it is specifically you need to do in your
> > > subclass--might it still be possible to do with a subclass of the
> > > setuptools Extension?  Not saying I disagree with the overall idea,
> > > but I also wonder if there isn't a better way.
> >
> > I know this is a terrible and ugly hack, but the projects I work in
> > have a 'fake_pyrex' directory, that fools setuptools into thinking
> > that 'pyrex' is installed, and therefore prevents it from doing the
> > .pyx -> .c filename conversions in the extension:
> >
> > https://github.com/regreg/regreg/blob/master/setup.py#L33
> >
> > Cheers,
> >
> > Matthew
> > ___
> > cython-devel mailing list
> > cython-devel@python.org
> > https://mail.python.org/mailman/listinfo/cython-devel
>
>
> ___
> cython-devel mailing list
> cython-devel@python.org
> https://mail.python.org/mailman/listinfo/cython-devel
>
___
cython-devel mailing list
cython-devel@python.org
https://mail.python.org/mailman/listinfo/cython-devel


Re: [Cython] Cannot cythonize subclasses of setuptools.extension._Extension

2016-04-16 Thread Manuel Nuno Melo
Hi Erik,
Please post your solution; I'm curious to see it.

Currently, we're also using setup_requires but are moving away from it
because of two main issues:

1- there's no flexibility to hook into it via setuptools.Distribution
subclassing (unless you rewrite the entire __init__);

2- (and more serious) setuptools installs setup_requires dependencies in
the local directory, not as a system-wide install. Even worse, it then puts
a path link to the current directory under the system-wide
easy_install.pth. This means that if you install from the source directory
via 'sudo ./setup.py install', you get a Cython egg downloaded into that
directory, and its path added to sys.path (!!). Needless to say this can
break in many nasty ways...

On Sat, Apr 16, 2016 at 1:10 PM, Erik Bray  wrote:

> On Apr 14, 2016 21:07, "Manuel Nuno Melo" 
> wrote:
> >
> > Our need to control cythonization comes from the fact that we implement
> cython as a lazy and optional dependency. Lazy in the sense that we delay
> as much as possible cythonization so that setuptools or pip have time to
> install cython, if needed. Optional because we distribute both .pyx and
> cythonized .c files, and decide on which to use based on user flags.
> >
> > We, therefore, need an Extension class that only cythonizes if we decide
> to.
> >
> > Thanks for the feedback,
> > Manel
>
> I have a solution for exactly this in astropy_helpers which makes it
> possible, for example, to pull in Cython via setup_requires, but only if
> it's needed.
>
> Remind me on Monday and I can point out how it works.
>
> > On Apr 14, 2016 8:17 PM, "Matthew Brett" 
> wrote:
> > >
> > > On Thu, Apr 14, 2016 at 6:08 AM, Erik Bray 
> wrote:
> > > > On Wed, Apr 13, 2016 at 9:35 PM, Manuel Nuno Melo
> > > >  wrote:
> > > >> Hello devs,
> > > >>
> > > >> I'm developing the setup.py for a scientific package, MDAnalysis
> (see PR
> > > >> #799). We depend on distutils and setuptool. Namely, we use
> > > >> setuptools.extension.Extension class for our extensions.
> > > >>
> > > >> Some older versions of setuptools (<18.0) do filename cythonization
> > > >> themselves upon initialization of the Extension object.
> > > >>
> > > >> Because we want to control name cythonization ourselves I try to
> directly
> > > >> use distutils.extension.Extension, which has none of setuptools'
> > > >> cythonization. However, this doesn't work because setuptools patches
> > > >> distutils, so that distutils.extension.Extension effectively becomes
> > > >> setuptools.extension.Extension.
> > > >
> > > > I'm wondering what it is specifically you need to do in your
> > > > subclass--might it still be possible to do with a subclass of the
> > > > setuptools Extension?  Not saying I disagree with the overall idea,
> > > > but I also wonder if there isn't a better way.
> > >
> > > I know this is a terrible and ugly hack, but the projects I work in
> > > have a 'fake_pyrex' directory, that fools setuptools into thinking
> > > that 'pyrex' is installed, and therefore prevents it from doing the
> > > .pyx -> .c filename conversions in the extension:
> > >
> > > https://github.com/regreg/regreg/blob/master/setup.py#L33
> > >
> > > Cheers,
> > >
> > > Matthew
> > > ___
> > > cython-devel mailing list
> > > cython-devel@python.org
> > > https://mail.python.org/mailman/listinfo/cython-devel
> >
> >
> > ___
> > cython-devel mailing list
> > cython-devel@python.org
> > https://mail.python.org/mailman/listinfo/cython-devel
> >
>
> ___
> cython-devel mailing list
> cython-devel@python.org
> https://mail.python.org/mailman/listinfo/cython-devel
>
>
___
cython-devel mailing list
cython-devel@python.org
https://mail.python.org/mailman/listinfo/cython-devel