Re: [Python-Dev] Choosing a best practice solution for Python/extension modules

On Mon, Feb 23, 2009 at 13:23, Nick Coghlan wrote: > Brett Cannon wrote: > > Well, neither do I as your proposed approach below is what I do for > > warnings. > > It's possible I actually had test_warnings.py open in another window > while writing that example function... ;) > > As Steven said, y

Re: [Python-Dev] Choosing a best practice solution for Python/extension modules

Brett Cannon wrote: > Well, neither do I as your proposed approach below is what I do for > warnings. It's possible I actually had test_warnings.py open in another window while writing that example function... ;) As Steven said, your concerns are precisely why I'm suggesting hiding this in a help

Re: [Python-Dev] Choosing a best practice solution for Python/extension modules

On Mon, Feb 23, 2009 at 04:02, Nick Coghlan wrote: > For example, a version that allows any number of extension modules to be > suppressed when importing a module (defaulting to the Foo/_Foo naming): > > import sys > def import_python_only(mod_name, *ext_names): >if not ext_names: >e

Re: [Python-Dev] Choosing a best practice solution for Python/extension modules

On Mon, Feb 23, 2009 at 04:02, Nick Coghlan wrote: > Brett Cannon wrote: > > I don't want to move it because this isn't some idea for a new feature > > that may or may not be useful; this isn't an "idea", it's needed. > > It is needed, but it's only really needed in the test suite. The > "sys.mod

Re: [Python-Dev] Choosing a best practice solution for Python/extension modules

On Sun, Feb 22, 2009 at 22:41, Aahz wrote: > On Sun, Feb 22, 2009, Brett Cannon wrote: > > On Sat, Feb 21, 2009 at 20:12, Aahz wrote: > >> On Sat, Feb 21, 2009, Brett Cannon wrote: > >>> On Sat, Feb 21, 2009 at 15:46, Aahz wrote: > On Sat, Feb 21, 2009, Brett Cannon wrote: > > > >

Re: [Python-Dev] Choosing a best practice solution for Python/extension modules

Brett Cannon wrote: > I don't want to move it because this isn't some idea for a new feature > that may or may not be useful; this isn't an "idea", it's needed. It is needed, but it's only really needed in the test suite. The "sys.modules hackery" needed to get a Python-only version using the exis

Re: [Python-Dev] Choosing a best practice solution for Python/extension modules

On Sun, Feb 22, 2009, Brett Cannon wrote: > On Sat, Feb 21, 2009 at 20:12, Aahz wrote: >> On Sat, Feb 21, 2009, Brett Cannon wrote: >>> On Sat, Feb 21, 2009 at 15:46, Aahz wrote: On Sat, Feb 21, 2009, Brett Cannon wrote: > > I am seeing two approaches emerging. One is where pickle co

Re: [Python-Dev] Choosing a best practice solution for Python/extension modules

On Sun, Feb 22, 2009 at 10:29, Michael Foord wrote: > Steven Bethard wrote: > >> On Fri, Feb 20, 2009 at 1:45 PM, Brett Cannon wrote: >> >> >>> But there is another issue with this: the pure Python code will never >>> call >>> the extension code because the globals will be bound to _pypickle and

Re: [Python-Dev] Choosing a best practice solution for Python/extension modules

On Sat, Feb 21, 2009 at 20:12, Aahz wrote: > On Sat, Feb 21, 2009, Brett Cannon wrote: > > On Sat, Feb 21, 2009 at 15:46, Aahz wrote: > >> On Sat, Feb 21, 2009, Brett Cannon wrote: > >>> > >>> I am seeing two approaches emerging. One is where pickle contains all > >>> Python code and then uses s

Re: [Python-Dev] Choosing a best practice solution for Python/extension modules

Steven Bethard wrote: On Fri, Feb 20, 2009 at 1:45 PM, Brett Cannon wrote: But there is another issue with this: the pure Python code will never call the extension code because the globals will be bound to _pypickle and not _pickle. So if you have something like:: # _pypickle def A(): r

Re: [Python-Dev] Choosing a best practice solution for Python/extension modules

On Fri, Feb 20, 2009 at 1:45 PM, Brett Cannon wrote: > But there is another issue with this: the pure Python code will never call > the extension code because the globals will be bound to _pypickle and not > _pickle. So if you have something like:: > > # _pypickle > def A(): return _B() > de

Re: [Python-Dev] Choosing a best practice solution for Python/extension modules

On Feb 21, 2009, at 2:17 PM, Brett Cannon wrote: The other approach is having pickle contain code known not to be overridden by anyone, import _pypickle for stuff that may be overridden, and then import _pickle for whatever is available. This approach has the perk of using a standard practic

Re: [Python-Dev] Choosing a best practice solution for Python/extension modules

On Sat, Feb 21, 2009, Brett Cannon wrote: > On Sat, Feb 21, 2009 at 15:46, Aahz wrote: >> On Sat, Feb 21, 2009, Brett Cannon wrote: >>> >>> I am seeing two approaches emerging. One is where pickle contains all >>> Python code and then uses something like use_extension to make sure >>> the original

Re: [Python-Dev] Choosing a best practice solution for Python/extension modules

On Sat, Feb 21, 2009 at 15:46, Aahz wrote: > On Sat, Feb 21, 2009, Brett Cannon wrote: > > > > I am seeing two approaches emerging. One is where pickle contains all > > Python code and then uses something like use_extension to make sure > > the original Python objects are still reachable at some

Re: [Python-Dev] Choosing a best practice solution for Python/extension modules

So go ahead and tear this apart so that we can hopefully reach a consensus that makes sense so that at least testing can easily be done. If I was developing an application and wanted to deal with two different versions of the same library, I would simply make sure that the version I

Re: [Python-Dev] Choosing a best practice solution for Python/extension modules

On Sat, Feb 21, 2009, Brett Cannon wrote: > > I am seeing two approaches emerging. One is where pickle contains all > Python code and then uses something like use_extension to make sure > the original Python objects are still reachable at some point. This > has the drawback that you have to use som

Re: [Python-Dev] Choosing a best practice solution for Python/extension modules

Brett Cannon wrote: > > > On Sat, Feb 21, 2009 at 11:43, > wrote: > > On 07:07 pm, br...@python.org wrote: > > On Sat, Feb 21, 2009 at 09:17, Jean-Paul Calderone > mailto:exar...@divmod.com>>wrote: > > > B

Re: [Python-Dev] Choosing a best practice solution for Python/extension modules

On Sat, Feb 21, 2009 at 11:32, Jean-Paul Calderone wrote: > On Sat, 21 Feb 2009 11:07:07 -0800, Brett Cannon wrote: > >> On Sat, Feb 21, 2009 at 09:17, Jean-Paul Calderone > >wrote: >> >> On Fri, 20 Feb 2009 13:45:26 -0800, Brett Cannon >>> wrote: >>> >>> On Fri, Feb 20, 2009 at 12:53, Aahz w

Re: [Python-Dev] Choosing a best practice solution for Python/extension modules

On Sat, Feb 21, 2009 at 11:43, wrote: > On 07:07 pm, br...@python.org wrote: > >> On Sat, Feb 21, 2009 at 09:17, Jean-Paul Calderone > >wrote: >> > > But there is another issue with this: the pure Python code will never call the extension code because the globals will be bound to _pypickle

Re: [Python-Dev] Choosing a best practice solution for Python/extension modules

On Sat, 21 Feb 2009 11:07:07 -0800, Brett Cannon wrote: On Sat, Feb 21, 2009 at 09:17, Jean-Paul Calderone wrote: On Fri, 20 Feb 2009 13:45:26 -0800, Brett Cannon wrote: On Fri, Feb 20, 2009 at 12:53, Aahz wrote: On Fri, Feb 20, 2009, Brett Cannon wrote: > On Fri, Feb 20, 2009 at 12:37,

Re: [Python-Dev] Choosing a best practice solution for Python/extension modules

On 07:07 pm, br...@python.org wrote: On Sat, Feb 21, 2009 at 09:17, Jean-Paul Calderone wrote: But there is another issue with this: the pure Python code will never call the extension code because the globals will be bound to _pypickle and not _pickle. So if you have something like:: # _

Re: [Python-Dev] Choosing a best practice solution for Python/extension modules

On Sat, Feb 21, 2009 at 09:17, Jean-Paul Calderone wrote: > On Fri, 20 Feb 2009 13:45:26 -0800, Brett Cannon wrote: > >> On Fri, Feb 20, 2009 at 12:53, Aahz wrote: >> >> On Fri, Feb 20, 2009, Brett Cannon wrote: >>> > On Fri, Feb 20, 2009 at 12:37, Brett Cannon wrote: >>> >> On Fri, Feb 20, 20

Re: [Python-Dev] Choosing a best practice solution for Python/extension modules

On Fri, 20 Feb 2009 13:45:26 -0800, Brett Cannon wrote: On Fri, Feb 20, 2009 at 12:53, Aahz wrote: On Fri, Feb 20, 2009, Brett Cannon wrote: > On Fri, Feb 20, 2009 at 12:37, Brett Cannon wrote: >> On Fri, Feb 20, 2009 at 12:31, Daniel Stutzbach < >> dan...@stutzbachenterprises.com> wrote: >>

Re: [Python-Dev] Choosing a best practice solution for Python/extension modules

Daniel Stutzbach wrote: No, I'm afraid Brett is quite right. Globals are looked up when the function is executed, true, but they are looked up within the module that defined the function. I was thinking you could fix that by going over the imported functions and stuffing the current globals

Re: [Python-Dev] Choosing a best practice solution for Python/extension modules

Daniel Stutzbach wrote: > On Fri, Feb 20, 2009 at 5:27 PM, Nick Coghlan > wrote: > > Brett Cannon wrote: > > If you import pickle and call pickle.A() you will get -13 which is not > > what you are after. > > Ah, you may want to think about that a bit mo

Re: [Python-Dev] Choosing a best practice solution for Python/extension modules

On Fri, Feb 20, 2009 at 5:27 PM, Nick Coghlan wrote: > Brett Cannon wrote: > > If you import pickle and call pickle.A() you will get -13 which is not > > what you are after. > > Ah, you may want to think about that a bit more. There's a reason > globals are looked up when they're used rather than

Re: [Python-Dev] Choosing a best practice solution for Python/extension modules

Brett Cannon wrote: > But there is another issue with this: the pure Python code will never > call the extension code because the globals will be bound to _pypickle > and not _pickle. So if you have something like:: > > # _pypickle > def A(): return _B() > def _B(): return -13 > > # _pick

Re: [Python-Dev] Choosing a best practice solution for Python/extension modules

Brett Cannon wrote: So while this alleviates the worry above, it does mean that anything that gets rewritten needs to have a name that does not lead with an underscore for this to work. You can use an __all__ list to explicitly say what is to be exported. -- Greg _

Re: [Python-Dev] Choosing a best practice solution for Python/extension modules

On Fri, Feb 20, 2009 at 12:53, Aahz wrote: > On Fri, Feb 20, 2009, Brett Cannon wrote: > > On Fri, Feb 20, 2009 at 12:37, Brett Cannon wrote: > >> On Fri, Feb 20, 2009 at 12:31, Daniel Stutzbach < > >> dan...@stutzbachenterprises.com> wrote: > >>> > >>> A slight change would make it work for mod

Re: [Python-Dev] Choosing a best practice solution for Python/extension modules

On Fri, Feb 20, 2009, Brett Cannon wrote: > On Fri, Feb 20, 2009 at 12:37, Brett Cannon wrote: >> On Fri, Feb 20, 2009 at 12:31, Daniel Stutzbach < >> dan...@stutzbachenterprises.com> wrote: >>> >>> A slight change would make it work for modules where only key functions >>> have been rewritten. F

Re: [Python-Dev] Choosing a best practice solution for Python/extension modules

Brett Cannon wrote: On Fri, Feb 20, 2009 at 12:31, Daniel Stutzbach > wrote: On Fri, Feb 20, 2009 at 1:44 PM, Brett Cannon mailto:br...@python.org>> wrote: Now, from what I can tell, Antoine is suggesting having _pyio and a _io and

Re: [Python-Dev] Choosing a best practice solution for Python/extension modules

On Fri, Feb 20, 2009 at 12:37, Brett Cannon wrote: > > > On Fri, Feb 20, 2009 at 12:31, Daniel Stutzbach < > dan...@stutzbachenterprises.com> wrote: > >> On Fri, Feb 20, 2009 at 1:44 PM, Brett Cannon wrote: >> >>> Now, from what I can tell, Antoine is suggesting having _pyio and a _io >>> and th

Re: [Python-Dev] Choosing a best practice solution for Python/extension modules

On Fri, Feb 20, 2009 at 12:31, Daniel Stutzbach < dan...@stutzbachenterprises.com> wrote: > On Fri, Feb 20, 2009 at 1:44 PM, Brett Cannon wrote: > >> Now, from what I can tell, Antoine is suggesting having _pyio and a _io >> and then io is simply: >> >> try: from _io import * >> except Im

Re: [Python-Dev] Choosing a best practice solution for Python/extension modules

On Fri, Feb 20, 2009 at 1:44 PM, Brett Cannon wrote: > Now, from what I can tell, Antoine is suggesting having _pyio and a _io and > then io is simply: > > try: from _io import * > except ImportError: from _pyio import * > > That works for testing as you can then have test classes have an

[Python-Dev] Choosing a best practice solution for Python/extension modules

With io getting rewritten as an extension module, I think it's time to try to come up with a good best practice scenario for how to be able to control when a module uses a pure Python implementation and when it uses extension module optimizations. This is really only important for testing as if th