[Python-Dev] Re: PEP 659: Specializing Adaptive Interpreter
To potentially help provide a little bit of additional detail around our approach I've spent some time writing up our internal details of the shadow byte code implementation, and landed that in our Cinder repo here: https://github.com/facebookincubator/cinder/blob/cinder/3.8/CinderDoc/shadowcode.rst. That might at least spark discussion or ideas about possible internal implementation details or things which could be different/more efficient in our implementation. I've also had a version of it against 3.10 going for a while (as internally we're still at 3.8) and I've updated it to a relatively recent merge of 3.11 main. I've pushed the latest version of that here here: https://github.com/DinoV/cpython/tree/shadowcode_rebase_2021_05_12. The 3.11 version obviously isn't as battle tested as what we've been running in production for some time now but it pretty much the same. It is missing our improved global caching which uses dictionary watches though. And it is a rather large PR (almost 7k lines) but over 1/3rd of that is the test cases. Also just to inform the discussion around potential performance benefits, here's how that alone is currently benchmarking versus the base commit: cpython_310_opt_rig.json Performance version: 1.0.1 Report on Linux-5.2.9-229_fbk15_hardened_4185_g357f49b36602-x86_64-with-glibc2.28 Number of logical CPUs: 48 Start date: 2021-05-17 21:57:08.095822 End date: 2021-05-17 22:40:33.374232 cpython_ghdino_opt_rig.json === Performance version: 1.0.1 Report on Linux-5.2.9-229_fbk15_hardened_4185_g357f49b36602-x86_64-with-glibc2.28 Number of logical CPUs: 48 Start date: 2021-05-21 17:25:24.410644 End date: 2021-05-21 18:02:53.524314 +-+--+-+--+---+ | Benchmark | cpython_310_opt_rig.json | cpython_ghdino_opt_rig.json | Change | Significance | +=+==+=+==+===+ | 2to3| 498 ms | 459 ms | 1.09x faster | Significant (t=15.60) | +-+--+-+--+---+ | chameleon | 13.4 ms | 12.6 ms | 1.07x faster | Significant (t=11.10) | +-+--+-+--+---+ | chaos | 163 ms | 135 ms | 1.21x faster | Significant (t=33.07) | +-+--+-+--+---+ | crypto_pyaes| 171 ms | 147 ms | 1.16x faster | Significant (t=24.93) | +-+--+-+--+---+ | deltablue | 11.7 ms | 8.38 ms | 1.40x faster | Significant (t=70.51) | +-+--+-+--+---+ | django_template | 73.7 ms | 68.1 ms | 1.08x faster | Significant (t=13.12) | +-+--+-+--+---+ | dulwich_log | 108 ms | 98.6 ms | 1.10x faster | Significant (t=18.11) | +-+--+-+--+---+ | fannkuch| 734 ms | 731 ms | 1.00x faster | Not significant | +-+--+-+--+---+ | float | 166 ms | 140 ms | 1.18x faster | Significant (t=29.38) | +-+--+-+--+---+ | go | 345 ms | 305 ms | 1.13x faster | Significant (t=31.29) | +-+--+-+--+---+ | hexiom | 14.4 ms | 13.1 ms | 1.10x faster | Significant (t=15.95) | +-+--+-+--+---+ | json_dumps | 19.6 ms | 18.1 ms | 1.09x faster | Significant (t=13.85) | +-+--+-+--+---+ | json_loads | 37.5 us | 34.8 us
[Python-Dev] Re: PEP 659: Specializing Adaptive Interpreter
> > The Instagram team behind the Cinder project may be interested to > review Mark's work before it's merged into Python development branch. > A design PEP would be more convenient than reviewing an concrete > implementation. I can't speak for others on the Cinder team, but I would definitely be happy to help review, and a design document would be great for that. I'm certainly curious what the design is for the Microsoft implementation and how it differs from our shadow code implementation. Right now I certainly don't have enough information to know what the differences are. And of course the reason to open source Cinder was to help avoid duplication of effort as obviously there's a lot of interest in making Python faster! So if there's interest in our shadow code implementation I'd be happy to work to break it out into more reviewable pieces as well - it's got some pretty natural ways to split it up. On Thu, May 27, 2021 at 2:21 PM Victor Stinner wrote: > Hi, > > The gilectomy was mentioned earlier in the thread. This project was > created by a core dev who had the permission to push directly his work > into the development branch, but it was done on a work. > > Another optimization project example was my experimental "FAT Python" > project. The main idea was to specialize functions with some > assumptions, and check these assumptions at the function entry. I > started in a fork and then wrote 3 PEPs to propose to merge the main > changes into Python development branch: > > * PEP 509 -- Add a private version to dict > * PEP 510 -- Specialize functions with guards > * PEP 511 -- API for code transformers > > The PEP 509 was accepted since it could be used for other > optimizations: Python now uses the dictionary version to optimize > LOAD_GLOBAL. > > The two other PEPs were rejected. Even if I was very disappointed when > they were rejected, it seems like it was a wize choice since I was > never able to make Python significantly faster (like 2x faster). It > was only 10-20% faster on some micro-benchmarks. I also had the > permission to push directly, and I think that it was nice to confront > my design and ideas to the community. > > I also understand that optimizing Python is really hard and it > requires a lot of preparation work. It's hard to sell the preparation > work since it only introduces regressions and noise without any > concrete speedup. All of this work is required to implement the real > interesting optimization. Moreover, few people are earger to review a > Python fork with deep and large changes in Python internals. The work > must be re-done step by step with more "atomic" (small) changes to > have a readable Git history. > > The Instagram team behind the Cinder project may be interested to > review Mark's work before it's merged into Python development branch. > A design PEP would be more convenient than reviewing an concrete > implementation. > > Victor > ___ > Python-Dev mailing list -- python-dev@python.org > To unsubscribe send an email to python-dev-le...@python.org > https://mail.python.org/mailman3/lists/python-dev.python.org/ > Message archived at > https://mail.python.org/archives/list/python-dev@python.org/message/CS7WDQYHJN7QV6TQW3CJZ2XRQRKX2RWT/ > Code of Conduct: http://python.org/psf/codeofconduct/ > ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/VGEBI3YC5I6ESO7SKZL44A3AGNO5OLFZ/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Making Cinder more broadly available
Hi, Cinder <https://github.com/facebookincubator/cinder> is Meta’s performance-oriented version of CPython 3.8. It has been in use as the production Python behind Instagram server for years, as well as powering various other Python applications across Meta. We are interested in making the improvements in Cinder more broadly available to the Python community. As a collection of features and components built on top of CPython, we’d like to: 1. Merge the feature into CPython for core features that are highly coupled with CPython internals (after we port it over to the main branch) 2. For features that *can* work as extension modules (like Cinder JIT) - extract them as a new open source standalone pip-installable extension module. 3. To enable the pip-installable extension module, we will need to add a few extension hooks to CPython, which we intend to carve out and port over to the main branch in order to merge upstream. Please read more about our upstreaming strategy and a few Cinder features in this document: https://docs.google.com/document/d/1l8I-FDE1xrIShm9eSNJqsGmY_VanMDX5-aK_gujhYBI/edit?usp=sharing Complete source code of our CPython fork: https://github.com/facebookincubator/cinder We started filing bpo issues for a few of the pieces we want to merge upstream (links are in the google doc), and are looking forward to feedback from the community on the issues, overall approach, or any feedback on the features themselves. Cheers, Dino ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/4HLXWXJTZD2YHMBZIDBJ2RCWNDHWSUYV/ Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-Dev] Experiment an opt-in new C API for Python? (leave current API unchanged)
> > That's exactly why I dislike "New", it's like adding "Ex" or "2" to a > function name :-) > > Well, before bikeshedding on the C define name, I would prefer to see > if the overall idea of trying to push code for the new C API in the > master branch is a good idea, or if it's too early and the experiment > must continue in a fork. Rather than adding yet another pre-processor directive for this I would suggest just adding a new header file that only has the new stable API. For example it could just be "py.h" or "pyapi.h". It would have all of the definitions for the stable API. While that would involve some duplication from the existing headers, I don't think it would be such a big deal - the idea is the API won't change, methods won't be removed, and occasionally new methods will get added in a very thoughtful manner. Having it be separate will force thought and conversation about it. It would also make it very easy to look and see what exactly is in the stable API as well. There's be a pretty flat list which can be consulted, and hopefully it ends up not being super huge either. BTW, thanks for continuing to push on this Victor, it seems like great progress! On Fri, Nov 9, 2018 at 4:57 PM Victor Stinner wrote: > Le sam. 10 nov. 2018 à 01:49, Michael Selik a écrit : > >> It uses an opt-in option (Py_NEWCAPI define -- I'm not sure about the > >> name) to get the new API. The current C API is unchanged. > > > > While one can hope that this will be the only time the C API will be > revised, it may be better to number it instead of calling it "NEW". 20 > years from now, it won't feel new anymore. > > That's exactly why I dislike "New", it's like adding "Ex" or "2" to a > function name :-) > > Well, before bikeshedding on the C define name, I would prefer to see > if the overall idea of trying to push code for the new C API in the > master branch is a good idea, or if it's too early and the experiment > must continue in a fork. > > Victor > ___ > Python-Dev mailing list > Python-Dev@python.org > https://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > https://mail.python.org/mailman/options/python-dev/dinoviehland%40gmail.com > ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Register-based VM [Was: Possible performance regression]
On Tue, Feb 26, 2019 at 3:56 PM Neil Schemenauer wrote: > Right. I wonder though, could we avoid allocating the Python frame > object until we actually need it? Two situations when you need a > heap allocated frame come to mind immediately: generators that are > suspended and frames as part of a traceback. I guess > sys._getframe() is another. Any more? > > I've been thinking about that as well... I think in some ways the easy part of this is actually the easy part of this is the reification of the frame it's self. You can have a PyFrameObject which is just declared on the stack and add a new field to it which captures the address of the PyFrameObject* f (e.g. PyFrameObject **f_stackaddr). When you need to move to aheap allocated one you copy everything over as you say and update *f_stackaddr to point at the new heap address. It seems a little bit annoying with the various levels of indirection from the frame getting created in PyEval_EvalCodeEx and flowing down into _PyEval_EvalFrameDefault - so there may need to be some breakage there for certain low-level tools. I'm also a little bit worried about things which go looking at PyThreadState and might make nasty assumptions about the frames already being heap allocated. FYI IronPython does support sys._getframe(), you just need to run it with a special flag (and there are various levels - e.g. -X:Frames and -X:FullFrames, the latter which guarantees your locals are in the frame too). IronPython is more challenged here in that it always generates "safe" code from a CLR perspective and tracking the address of stack-allocated frame objects is therefore challenging (although maybe more possible now then before with various C# ref improvements). I'm not sure exactly how much this approach would get though... It seems like the frame caches are pretty effective, and a lot of the cost of them is initializing them / decref'ing the things which are still alive in them. But it doesn't seem a like a super complicated change to try out... It's actually something I'd at least like to try prototyping at some point. ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Re: Mixed Python/C debugging
Python Tools for Visual Studio has supported mixed mode debugging for a while... I'm not sure if that support would have ever ended up in VS Code Python extension so it might be a windows only solution which might not help you much. On Sun, Dec 1, 2019, 9:04 AM Skip Montanaro wrote: > Having tried comp.lang.python with no response, I turn here... > > After at least ten years away from Python's run-time interpreter & > byte code compiler, I'm getting set to familiarize myself with that > again. This will, I think, entail debugging a mixed Python/C > environment. I'm an Emacs user and am aware that GDB since 7.0 has > support for debugging at the Python code level. Is Emacs+GDB my best > bet? Are there any Python IDEs which support C-level breakpoints and > debugging? > > Thanks, > > Skip > ___ > Python-Dev mailing list -- python-dev@python.org > To unsubscribe send an email to python-dev-le...@python.org > https://mail.python.org/mailman3/lists/python-dev.python.org/ > Message archived at > https://mail.python.org/archives/list/python-dev@python.org/message/L2KBZM64MYPXIITN4UU3X6L4PZS2YRTB/ > Code of Conduct: http://python.org/psf/codeofconduct/ > ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/6YC4WHAEQH556FNFZRDVO2IIJ37GRHCG/ Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-Dev] Expression optimizations
And slightly unrelated, but just showing how bizarre floats are:
>>> x = 1e6
>>> y = x/x
>>> cmp(y, y)
0
>>> cmp(x/x, x/x)
-1
Yeah object identity checks!
From: python-dev-bounces+dinov=microsoft@python.org
[mailto:python-dev-bounces+dinov=microsoft@python.org] On Behalf Of Daniel
Stutzbach
Sent: Tuesday, February 10, 2009 12:43 PM
To: cesare.dima...@a-tono.com
Cc: Python-Dev
Subject: Re: [Python-Dev] Expression optimizations
On Tue, Feb 10, 2009 at 2:36 PM, Cesare Di Mauro
wrote:
OK, so I can make assumptions only for built-in types.
Yes, but even there you have to be careful of odd corner-cases, such as:
>>> nan = float('nan')
>>> nan < nan
False
>>> nan >= nan
False
--
Daniel Stutzbach, Ph.D.
President, Stutzbach Enterprises, LLC
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Expression optimizations
I think it's comparing based upon object identity so it may be a little
non-deterministic:
>>> x= 1e66
>>> y = x/x
>>> z = x/x
>>> cmp(y, z)
1
>>> cmp(z, y)
-1
>>>
But I may have accidently run that on IronPython though where we're assigning
ids differently :)
-Original Message-
From: Cesare Di Mauro [mailto:cesare.dima...@a-tono.com]
Sent: Tuesday, February 10, 2009 1:12 PM
To: Dino Viehland
Cc: Daniel Stutzbach; cesare.dima...@a-tono.com; Python-Dev
Subject: RE: [Python-Dev] Expression optimizations
It's bizarre enough, since I have got a different result (with Python
2.6.1, 32 bit):
>>> x = 1e6
>>> y = x/x
>>> x
inf
>>> y
nan
>>> cmp(y, y)
0
>>> cmp(x/x, x/x)
1
:D
Cesare
On Mar, Feb 10, 2009 10:02PM, Dino Viehland wrote:
> And slightly unrelated, but just showing how bizarre floats are:
>
>>>> x = 1e6
>>>> y = x/x
>>>> cmp(y, y)
> 0
>>>> cmp(x/x, x/x)
> -1
>
> Yeah object identity checks!
>
> From: python-dev-bounces+dinov=microsoft@python.org
> [mailto:python-dev-bounces+dinov=microsoft@python.org] On Behalf Of
> Daniel Stutzbach
> Sent: Tuesday, February 10, 2009 12:43 PM
> To: cesare.dima...@a-tono.com
> Cc: Python-Dev
> Subject: Re: [Python-Dev] Expression optimizations
>
> On Tue, Feb 10, 2009 at 2:36 PM, Cesare Di Mauro
> wrote:
> OK, so I can make assumptions only for built-in types.
>
> Yes, but even there you have to be careful of odd corner-cases, such as:
>
>>>> nan = float('nan')
>>>> nan < nan
> False
>>>> nan >= nan
> False
> --
> Daniel Stutzbach, Ph.D.
> President, Stutzbach Enterprises, LLC
>
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 384: Defining a Stable ABI
Dirkjan Ochtman wrote: > > It would seem to me that optimizations are likely to require data > structure changes, for exactly the kind of core data structures that > you're talking about locking down. But that's just a high-level view, > I might be wrong. > In particular I would guess that ref counting is the biggest issue here. I would think not directly exposing the field and having inc/dec ref Functions (real methods, not macros) for it would give a lot more ability to change the API in the future. It also might make it easier for alternate implementations to support the same API so some modules could work cross implementation - but I suspect that's a non-goal of this PEP :). Other fields directly accessed (via macros or otherwise) might have similar problems but they don't seem as core as ref counting. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] IronPython specific code in inspect module
Michael Foord wrote: > I have IronPython specific versions of several of these functions which > use .NET reflection and inspect could fallback to if sys.platform == > 'cli'. Would it be ok for me to add these to the inspect module? > Obviously the tests would only run on IronPython... The behaviour for > CPython would be unaffected. What about instead defining __argspec__ for built-in functions/method objects and allowing all the implementations to implement it? We could all agree to return: [ (return_type, (arg_types,...)), (return_type, (arg_types,...)), ] Then inspect can check for that attribute and support introspection on built-ins. This would be an easy feature for us to implement and it may also be for Jython as well given that we both get the power of our platforms reflection capabilities. Any platform that implements it lights up w/o new platform specific code. And maybe this needs to go to python-ideas now :) ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Indentation oddness...
Consider the code:
code = "def Foo():\n\npass\n\n "
This code is malformed in that the final indentation (2 spaces) does not agree
with the previous indentation of the pass statement (4 spaces). Or maybe it's
just fine if you take the blank lines should be ignored statement from the docs
to be true. So let's look at different ways I can consume this code.
If I use compile to compile this:
compile(code, 'foo', 'single')
I get an IndentationError: unindent does not match any outer indentation level
But if I put this in a file:
f= file('indenttest.py', 'w')
f.write(code)
f.close()
import indenttest
It imports just fine.
If I run it through the tokenize module it also tokenizes just fine:
>>> import tokenize
>>> from cStringIO import StringIO
>>> tokenize.tokenize(StringIO(code).readline)
1,0-1,3:NAME'def'
1,5-1,8:NAME'Foo'
1,8-1,9:OP '('
1,9-1,10: OP ')'
1,10-1,11: OP ':'
1,11-1,12: NEWLINE '\n'
2,0-2,1:NL '\n'
3,0-3,4:INDENT ''
3,4-3,8:NAME'pass'
3,8-3,9:NEWLINE '\n'
4,0-4,1:NL '\n'
5,0-5,0:DEDENT ''
5,0-5,0:ENDMARKER ''
And if it fails anywhere it would seem tokenization is where it should fail -
especially given that tokenize.py seems to report this error on other occasions.
And stranger still if I add a new line then it will even compile fine:
compile(code + '\n', 'foo', 'single')
Which seems strange because in either case all of the trailing lines are blank
lines and as such should basically be ignored according to the documentation.
Is there some strange reason why compile rejects what everything else agrees is
perfectly valid code?
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Indentation oddness...
Consider the code:
code = "def Foo():\n\npass\n\n "
This code is malformed in that the final indentation (2 spaces) does not agree
with the previous indentation of the pass statement (4 spaces). Or maybe it's
just fine if you take the blank lines should be ignored statement from the docs
to be true. So let's look at different ways I can consume this code.
If I use compile to compile this:
compile(code, 'foo', 'single')
I get an IndentationError: unindent does not match any outer indentation level
But if I put this in a file:
f= file('indenttest.py', 'w')
f.write(code)
f.close()
import indenttest
It imports just fine.
If I run it through the tokenize module it also tokenizes just fine:
>>> import tokenize
>>> from cStringIO import StringIO
>>> tokenize.tokenize(StringIO(code).readline)
1,0-1,3:NAME'def'
1,5-1,8:NAME'Foo'
1,8-1,9:OP '('
1,9-1,10: OP ')'
1,10-1,11: OP ':'
1,11-1,12: NEWLINE '\n'
2,0-2,1:NL '\n'
3,0-3,4:INDENT ''
3,4-3,8:NAME'pass'
3,8-3,9:NEWLINE '\n'
4,0-4,1:NL '\n'
5,0-5,0:DEDENT ''
5,0-5,0:ENDMARKER ''
And if it fails anywhere it would seem tokenization is where it should fail -
especially given that tokenize.py seems to report this error on other occasions.
And stranger still if I add a new line then it will even compile fine:
compile(code + '\n', 'foo', 'single')
Which seems strange because in either case all of the trailing lines are blank
lines and as such should basically be ignored according to the documentation.
Is there some strange reason why compile rejects what everything else agrees is
perfectly valid code?
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Indentation oddness...
> The 'single' mode, which is used for the REPL, is a bit different than > 'exec', > which is used for modules. This difference lets you insert "blank" > lines of > whitespace into a function definition without exiting the definition. > Ending > with a truly empty line does not cause the IndentationError, so the > REPL can > successfully compile the code, signaling that the user has finished > typing the > function. Sorry, I probably should have mentioned this but it repros w/ compile(..., "exec") as well: >>> code = "def Foo():\n\npass\n\n " >>> compile(code, 'foo', 'exec') Traceback (most recent call last): File "", line 1, in File "foo", line 5 IndentationError: unindent does not match any outer indentation level It also repros when passing in PyCF_DONT_IMPLY_DEDENT for flags under single and exec. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Indentation oddness...
Unfortunately my problem is the opposite one - trying to emulate what compile does for IronPython rather than just trying to make some code compile. So adding newlines doesn't help me. But this case isn't really that important - it was just a wacky corner case I ran into while trying to get other behavior right. I think I can safely ignore this one especially if it's just a bug. > -Original Message- > From: gvanros...@gmail.com [mailto:gvanros...@gmail.com] On Behalf Of > Guido van Rossum > Sent: Friday, May 29, 2009 7:20 PM > To: Dino Viehland > Cc: Robert Kern; python-dev@python.org > Subject: Re: [Python-Dev] Indentation oddness... > > I usually append some extra newlines before passing a string to > compile(). That's the usual work-around. There's probably a subtle bug > in the tokenizer when reading from a string -- if you find it, please > upload a patch to the tracker! > > --Guido > > On Fri, May 29, 2009 at 5:52 PM, Dino Viehland > wrote: > >> The 'single' mode, which is used for the REPL, is a bit different > than > >> 'exec', > >> which is used for modules. This difference lets you insert "blank" > >> lines of > >> whitespace into a function definition without exiting the definition. > >> Ending > >> with a truly empty line does not cause the IndentationError, so the > >> REPL can > >> successfully compile the code, signaling that the user has finished > >> typing the > >> function. > > > > Sorry, I probably should have mentioned this but it repros w/ > > compile(..., "exec") as well: > > > >>>> code = "def Foo():\n\npass\n\n " > >>>> compile(code, 'foo', 'exec') > > Traceback (most recent call last): > > File "", line 1, in > > File "foo", line 5 > > > > IndentationError: unindent does not match any outer indentation level > > > > It also repros when passing in PyCF_DONT_IMPLY_DEDENT for flags under > > single and exec. > > ___ > > Python-Dev mailing list > > Python-Dev@python.org > > http://mail.python.org/mailman/listinfo/python-dev > > Unsubscribe: http://mail.python.org/mailman/options/python- > dev/guido%40python.org > > > > > > -- > --Guido van Rossum (home page: http://www.python.org/~guido/) ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] syntax warnings don't go through warnings.warn?
I'm just a little surprised by this - Is there a reason why syntax warnings are
special and untrappable via warnings.warn?
>>> import warnings
>>> def mywarn(*args): print 'xx', args
...
>>> warnings.warn = mywarn
>>> compile("def f():\na = 1\nglobal a\n", "", "exec")
:3: SyntaxWarning: name 'a' is assigned to before global declaration
at 012DB410, file "", line 1>
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] syntax warnings don't go through warnings.warn?
I work on IronPython so I am asking a question about development of Python. In particular my goal is to make sure that IronPython is behaving the same as CPython. If the normal discussion list is more appropriate for these questions I'd be happy to take it there. But before I do that please consider nt.tempnam - a built-in function. This goes through the normal warning mechanism but the parser doesn't. I would assume that this would go through PyErr_WarnEx (sorry, I can't actually look at the CPython code) given that tempnam is implemented in C. Why wouldn't the parser also go through PyErr_WarnEx? And warnings.catch_warnings doesn't work w/ parse warnings either so I'm not sure what the point of bringing that up is. -Original Message- From: Matthew Wilkes [mailto:matt...@matthewwilkes.co.uk] Sent: Monday, June 01, 2009 10:29 AM To: Dino Viehland Cc: Python-Dev Subject: Re: [Python-Dev] syntax warnings don't go through warnings.warn? On 1 Jun 2009, at 17:50, Dino Viehland wrote: > I'm just a little surprised by this - Is there a reason why syntax > warnings are special and untrappable via warnings.warn? Why should this work? From the docs... "Python programmers issue warnings by calling the warn() function defined in this module. (C programmers use PyErr_WarnEx; see Exception Handling for details)." Check out the warnings.catch_warnings context manager, but if you have any further questions please direct them to the normal Python mailing list, this is for development _of_ Python only. Matthew ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] syntax warnings don't go through warnings.warn?
Ahh, ok, now I think I see... there is clearly some other mechanism that this warning is going through that we're not handling correctly. I'll try and track it down. Thanks! -Original Message- From: Matthew Wilkes [mailto:matt...@matthewwilkes.co.uk] Sent: Monday, June 01, 2009 10:49 AM To: Michael Foord Cc: Dino Viehland; Python-Dev Subject: Re: [Python-Dev] syntax warnings don't go through warnings.warn? On 1 Jun 2009, at 18:42, Michael Foord wrote: > Dino is developing Python - he's one of the core developers of > IronPython Ah, sorry, I'm bad with names, don't always pick up on who is who! > and I suspect he is asking whether this is intentional, and > IronPython should implement the same behaviour, or whether it is a > bug. The docs do seem to be clear though, warnings.warn creates a warning, but there are multiple other ways to do it, it's a convenience method. Matt ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] [IronPython] Exception for setting attributes of built-in type
Guido wrote: > I should add that this policy is also forced somewhat by the existence > of the "multiple interpreters in one address space" feature, which is > used e.g. by mod_python. This feature attempts to provide isolation > between interpreters to the point that each one can have a completely > different set of modules loaded and can be working on a totally > different application. The implementation of CPython shares built-in > types between multiple interpreters (and it wouldn't be easy to change > this); if you were able to modify a built-in type from one > interpreter, all other interpreters would see that same modification. IronPython is in the exact same boat here - we share built-in types Across multiple Python engines as well. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] [IronPython] Exception for setting attributes of built-in type
Seo wrote: > Exception for setting attributes of built-in type differs between > CPython and IronPython. This is not purely theoretical, as > zope.interface tries to set Implements declaration as __implemented__ > attribute of built-in type object, and excepts TypeError. > > Python 2.6.1 > >>> object.flag = True > TypeError: can't set attributes of built-in/extension type 'object' > > IronPython 2.6 > >>> object.flag = True > AttributeError: 'object' object has no attribute 'flag' > > I was surprised that CPython raises TypeError. Library Reference seems > to mention it here: > > exception AttributeError > Raised when an attribute reference or assignment fails. (When an > object does not support attribute references or attribute assignments > at all, TypeError is raised.) > http://docs.python.org/library/exceptions.html > > What does it mean that "an object does not support attribute > references or attribute assignments at all"? Even though Guido said we didn't have to match it I've updated IronPython so that we'll throw the same exception now. The change will be in tomorrow's source push to CodePlex. We've matched specific exceptions in the past and this one is at least documented :) ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] 2.6 object.__init__ & deling __new__
I'm updating IronPython to match CPython's behavior w/ for this issue: http://bugs.python.org/issue1683368 One thing that I've noticed is that this doesn't seem to be respecting the deletion of attributes (on 2.6.2): class x(object): pass x().__init__(2,3,4) # throws - seems right class x(object): def __new__(cls, *args): return object.__new__(cls) x().__init__(2,3,4) # doesn't throw - seems right del x.__new__ x().__init__(2,3,4) # doesn't throw - I would expect this to throw. Is this just a bug in CPython not updating whether __new__ has been defined? Or is there something that makes this behavior expected which I'm just missing? ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] 2.6 object.__init__ & deling __new__
Benjamin wrote: > There's a wonderful comment about this in typeobject.c: > This is basically the same what I've gathered from the issue description which was quite helpful. But in this case we're dealing with mutating the type object and changing whether __new__ or __init__ exist at all at runtime - and object.__new__/__init__ don't seem to be picking up on the change. I believe that the comments here w.r.t. mutability/immutability are more about whether the instances are immutable and not the type objects themselves - for example list vs tuple where tuple has __new__ but no __init__. Based upon the behavior I'm seeing it seems to me that the presence of __new__ / __init__ must be getting cached somewhere and the deletion isn't updating the cache and that's specifically what struck me as odd here. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] (try-except) conditional expression similar to (if-else) conditional (PEP 308)
On option 1 is this legal then?
x = float(string) except float('nan') if some_check() else float('inf') if
ValueError
-Original Message-
From: python-dev-bounces+dinov=microsoft@python.org
[mailto:python-dev-bounces+dinov=microsoft@python.org] On Behalf Of Nick
Coghlan
Sent: Thursday, August 06, 2009 3:48 AM
To: P.J. Eby
Cc: python-dev@python.org; Jeff McAninch
Subject: Re: [Python-Dev] (try-except) conditional expression similar to
(if-else) conditional (PEP 308)
P.J. Eby wrote:
> At 05:59 PM 8/5/2009 -0700, Raymond Hettinger wrote:
>> [Jeffrey E. McAninch, PhD]
>>> I very often want something like a try-except conditional expression
>>> similar to the if-else conditional.
>>>
>>> An example of the proposed syntax might be:
>>>x = float(string) except float('nan') or possibly
>>>x = float(string) except ValueError float('nan')
>>
>> +1 I've long wanted something like this.
>> One possible spelling is:
>>
>> x = float(string) except ValueError else float('nan')
>
> I think 'as' would be better than 'else', since 'else' has a different
> meaning in try/except statements, e.g.:
>
>x = float(string) except ValueError, TypeError as float('nan')
>
> Of course, this is a different meaning of 'as', too, but it's not "as"
> contradictory, IMO... ;-)
(We're probably well into python-ideas territory at this point, but I'll keep
things where the thread started for now)
The basic idea appears sound to me as well. I suspect finding an acceptable
syntax is going to be the sticking point.
Breaking the problem down, we have three things we want to separate:
1. The expression that may raise the exception 2. The expression defining the
exceptions to be caught 3. The expression to be used if the exception actually
is caught
>From there it is possible to come up with all sorts of variants.
Option 1:
Change the relative order of the clauses by putting the exception definition
last:
x = float(string) except float('nan') if ValueError
op(float(string) except float('nan') if ValueError)
I actually like this one (that's why I listed it first). It gets the clauses
out of order relative to the statement, but the meaning still seems pretty
obvious to me.
Option 2:
Follow the lamba model and allow a colon inside this form of expression:
x = float(string) except ValueError: float('nan')
op(float(string) except ValueError: float('nan'))
This has the virtue of closely matching the statement syntax, but embedding
colons inside expressions is somewhat ugly. Yes, lambda already does it, but
lambda can hardly be put forward as a paragon of beauty.
Option 3a/3b:
Raymond's except-else suggestion:
x = float(string) except ValueError else float('nan')
op(float(string) except ValueError else float('nan'))
This has the problem of inverting the sense of the else clause relative to the
statement form (where the else clause is executed only if no exception occurs)
A couple of extra keywords would get the sense correct again, but I'm not sure
the parser could cope with it and it is rather verbose (I much prefer option 1
to this idea):
x = float(string) if not except ValueError else float('nan')
op(float(string) if not except ValueError else float('nan'))
Option 4:
PJE's except-as suggestion:
x = float(string) except ValueError as float('nan')
op(float(string) except ValueError as float('nan'))
Given that we now use "except ValueError as ex" in exception statements, the
above strikes me a really confusing idea.
Cheers,
Nick.
--
Nick Coghlan | ncogh...@gmail.com | Brisbane, Australia
---
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/dinov%40microsoft.com
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] (try-except) conditional expression similar to (if-else) conditional (PEP 308)
MRAB wrote:
> Dino Viehland wrote:
> > On option 1 is this legal then?
> >
> > x = float(string) except float('nan') if some_check() else float('inf') if
> ValueError
> >
> Well, is this is legal?
>
> try:
> x = float(string)
> except some_check():
> x = float('nan')
> except ValueError:
> x = float('inf')
>
I was thinking this was would be equal to:
x = float(string) except (float('nan') if some_check() else float('inf')) if
ValueError
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] operator precedence of __eq__, __ne__, etc, if both object have implementations
Willem wrote: > > It's easy for other implementations to get this wrong. :) For IronPython we wrote a set of tests which go through and define the various operator methods in all sorts of combinations on both new-style and old-style classes as well as subclasses of those classes and then do the comparisons w/ logging. We've got this for both comparisons and the normal binary operators. We've also done it in other areas such as throwing exceptions and other complicated areas. But having the the coercion rules documented still helped a lot (although my recollection is they were only about 95% right). ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] unsubscriptable vs object does not support indexing
Is there a reason or a rule by which CPython reports different error message for different failures to subscript? For example: >>> set()[2] Traceback (most recent call last): File "", line 1, in TypeError: 'set' object does not support indexing >>> class c(object): pass ... >>> c()[2] Traceback (most recent call last): File "", line 1, in TypeError: 'c' object does not support indexing But compare this to: >>> [].append[42] Traceback (most recent call last): File "", line 1, in TypeError: 'builtin_function_or_method' object is unsubscriptable >>> (lambda: 42)[42] Traceback (most recent call last): File "", line 1, in TypeError: 'function' object is unsubscriptable >>> property()[42] Traceback (most recent call last): File "", line 1, in TypeError: 'property' object is unsubscriptable IronPython had a bug report that we were getting this wrong for set objects and that "does not support indexing" was also a clearer error message. I'm wondering if there's some reason why the different error messages occur which I'm missing. Otherwise I could switch to using the more clear message or start marking types which should report the unsubscriptable error. Does anyone have any insights into this? ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] operator precedence of __eq__, __ne__, etc, if both object have implementations
Frank wrote: > > For IronPython we wrote a set of tests which go through and define > > the various operator methods in all sorts of combinations on both > > new-style and old-style classes as well as subclasses of those > classes > > and then do the comparisons w/ logging. > We've talked about this before, but now I'm reminded again -- do you > happen to have a pointer to these tests just in case someone might > want to grab them :) ? > We include them in the source distribution which you can download the 2.6RC1 release which just came out today here: http://ironpython.codeplex.com/Release/ProjectReleases.aspx?ReleaseId=30315#DownloadId=84669 All of the side-by-side tests are in: IronPython-2.6\Src\Tests\compat Or you can view the source via the web at: http://ironpython.codeplex.com/SourceControl/ListDownloadableCommits.aspx (this page also has instructions for Subversion and other source control clients) And the latest version there is in: IronPython_Main\Src\Tests\compat Hopefully the infrastructure will just work on Jython because it also runs on CPython but there may be some Windows specific code in there (e.g. import nt, a general problem w/ our tests from the days before we always ran tests w/ the CPython std lib present). If you run into any problems let me know and I'd be happy to fix it to make them more compatible. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] unsubscriptable vs object does not support indexing
R. David Murray wrote: > Looking at the source, these messages are generated from abstract.c, > and the difference appears to be whether or not the tp_as_sequence slot > is filled in or not. If it is, but there is no sq_item method, then > PySequence_GetItem gives the "does not support indexing" message. > If it isn't filled in, you get the 'not subscriptable"(*) message from > PyObject_GetItem. > > The logic appears to be, roughly, if an object does not have mapping > methods, or has them but does not have mp_subscript, and does have > sequence methods but does not have sq_item, then you get a 'does not > support indexing'. Otherwise you get 'not subscriptable'. > > The question is, is this a useful distinction, or is it an > implementation artifact? First on the useful distinction: for better or worse users do seem to have an affinity towards CPython's error messages and especially so when they're better than ours :). But I certainly hope no one is taking a dependency on them by parsing them or anything like that such that it would actually break someone's program. So basically I regard all error messages as implementation artifacts but I have no problem matching them if it'll make someone happier and it's not too crazy to do it. That being said there does seem to be a useful distinction here. It sounds like the idea here is to report a better error message when something is more collection like than a full blown sequence - in particular I'm thinking of .NET's ICollection and Java's Collection interfaces that are very sequence like but don't support getting elements by index. The only oddity in the error message to me is that user defined objects are effectively always treated as sequences. That's completely an implementation artifact but I'm going to copy it too because it's easy, may prevent another bug report, and checking for all the various sequence methods seems like unnecessary overkill. Thanks for the information! ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] operator precedence of __eq__, __ne__, etc, if both object have implementations
Skip wrote: > Dino> For IronPython we wrote a set of tests which go through and define > Dino> the various operator methods in all sorts of combinations on both > Dino> new-style and old-style classes as well as subclasses of those > Dino> classes and then do the comparisons w/ logging. > > It would be very nice if these complex corner cases had a set of test > cases which could be run by all implementations (CPython, Jython, > IronPython, PyPy, etc). I don't know. Maybe the CPython test suite serves > that purpose, but it seems like it would be helpful if this sort of > "validation suite" was maintained as a separate project all implementations > could use and contribute to. We are going to start contributing tests back real soon now. I'm not sure that these are the best tests to contribute as they require a version of Python to compare against rather than being nice and stand alone. But I'm sure we have other tests which cover this as well just not as exhaustively. We could also possibly check in the baseline file and then CPython could compare it's self to previous versions but it'd probably be a pretty big file - so it probably shouldn't be included in the standard install in the tests directory. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] operator precedence of __eq__, __ne__, etc, if both object have implementations
Mark wrote: > On Wed, Sep 23, 2009 at 4:54 PM, Dino Viehland wrote: > > We are going to start contributing tests back real soon now. I'm not sure > > that these are the best tests to contribute as they require a version of > > Python to compare against rather than being nice and stand alone. But I'm > > sure we have other tests which cover this as well just not as exhaustively. > > We could also possibly check in the baseline file and then CPython could > > compare it's self to previous versions but it'd probably be a pretty > > big file - so it probably shouldn't be included in the standard install > > in the tests directory. > > How big is big? For comparison, CPython's Lib/test/decimaltestdata > directory alone is already over 4Mb, so maybe size isn't an issue? Running all of the side by side tests except our exceptions test it's about 73Mb. It might be highly compressible though. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] unsubscriptable vs object does not support indexing
Brett wrote: > Let's ignore history, which I bet is the reason for the distinction, > and just look at the error messages; does the distinction make sense > to a newbie? I would say no and that the "does not support indexing" > error message is more useful. For expert programmers they could figure > out the problem with either error message. The only help is if you are > trying to debug a type, but I am willing to bet most of us didn't know > the distinction at the C level until David looked it up. > > So I am +1 on unified the message and +1 on using the "does not > support indexing" one. I'd be +1 on the unified message as well - but it seems what that message should be may be contentious (and quite a bike shed discussion at that). The bug David linked to (http://bugs.python.org/issue5760) has a preference for subscript because that's what's used elsewhere in the language. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP about sys.implementation and implementation specific user site directory
Christian wrote: > sys.implementation is a PyStructSequence that contains various > information about the current implementation. Some fields are required > to be present on every implementation. Implementations may choose to > add > additional fields as they see fit. Some fields like compiler are useful > for introspection and are already part of sys.version. I like to > include > them for the sake of completeness. Given that this is all about establishing some common ground between implementations I would propose not using a CPython specific term here such as PyStructSequence :) The Python docs seem to use structseq for sys.float_info. > compiler (required): > verbose name of the compiler, for example "GCC 4.3.3", "Java > HotSpot(TM) Client VM (Apple Inc.)", "Mono JIT compiler version 2.0.1" What's the value of this attribute? The main reason I ask is there's no way that I know of to determine the JIT being used in .NET. We could of course fill in some dummy value here for IronPython but I'm also not sure why anyone would use this. Otherwise it looks good to me. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP about sys.implementation and implementation specific user site directory
Christian wrote: > > Martin has already answered both points to my satisfaction. Do you > agree with him? Sure, source level makes more sense - so we'd have "csc" or "gmcs" if compiled with Mono (assuming there's some way to do that...). ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 370 and IronPython
Michael wrote: > The IronPython team currently have legal issues distributing modified > versions of the standard library (Dino can correct me if I'm wrong > here). It's actually not due to legal issues although we'd have to check w/ the lawyers if we wanted to do it. It's mainly that we don't want to ship a modified version of the standard library because it seems like it should be standard. It's also a little more convenient for us to not have to maintain a separate copy (although we do already do that for the tests). ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 3003 - Python Language Moratorium
Stefan wrote: > I assume that this is artificially exaggerated to make a point, as this > behaviour is obviously not a technical requirement but an optimisation, > which could potentially be disabled. > If there's a way to disable this then that's fine and IMO when it was disabled you'd still be Python. But changing behavior in the name of optimization is no longer just an optimization. For what it's worth in IronPython we're going to look at making this faster in the future as well. We already cache built-in values locally in a module and invalidate the cache when the builtins changes. In the future I'd like to even combine loading a global and invoking it into a single call site for even better optimizations. And you could even imagine an extreme scenario where when built-ins changed all the code in the system that depended upon them is re-compiled punishing users who actually do this but I doubt we'll go that far. But the important thing is that the optimizations don't change the language behavior otherwise I don't think you're still Python. But of course that's what makes it so challenging and fun to try and optimize Python :) ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] First draft of "sysconfig"
Tarek wrote:
> == Installation schemes ==
>
> First, the module contains the installation schemes for each platform
> CPython uses.
> An install scheme is a mapping where the key is the "code" name for a
> directory, and
> the value the path of that directory, with some $variable that can be
> expanded.
>
> Install schemes are stored in a private mapping, where the keys are
> usually the value of os.name,
> and the value, the mapping I've mentionned earlier.
>
> So, for example, the paths for win32 are:
>
> _INSTALL_SCHEMES = {
> ...
> 'nt': {
> 'stdlib': '$base/Lib',
> 'platstdlib': '$base/Lib',
> 'purelib': '$base/Lib/site-packages',
> 'platlib': '$base/Lib/site-packages',
> 'include': '$base/include',
> 'platinclude': '$base/include',
> 'scripts': '$base/Scripts',
> 'data' : '$base',
> },
> ...
> }
Not mentioned here are the user schemes. Looking at the code it seems that
_getuserbase is adding "Python" to the path on nt. Isn't that redundant?
The paths in _INSTALL_SCHEMES already include "Python".
Also w/ user dirs in general - I think there should be some implementation
specific portion of the path as well. Right now I think IronPython and
CPython will end up with the same user paths for the same versions which
could cause problems if someone releases separate modules for IronPython
and CPython for some reason (or if users just want different sets of
things installed for different interpreters).
> * get_platform(): Return a string that identifies the current
> platform. (this one is used by site.py for example)
I wonder if this would make more sense a built-in. Ultimately it seems
like the interpreter implementation knows best about what aspects of
it's underlying platform would require different libraries.
With the existing code I think IronPython would return "cli" everywhere
(because os.name == 'nt' on Windows but posix on Linux & Mac OS/X but
we still don't have uname). I think Jython will return something like
java1.6.0_17 because it's os.name is java - which might be more specific
than is necessary.
Also if the purpose of this is for platform specific build directories
it might be interesting to have multiple return values. For example in
IronPython the minimal thing we'd want I think is a "cli" directory for
.NET assemblies. But there could also be native extensions which are
platform specific so we'd want "cli-x86" and "cli-x64" depending upon
the platform. Jython might want the same thing as they add ctypes
support.
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Microsoft contributor agreement received?
I'm not sure the best place to verify this so I'm starting here. I'm told we finally faxed in our contributor agreement (to the number listed at http://www.python.org/psf/contrib/) about a week and a half ago. I'd just like to make sure that someone has received it. Is anyone here able to confirm that it went through fine? ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] First draft of "sysconfig"
Tarek wrote: > > (I didn't digg on how Jython organizes things yet, any hint would be > appreciated) The installation directory looks like it's organized just like CPython but I have no clue how user directories would/should be arranged. > > > > > Also if the purpose of this is for platform specific build directories > > it might be interesting to have multiple return values. For example in > > IronPython the minimal thing we'd want I think is a "cli" directory for > > .NET assemblies. But there could also be native extensions which are > > platform specific so we'd want "cli-x86" and "cli-x64" depending upon > > the platform. Jython might want the same thing as they add ctypes > > support. > > So like, having an architecture marker, that defaults to the current ? > >get_platform(architecture=platform.architecture) > How would you know what other architectures would be valid to pass in here? Returning a list would let the implementation say that it knows a certain set of architectural binaries are valid. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] First draft of "sysconfig"
Antoine wrote: > Dino Viehland microsoft.com> writes: > > > > > * get_platform(): Return a string that identifies the current > > > platform. (this one is used by site.py for example) > > > > I wonder if this would make more sense a built-in. Ultimately it seems > > like the interpreter implementation knows best about what aspects of > > it's underlying platform would require different libraries. > > I don't think it makes sense to make a builtin of such a little used, > non-performance critical API. > If we want to factor out all implementation-specific things, we could add a > module dedicated to that (e.g. "_interpreter") and have other modules (os, > platform, sysconfig...) draw from that. Putting this in an _interpreter module is fine with me (or even putting it somewhere in sys works - e.g. there was a sys.implementation discussion not too long ago). The important thing is that the interpreter implementation really is the one that understands what binaries they would be compatible with it regardless of how often it gets used and how it performs. If it's not a built-in then I think we'd be in a world where either every implementation needs to patch this file in their distribution or contribute a change back to support their implementation and that just seems ugly. And it's already a very large function even w/o IronPython, Jython, PyPy and Unladen Swallow support. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] First draft of "sysconfig"
Tarek wrote: > How would you use it when a list is returned ? Can you provide a few > examples where > the code wants to know the default architecture for the current platform ? > etc. The consumer could enumerate over it and then do whatever they were doing w/ the platform multiple times. If an earlier one succeeds at what they're attempting to do then they're done. If there's no "success", lets say they're appending something to sys.path, then they'd do it for all elements in the sequence. It's not an absolute requirement or anything like that - it just jumped out at me because IronPython and Jython do have multiple platforms which could be relevant at one time. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Very Strange Argument Handling Behavior
Mark Dickinson wrote:
> Removing it certainly seems in keeping with the goal of making life
> easier for alternate implementations. (Out of curiosity, does anyone
> know what IronPython does here?)
>
> I've opened http://bugs.python.org/issue8419
It looks like IronPython reports a type error as well:
IronPython 2.6.1 DEBUG (2.6.10920.0) on .NET 2.0.50727.4927
Type "help", "copyright", "credits" or "license" for more information.
>>> def f(**kwargs):
... print(kwargs)
...
>>> kwargs = {1: 3}
>>>
>>> dict({}, **kwargs)
Traceback (most recent call last):
File "", line unknown, in
TypeError: expected string for dictionary argument got 1
>>> d = {1:2}
>>> d.update({3:4}, **{5:6})
Traceback (most recent call last):
File "", line unknown, in
TypeError: expected string for dictionary argument got 5
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Very Strange Argument Handling Behavior
Benjamin wrote: > 2010/4/17 Guido van Rossum : > > On Sat, Apr 17, 2010 at 9:22 AM, Nick Coghlan > wrote: > >> Guido van Rossum wrote: > >>> Because Python promises that the object the callee sees as 'kwargs' > is > >>> "just a dict". > >> > >> Huh, I thought kwargs was allowed to be implemented as a > >> string-keys-only dict (similar to class and module namespaces) while > >> still be a valid Python implementation. I guess I was wrong. > > > > Actually I don't know about that. Is there language anywhere in the > > language reference that says this? What do IronPython, Jython, PyPy > > actually do? > > Similar to CPython, PyPy has dict versions optimized for strings, > which fall back to the general version when given non-string keys. IronPython as well. The only place we use a string only dict is for new-style classes whose dict's are wrapped in a dictproxy. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Very Strange Argument Handling Behavior
Maciej wrote: > On Sat, Apr 17, 2010 at 11:38 AM, Dino Viehland > wrote: > > Benjamin wrote: > >> 2010/4/17 Guido van Rossum : > >> > On Sat, Apr 17, 2010 at 9:22 AM, Nick Coghlan > >> wrote: > >> >> Guido van Rossum wrote: > >> >>> Because Python promises that the object the callee sees as > 'kwargs' > >> is > >> >>> "just a dict". > >> >> > >> >> Huh, I thought kwargs was allowed to be implemented as a > >> >> string-keys-only dict (similar to class and module namespaces) > while > >> >> still be a valid Python implementation. I guess I was wrong. > >> > > >> > Actually I don't know about that. Is there language anywhere in > the > >> > language reference that says this? What do IronPython, Jython, > PyPy > >> > actually do? > >> > >> Similar to CPython, PyPy has dict versions optimized for strings, > >> which fall back to the general version when given non-string keys. > > > > IronPython as well. The only place we use a string only dict is for > > new-style classes whose dict's are wrapped in a dictproxy. > > And yet that breaks some code :-) Sure, if you do: class C(object): locals()[object()] = 42 dir(C) You lose. Once I'm aware of some piece of code in the wild doing this then I'll be happy to change IronPython to be more compatible. :) ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] A grammatical oddity: trailing commas in argument lists.
Terry wrote: > This violates the important principle that allowed def and call arg > sequences should match to the extent sensible and possible. In this > sense, the SyntaxError is a bug. So I would fix this now for 3.2 and > notify the other implementors. +1 on fixing it - trailing commas are awesome. I'm always annoyed in C# where I frequently can't use them. This seems like a bug fix level change that should be easy for the other implementations to fix. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] 2.4 & 2.5 beta 3 crash
We've been working on fixing some exception handling bugs in IronPython where we differ from CPython. Along the way we ran into this issue which causes CPython to crash when the code below is run. It crashes on both 2.4 and 2.5 beta 3. The code's technically illegal, but it probably shouldn't crash either :) def test(): for abc in range(10): try: pass finally: try: continue except: pass test() ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] 2.4 & 2.5 beta 3 crash
Yeah, continue inside the finally block is illegal. If you don't have the extra try/except nesting then continue is detected as a syntax error. -Original Message- From: Dennis Allison [mailto:[EMAIL PROTECTED] Sent: Wednesday, August 16, 2006 10:22 AM To: Josiah Carlson Cc: Dino Viehland; python-dev@python.org Subject: Re: [Python-Dev] 2.4 & 2.5 beta 3 crash def test(): for abc in range(10): try: pass finally: try: pass except: pass test() does not raise a segmentation fault. On Wed, 16 Aug 2006, Josiah Carlson wrote: > > Dino Viehland <[EMAIL PROTECTED]> wrote: > > > > We've been working on fixing some exception handling bugs in > > IronPython where we differ from CPython. Along the way we ran into > > this issue which causes CPython to crash when the code below is run. > > It crashes on both 2.4 and 2.5 beta 3. The code's technically > > illegal, but it probably shouldn't crash either :) > > > def test(): > > for abc in range(10): > > try: pass > > finally: > > try: > > continue > > except: > > pass > > > > > > test() > > It also reliably crashes 2.3, though I don't see anything in there as > being illegal, but maybe I don't understand the langauge as well as I > think I do. > > Note that there is another segfaulting bug in CPython with regards to > threads that was recently closed, but which is still a problem: > http://python.org/sf/780714 > > Does IronPython survive in that case? > > > - Josiah > > ___ > Python-Dev mailing list > Python-Dev@python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > http://mail.python.org/mailman/options/python-dev/allison%40shasta.sta > nford.edu > -- ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] 2.4 & 2.5 beta 3 crash
IronPython actually enables / disables stack overflow checking through either a command line option or by calling sys.setrecursionlimit. By default we have no recursion limit. Without those command line options we'll end up having the CLR throw an unrecoverable stack overflow exception (and the process will be terminated). It's a little bit better than a seg fault, but not much. With stack overflow checking enabled we'll actually catch the recursion for this and throw a RuntimeError stating maximum recursion depth reached. -Original Message- From: Josiah Carlson [mailto:[EMAIL PROTECTED] Sent: Wednesday, August 16, 2006 10:11 AM To: Dino Viehland; python-dev@python.org Subject: Re: [Python-Dev] 2.4 & 2.5 beta 3 crash Dino Viehland <[EMAIL PROTECTED]> wrote: > > We've been working on fixing some exception handling bugs in > IronPython where we differ from CPython. Along the way we ran into > this issue which causes CPython to crash when the code below is run. > It crashes on both 2.4 and 2.5 beta 3. The code's technically > illegal, but it probably shouldn't crash either :) > def test(): > for abc in range(10): > try: pass > finally: > try: > continue > except: > pass > > > test() It also reliably crashes 2.3, though I don't see anything in there as being illegal, but maybe I don't understand the langauge as well as I think I do. Note that there is another segfaulting bug in CPython with regards to threads that was recently closed, but which is still a problem: http://python.org/sf/780714 Does IronPython survive in that case? - Josiah ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] .pyc file has different result for value "1.79769313486232e+308" than .py file
We've noticed a strange occurance on Python 2.4.3 w/ the floating point value
1.79769313486232e+308 and how it interacts w/ a .pyc. Given x.py:
def foo():
print str(1.79769313486232e+308)
print str(1.79769313486232e+308) == "1.#INF"
The 1st time you run this you get the correct value, but if you reload the
module after a .pyc is created then you get different results (and the
generated byte code appears to have changed).
Python 2.4.3 (#69, Mar 29 2006, 17:35:34) [MSC v.1310 32 bit (Intel)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import x
>>> import dis
>>> dis.dis(x.foo)
2 0 LOAD_GLOBAL 0 (str)
3 LOAD_CONST 1 (1.#INF)
6 CALL_FUNCTION1
9 PRINT_ITEM
10 PRINT_NEWLINE
3 11 LOAD_GLOBAL 0 (str)
14 LOAD_CONST 1 (1.#INF)
17 CALL_FUNCTION1
20 LOAD_CONST 2 ('1.#INF')
23 COMPARE_OP 2 (==)
26 PRINT_ITEM
27 PRINT_NEWLINE
28 LOAD_CONST 0 (None)
31 RETURN_VALUE
>>> reload(x)
>>> dis.dis(x.foo)
2 0 LOAD_GLOBAL 0 (str)
3 LOAD_CONST 1 (1.0)
6 CALL_FUNCTION1
9 PRINT_ITEM
10 PRINT_NEWLINE
3 11 LOAD_GLOBAL 0 (str)
14 LOAD_CONST 1 (1.0)
17 CALL_FUNCTION1
20 LOAD_CONST 2 ('1.#INF')
23 COMPARE_OP 2 (==)
26 PRINT_ITEM
27 PRINT_NEWLINE
28 LOAD_CONST 0 (None)
31 RETURN_VALUE
>>> ^Z
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] .pyc file has different result for value "1.79769313486232e+308" than .py file
Thanks for the link - it's a good explanation. FYI I've opened a bug against the VC++ team to fix their round tripping on floating point values (doesn't sound like it'll make the next release, but hopefully it'll make it someday). -Original Message- From: Tim Peters [mailto:[EMAIL PROTECTED] Sent: Wednesday, September 13, 2006 11:39 AM To: Dino Viehland Cc: python-dev@python.org; Haibo Luo Subject: Re: [Python-Dev] .pyc file has different result for value "1.79769313486232e+308" than .py file [Dino Viehland] > We've noticed a strange occurance on Python 2.4.3 w/ the floating > point value 1.79769313486232e+308 and how it interacts w/ a .pyc. Given x.py: > > def foo(): > print str(1.79769313486232e+308) > print str(1.79769313486232e+308) == "1.#INF" > > > The 1st time you run this you get the correct value, but if you reload > the module after a .pyc is created then you get different results (and > the generated byte code appears to have changed). > ... Exhaustively explained in this recent thread: http://mail.python.org/pipermail/python-list/2006-August/355986.html ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] locals(), closures, and IronPython...
def a():
x = 4
y = 2
def b():
print y, locals()
print locals()
b()
a()
in CPython prints:
{'y': 2, 'x': 4, 'b': }
2 {'y': 2}
I'm wondering if it's intentional that these don't print dictionaries w/ the
same contents or if it's more an accident of the implementation. In other
words would it be reasonable for IronPython to promote all of the locals of a
into b's dictionary when both a and b call locals?
We currently match CPython's behavior here - at least in what we display
although possibly not in the lifetimes of objects. We're considering an
internal change which might alter the behavior here though and end up
displaying all the members.
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] locals(), closures, and IronPython...
Thanks Guido. It might take some time (and someone may very well beat me to it
if they care a lot) but I'll see if we can get the PEP started.
-Original Message-
From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Guido van Rossum
Sent: Monday, March 05, 2007 2:14 PM
To: Dino Viehland
Cc: python-dev@python.org
Subject: Re: [Python-Dev] locals(), closures, and IronPython...
Jeremy Hylton has been asking questions about this too at the sprint
after PyCon. I'm tempted to accept that the exact behavior of locals()
is implementation-defined (IOW undefined :-) as long as it includes
the locals defined in the current scope; whether it also includes free
variables could be debatable. I don't know too many good use cases for
locals() apart from "learning about the implementation" I think this
might be okay. Though a PEP might be in order to get agreement between
users, developers and other implementation efforts (e.g. PyPy,
Jython).
On 3/5/07, Dino Viehland <[EMAIL PROTECTED]> wrote:
>
>
>
>
> def a():
>
> x = 4
>
> y = 2
>
> def b():
>
> print y, locals()
>
> print locals()
>
> b()
>
>
>
> a()
>
>
>
> in CPython prints:
>
>
>
> {'y': 2, 'x': 4, 'b': }
>
> 2 {'y': 2}
>
>
>
> I'm wondering if it's intentional that these don't print dictionaries w/ the
> same contents or if it's more an accident of the implementation. In other
> words would it be reasonable for IronPython to promote all of the locals of
> a into b's dictionary when both a and b call locals?
>
>
>
> We currently match CPython's behavior here - at least in what we display
> although possibly not in the lifetimes of objects. We're considering an
> internal change which might alter the behavior here though and end up
> displaying all the members.
>
>
>
>
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> http://mail.python.org/mailman/options/python-dev/guido%40python.org
>
>
--
--Guido van Rossum (home page: http://www.python.org/~guido/)
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] locals(), closures, and IronPython...
The lifetime issue is bad - unfortunately we have the same issue in v1.x we
just don't show you the names/values. That alone should (and hopefully will)
drive us to clean this up but right now we'll only be worse in that we are
explicit about keeping the dictionaries alive.
Classes are interesting for us as well... Our model is that we chain all of our
dictionaries together including the classes in FunctionEnvironment's (just an
optimized dictionary, usually only containing the variables used in the
closure) and ultimately that chains out to the globals dictionary. The change
that exposes all of the locals is to drop the explicit list of keys in the
outer scopes. The classes dictionaries get mixed in here as well and so now
when doing a fully dynamic name-lookup we need to know to skip the classes
which is strange.
How would "nonlocals" be used in the method? Wouldn't you need a new opcode
which would resolve to a non-local variable or add it explicitly into the LEGB
search?
-Original Message-
From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Jeremy Hylton
Sent: Tuesday, March 06, 2007 4:54 AM
To: Dino Viehland
Cc: Guido van Rossum; python-dev@python.org
Subject: Re: [Python-Dev] locals(), closures, and IronPython...
On 3/5/07, Dino Viehland <[EMAIL PROTECTED]> wrote:
> Thanks Guido. It might take some time (and someone may very well beat me to
> it if they care a lot) but I'll see if we can get the PEP started.
Dino,
One of the questions I was puzzling over was what locals() should
return in a class scope. It's confusing in CPython, because the same
dictionary is used to stored names that are destined to become class
attributes and names that are used to created closures for methods
with free variables. The implementation was causing the latter names
to show up as class attributes when you called locals().
Terry is right that lifetime of variables is a key issue. It would
probably be bad if all the local variables of a function were kept
alive because a nested function used locals().
One idea was discussed at PyCon was having a different representation
for an environment so that it would be easier to introspect on it but
still possible to pass the environment to exec. One possibility is
just a third dictionary--globals, locals, nonlocals. Another
possibility is an object where you can ask about the scope of each
variable but also extract a dictionary to pass to locals.
Jeremy
>
> -Original Message-
> From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Guido van
> Rossum
> Sent: Monday, March 05, 2007 2:14 PM
> To: Dino Viehland
> Cc: python-dev@python.org
> Subject: Re: [Python-Dev] locals(), closures, and IronPython...
>
> Jeremy Hylton has been asking questions about this too at the sprint
> after PyCon. I'm tempted to accept that the exact behavior of locals()
> is implementation-defined (IOW undefined :-) as long as it includes
> the locals defined in the current scope; whether it also includes free
> variables could be debatable. I don't know too many good use cases for
> locals() apart from "learning about the implementation" I think this
> might be okay. Though a PEP might be in order to get agreement between
> users, developers and other implementation efforts (e.g. PyPy,
> Jython).
>
> On 3/5/07, Dino Viehland <[EMAIL PROTECTED]> wrote:
> >
> >
> >
> >
> > def a():
> >
> > x = 4
> >
> > y = 2
> >
> > def b():
> >
> > print y, locals()
> >
> > print locals()
> >
> > b()
> >
> >
> >
> > a()
> >
> >
> >
> > in CPython prints:
> >
> >
> >
> > {'y': 2, 'x': 4, 'b': }
> >
> > 2 {'y': 2}
> >
> >
> >
> > I'm wondering if it's intentional that these don't print dictionaries w/ the
> > same contents or if it's more an accident of the implementation. In other
> > words would it be reasonable for IronPython to promote all of the locals of
> > a into b's dictionary when both a and b call locals?
> >
> >
> >
> > We currently match CPython's behavior here - at least in what we display
> > although possibly not in the lifetimes of objects. We're considering an
> > internal change which might alter the behavior here though and end up
> > displaying all the members.
> >
> >
> >
> >
> > ___
> > Python-Dev mailing list
> > Python-Dev@python.org
> > http://mail.py
Re: [Python-Dev] Policy for making changes to the AST
Guido wrote: > On Mon, Apr 4, 2011 at 10:05 AM, fwierzbi...@gmail.com > wrote: > > As a re-implementor of ast.py that tries to be node for node > > compatible, I'm fine with #1 but would really like to have tests that > > will fail in test_ast.py to alert me! > > [and] > > On Mon, Apr 4, 2011 at 10:38 AM, Michael Foord > wrote: > > A lot of tools that work with Python source code use ast - so even > > though other implementations may not use the same ast "under the hood" > > they will probably at least *want* to provide a compatible > > implementation. IronPython is in that boat too (although I don't know > > if we *have* a compatible implementation yet - we certainly feel like we > *should* have one). > > Ok, so it sounds like ast is *not* limited to CPython? That makes it harder to > justify changing it just so as to ease the compilation process in CPython (as > opposed to add new language features). Even so I think adding new features does allow new changes to the AST. We'll need to do the work to add support for the new features anyway so updating the AST accordingly won't be much more work. I agree with Frank that as long as there are tests for the new features it's fine. I think it'll also be better for consumers who would probably prefer to see a YieldFrom node rather than its expansion (and not all new language features will necessarily have a reasonable expansion - consider if goto ever happened ;) ). Also, IronPython doesn't have ast yet but I think it has been requested that we implement it - we just haven't gotten around to it yet. So I'm +1 on allowing changes to it. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Policy for making changes to the AST
Terry wrote: > Are at least some of the implementation methods similar enough that they > could use the same AST? It is, after all, a *semantic* translation into > another > language, and that need not depend on subsequent transforation and > compilation to the ultimate target. A multiple-implementation AST could still > be x.y dependent. For IronPython we have our own AST which is closely tied to the DLR ASTs (our AST nodes are actually subclasses of the core DLR Expression node which then "reduce" to the core DLR nodes on-demand). We already do a huge amount of manipulation of those ASTs from optimizations (constant folding being the primary one) to re-writing them completely for things like generators or sys.settrace support and other optimizations like runtime optimized fast exception support. But our ASTs are probably sufficiently different and sufficiently tied to the DLR that we couldn't share the exact same optimizations on the ASTs but it would probably make it easier to steal ideas from CPython if you did them at the AST level as well. They also have other differences such as the fact that they're effectively immutable. Likely when we implement the _ast module it'll just transform our ASTs into the shared ASTs via some additional attributes we attach to our ASTs rather than making them the core AST implementation. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] New Super PEP
Just to chime in from the IronPython side (better late than never I suppose): If we need to get access to the frame which is calling super then we can make that happen in IronPython. It just means that super gets treated like we treat eval today and won't work if it's been aliased. -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Calvin Spealman Sent: Sunday, April 29, 2007 12:31 PM To: Collin Winter Cc: Python Mailing List Subject: Re: [Python-Dev] New Super PEP On 4/29/07, Collin Winter <[EMAIL PROTECTED]> wrote: > On 4/29/07, Calvin Spealman <[EMAIL PROTECTED]> wrote: > > On 4/29/07, Collin Winter <[EMAIL PROTECTED]> wrote: > > > What if the instance isn't called "self"? PEP 3099 states that "self > > > will not become implicit"; it's talking about method signatures, but I > > > think that dictum applies equally well in this case. > > > > I don't use the name self. I use whatever the first argument name is, > > found by this line of python code: > > > > instance_name = calling_frame.f_code.co_varnames[0] > > So I can't use super with anything but the method's invocant? That > seems arbitrary. This will be added to the open issues, but it comes back to the problem with allow the exact same super implementation both operate in the super(Class, Object).foo() form and also the super.__call__() form in the new version. Any suggestions are welcome for how to solve this. > > > Also, it's my understanding that not all Python implementations have > > > an easy analogue to CPython's frames; have you given any thought to > > > whether and how PyPy, IronPython, Jython, etc, will implement this? > > > > I'll bring this up for input from PyPy and IronPython people, but I > > don't know any Jython people. Are we yet letting the alternative > > implementations influence so strongly what we do in CPython? I'm not > > saying "screw them", just pointing out that there is always a way to > > implement anything, and if its some trouble for them, well, 2.6 or 3.0 > > targetting is far down the road for any of them yet. > > It's a smell test: if a given proposal is unduly difficult for > anything but CPython to implement, it's probably a bad idea. The > language shouldn't go down the Perl 5 road, where python (the C > interpreter) becomes the only thing that can implement Python (the > language). Understandable. I still haven't contacted anyone about it on in the PyPy or IronPython worlds, and anyone familiar with Jython who can comment would be appreciated. Ditto for PyPy and IronPython, even though I should be able to find some information there myself. -- Read my blog! I depend on your acceptance of my opinion! I am interesting! http://ironfroggy-code.blogspot.com/ ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/dinov%40microsoft.com ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] New Super PEP
>>> Being able to access the calling frame from IronPython would be really
>>> useful...
We do have a -X:Frames option but it's going to hurt your performance, but for
example:
IronPython 1.0.60816 on .NET 2.0.50727.312
Copyright (c) Microsoft Corporation. All rights reserved.
>>> def f():
... x = locals
... print x()
...
>>> f()
{'__name__': '__main__', '__builtins__': , '__doc__': None,
'site': , '
f': }
>>> ^Z
C:\Product\Released\IronPython-1.0>.\ipy.exe -X:Frames
IronPython 1.0.60816 on .NET 2.0.50727.312
Copyright (c) Microsoft Corporation. All rights reserved.
>>> def f():
... x = locals
... print x()
...
>>> f()
{'x': }
>>> ^Z
But then we'll NEVER use the CLR stack for storing locals, but we can also
always get the calling frames.
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] New lines, carriage returns, and Windows
We ran into an interesting user-reported issue w/ IronPython and the way Python writes to files and I thought I'd get python-dev's opinion. When writing a string in text mode that contains \r\n we both write \r\r\n because the default write mode is to replace \n with \r\n. This works great as long as you stay within an entirely Python world. Because Python uses \n for everything internally you'll never end up writing out a \r\n that gets transformed into a \r\r\n. But when interoperating with other native code (or .NET code in our case) it's fairly easy to be exposed to a string which contains \r\n. Ultimately we see odd behavior when round tripping the contents of a multi-line text box through a file. So today users have to be aware of the fact that Python internally always uses \n. They also need to be aware of any APIs that they call that might return a string with an embedded \r\n inside of them and transform the string back into the Python version. It could be argued that there's little value in doing the simple transformation from \r\n -> \r\r\n. Ultimately that creates a file that has line endings which aren't good on any platform. On the other hand it could also be argued that Python defines new-lines as \n and there should be no deviation from that. And doing so could be considered a slippery slope, first file deals with it, and next the standard libraries, etc... Finally this might break some apps and if we changed IronPython to behave differently we could introduce incompatibilities which we don't want. So I'm curious: Is there a reason this behavior is useful that I'm missing? Would there be a possibility (or objections to) making \r\n be untransformed in the Py3k timeframe? Or should we just tell our users to open files in binary mode? :) ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] New lines, carriage returns, and Windows
My understanding is that users can write code that uses only \n and Python will
write the end-of-line character(s) that are appropriate for the platform when
writing to a file. That's what I meant by uses \n for everything internally.
But if you write \r\n to a file Python completely ignores the presence of the
\r and transforms the \n into a \r\n anyway, hence the \r\r in the resulting
stream. My last question is simply does anyone find writing \r\r\n when the
original string contained \r\n a useful behavior - personally I don't see how
it is.
But Guido's response makes this sound like it's a problem w/ VC++ stdio
implementation and not something that Python is explicitly doing. Anyway, it'd
might be useful to have a text-mode file that you can write \r\n to and only
get \r\n in the resulting file. But if the general sentiment is
s.replace('\r', '') is the way to go we can advice our users of the behavior
when interoperating w/ APIs that return \r\n in strings.
-Original Message-
From: "Martin v. Löwis" [mailto:[EMAIL PROTECTED]
Sent: Wednesday, September 26, 2007 3:01 PM
To: Dino Viehland
Cc: python-dev@python.org
Subject: Re: [Python-Dev] New lines, carriage returns, and Windows
> This works great as long as you stay within an entirely Python world.
> Because Python uses \n for everything internally
I think you misunderstand fairly significantly how this all works
together. Python does not use \n "for everything internally". Python
is well capable of representing \r separately, and does so if you
ask it to.
> So I'm curious: Is there a reason this behavior is useful that I'm
> missing?
I think you are missing how it works in the first place (or else
you failed to communicate to me what precise behavior you find
puzzling).
Regards,
Martin
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] [python] Re: New lines, carriage returns, and Windows
And if this is fine for you, given that you may have the largest WinForms /
IronPython code base, I tend to think the replace may be reasonable. But we
have had someone get surprised by this behavior.
-Original Message-
From: Michael Foord [mailto:[EMAIL PROTECTED]
Sent: Wednesday, September 26, 2007 3:15 PM
To: Dino Viehland
Cc: python-dev@python.org
Subject: Re: [python] Re: [Python-Dev] New lines, carriage returns, and Windows
Dino Viehland wrote:
> My understanding is that users can write code that uses only \n and Python
> will write the end-of-line character(s) that are appropriate for the platform
> when writing to a file. That's what I meant by uses \n for everything
> internally.
>
> But if you write \r\n to a file Python completely ignores the presence of the
> \r and transforms the \n into a \r\n anyway, hence the \r\r in the resulting
> stream. My last question is simply does anyone find writing \r\r\n when the
> original string contained \r\n a useful behavior - personally I don't see how
> it is.
>
> But Guido's response makes this sound like it's a problem w/ VC++ stdio
> implementation and not something that Python is explicitly doing. Anyway,
> it'd might be useful to have a text-mode file that you can write \r\n to and
> only get \r\n in the resulting file. But if the general sentiment is
> s.replace('\r', '') is the way to go we can advice our users of the behavior
> when interoperating w/ APIs that return \r\n in strings.
>
We always do replace('\r\n','\n') but same difference...
Michael
>
> -Original Message-
> From: "Martin v. Löwis" [mailto:[EMAIL PROTECTED]
> Sent: Wednesday, September 26, 2007 3:01 PM
> To: Dino Viehland
> Cc: python-dev@python.org
> Subject: Re: [Python-Dev] New lines, carriage returns, and Windows
>
>
>> This works great as long as you stay within an entirely Python world.
>> Because Python uses \n for everything internally
>>
>
> I think you misunderstand fairly significantly how this all works
> together. Python does not use \n "for everything internally". Python
> is well capable of representing \r separately, and does so if you
> ask it to.
>
>
>> So I'm curious: Is there a reason this behavior is useful that I'm
>> missing?
>>
>
> I think you are missing how it works in the first place (or else
> you failed to communicate to me what precise behavior you find
> puzzling).
>
> Regards,
> Martin
>
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk
>
>
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] New lines, carriage returns, and Windows
> This I don't understand. Why don't you just use binary mode then? > At least for Python 2.x, the *only* difference between text and > binary mode is the treatment of line endings. That just flips the problem to the other side. Now if I have a Python library that I'm mixing w/ .NET code I need to be sure to transform the line endings, but now from \n -> \r\n (and hopefully they'd detect the new-line style they should use so it works correctly on Mono on *nix or Silverlight on OS X as well). ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Reserving an arg space for Jython
IronPython already uses -X:OptionName for special IronPython only options so +1 for -X. -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Frank Wierzbicki Sent: Friday, April 11, 2008 12:14 PM To: Brett Cannon Cc: Guido van Rossum; python-dev@python.org Subject: Re: [Python-Dev] Reserving an arg space for Jython On Fri, Apr 11, 2008 at 2:51 PM, Brett Cannon <[EMAIL PROTECTED]> wrote: > On Fri, Apr 11, 2008 at 7:40 PM, Guido van Rossum <[EMAIL PROTECTED]> wrote: > > > Works for me. We should have a patch to CPython that looks for -J and > > rejects it with "-J is reserved for Jython". > > > > Do we want it to be Jython-specific, or should it be available to any > alternative VM? I don't know if the IronPython folks need anything for > .NET, but maybe they would like one? -X was suggested on Jython's irc. I kind of like -J, but -X would work for us too. -Frank ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/dinov%40microsoft.com ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] bytearray methods returning self...
I've been implementing bytearray for IronPython and I noticed a couple of spots where the non-mutating methods of bytearray return self. In 2.6 but not in 3.0 RC2: x = bytearray(b'abc') y = x.replace(b'abc', b'bar', 0) id(x) == id(y) In 2.6 and in 3.0 RC2: t = bytearray() for i in range(256): t.append(i) x = bytearray(b'') y = x.translate(t) id(x) == id(y) Given that bytearray always returns a new instance on all of the other methods shouldn't it return a new instance here as well? I'm just thinking someone could be planning on mutating the result here and not expecting their original bytearray to be mutated. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] bytearray methods returning self...
Sure, it's now bug 4348 - http://bugs.python.org/issue4348 -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Guido van Rossum Sent: Tuesday, November 18, 2008 1:14 PM To: Dino Viehland Cc: python-dev@python.org dev Subject: Re: [Python-Dev] bytearray methods returning self... Good catch, this is a bug IMO and we should fix it in 2.6.1 and 3.0rc3. Mind filing a bug so we can keep track of it? --Guido On Tue, Nov 18, 2008 at 1:01 PM, Dino Viehland <[EMAIL PROTECTED]> wrote: > I've been implementing bytearray for IronPython and I noticed a couple of > spots where the non-mutating methods of bytearray return self. > > In 2.6 but not in 3.0 RC2: > > x = bytearray(b'abc') > y = x.replace(b'abc', b'bar', 0) > id(x) == id(y) > > In 2.6 and in 3.0 RC2: > > t = bytearray() > for i in range(256): t.append(i) > > x = bytearray(b'') > y = x.translate(t) > id(x) == id(y) > > Given that bytearray always returns a new instance on all of the other > methods shouldn't it return a new instance here as well? I'm just thinking > someone could be planning on mutating the result here and not expecting their > original bytearray to be mutated. -- --Guido van Rossum (home page: http://www.python.org/~guido/) ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] format specification mini-language docs...
Ok, now I'm implementing __format__ support for IronPython. The format spec
mini-language docs say that a presentation type of None is the same as 'g' for
floating point / decimal values. But these two formats seem to differ based
upon how they handle whole numbers:
>>> 2.0.__format__('')
'2.0'
>>> 2.0.__format__('g')
'2'
The docs also say that 'g' prints it as fixed point unless the number is too
large. But the fixed point format differs from what 'f' would print. I guess
it didn't say they'd both print it as fixed point w/ a precision of 6 or
anything but it seems a little unclear.
>>> 2.0.__format__('g')
'2'
>>> 2.0.__format__('f')
'2.00'
Finally providing any sign character seems to cause +1.0#INF and friends to be
returned instead of inf as is documented:
>>> 10e667.__format__('+')
'+1.0#INF'
>>> 10e667.__format__('')
'inf'
Are these just doc bugs? The inf issue is the only one that seems particularly
weird to me.
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] format specification mini-language docs...
Yep, after the thanksgiving delay I've opened bug #4482
(http://bugs.python.org/issue4482).
I either don't know how to or don't have the power to change who a bug is
assigned to so it appears to be currently unassigned.
-Original Message-
From: Eric Smith [mailto:[EMAIL PROTECTED]
Sent: Tuesday, November 25, 2008 4:38 AM
To: Dino Viehland
Cc: python-dev@python.org dev
Subject: Re: [Python-Dev] format specification mini-language docs...
Dino Viehland wrote:
> Finally providing any sign character seems to cause +1.0#INF and friends to
> be returned instead of inf as is documented:
>
>>>> 10e667.__format__('+')
> '+1.0#INF'
>>>> 10e667.__format__('')
> 'inf'
>
>
> Are these just doc bugs? The inf issue is the only one that seems
> particularly weird to me.
I think the inf one is a bug. Would you mind opening a bug and assigning
it to me? Thanks.
Eric.
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Python-3.0, unicode, and os.environ
Does anyone know what Mono does here? Presumably they have the exact same problem as all strings in .NET are Unicode, and filenames/env vars/etc... are always strings. Maybe if it's gotta be broken at least it can be broken in a manner that's consistent with others :) > -Original Message- > From: [EMAIL PROTECTED] [mailto:python- > [EMAIL PROTECTED] On Behalf Of Adam Olsen > Sent: Thursday, December 04, 2008 6:32 PM > To: James Y Knight > Cc: "Martin v. Löwis"; python-dev List > Subject: Re: [Python-Dev] Python-3.0, unicode, and os.environ > > On Thu, Dec 4, 2008 at 6:14 PM, James Y Knight <[EMAIL PROTECTED]> wrote: > > On Dec 4, 2008, at 6:39 PM, Martin v. Löwis wrote: > >> > >> I'm in favour of a different, fifth solution: > >> > >> 5) represent all environment variables in Unicode strings, > >> including the ones that currently fail to decode. > >> (then do the same to file names, then drop the byte-oriented > >> file operations again) > > > > Yay, maybe we can have this whole discussion all over again! > > > > Let's bring out all the same arguments, come to no conclusion, and > let it > > taper off unresolved, yet again! :) > > > > FWIW, I still agree with Martin that that's the most reasonable > solution. > > It died because nobody presented a viable solution, and I maintain no > solution is possible. All suggestions involve arbitrary > transformations that fail to round trip correctly at some point or > another. They're simply about shuffling the failure around to > somewhere the poster happens to like. > > Please, if you have a *new* idea that doesn't have a failure mode, by > all means post it. But don't resurrect a pointless bikeshed. > > > -- > Adam Olsen, aka Rhamphoryncus > ___ > Python-Dev mailing list > Python-Dev@python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python- > dev/dinov%40microsoft.com ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] should list's call to __len__ swallow SystemExit?
We had a bug reported that effectively boils down to we’re not swallowing
exceptions when list calls __len__
(http://www.codeplex.com/WorkItem/View.aspx?ProjectName=IronPython&WorkItemId=20598).
We can obviously make the change to catch exceptions here in IronPython even if
it seems like a bad idea to me ☺ But CPython seems to catch not only normal
exceptions, but also SystemExit. It seems like there’s been a move away from
this so I thought I’d mention it here. I tested it on 2.6.1 and 3.0.
import sys
class A(object):
def __iter__(self): return iter(range(10))
def __len__(self):
try:
print('exiting')
sys.exit(1)
except Exception as e:
print('can I catch it?', e)
list(A())
which prints:
exiting
[0, 1, 2, 3, 4, 5, 6, 7, 8, 9]
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 393 Summer of Code Project
Guido wrote: > Which reminds me. The PEP does not say what other Python > implementations besides CPython should do. presumably Jython and > IronPython will continue to use UTF-16, so presumably the language > reference will still have to document that strings contain code units (not > code > points) and the objections Tom Christiansen raised against this will remain > true for those versions of Python. (I don't know about PyPy, they can > presumably decide when they start their Py3k > port.) > > OTOH perhaps IronPython 3.3 and Jython 3.3 can use a similar approach and > we can lay the narrow build issues to rest? Can someone here speak for > them? The biggest difficulty for IronPython here would be dealing w/ .NET interop. We can certainly introduce either an IronPython specific string class which is similar to CPython's PyUnicodeObject or we could have multiple distinct .NET types (IronPython.Runtime.AsciiString, System.String, and IronPython.Runtime.Ucs4String) which all appear as the same type to Python. But when Python is calling a .NET API it's always going to return a System.String which is UTF-16. If we had to check and convert all of those strings when they cross into Python it would be very bad for performance. Presumably we could have a 4th type of "interop" string which lazily computes this but if we start wrapping .Net strings we could also get into object identity issues. We could stop using System.String in IronPython all together and say when working w/ .NET strings you get the .NET behavior and when working w/ Python strings you get the Python behavior. I'm not sure how weird and confusing that would be but conversion from an Ipy string to a .NET string could remain cheap if both were UTF-16, and conversions from .NET strings to Ipy strings would only happen if the user did so explicitly. But it's a huge change - it'll almost certainly touch every single source file in IronPython. I would think we'd get 3.2 done first and then think about what to do here. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Python as a Metro-style App
We spent some time investigating Python/Win8 projections but we don't really have anything else to say right now, but it is certainly possible. I haven't been following this thread so maybe this was already discussed, but on the whole "new OS target" thing - if people want to write immersive apps in Python then there will need to be a new build of Python. One thing that might make that easier is the fact that the C runtime is still available to metro apps, even if the C runtime calls a banned API. So to the extent that Python is just a C program the "port" should be pretty easy and mostly involve disabling functionality that isn't available at all to metro apps. I have packaged up Python 2.7 in an appx and run the application verifier on it (this was a while ago, so things may have changed between now and then), the attached banned.txt includes the list of APIs which Python is using that aren't allowed for the curious. Also, people who write apps will need to distribute Python w/ their app, there's currently no sharing between apps. -Original Message- From: Jeff Hardy [mailto:jdha...@gmail.com] Sent: Sunday, January 08, 2012 10:13 PM To: Antoine Pitrou Cc: python-dev@python.org; Dino Viehland Subject: Re: [Python-Dev] Python as a Metro-style App On Sat, Jan 7, 2012 at 2:57 PM, Antoine Pitrou wrote: > Depending on the extent of removed/disabled functionality, it might > not be very interesting to have a Metro port at all. Win 8 is practically a new OS target - the nt module may need to be replaced with a metro module to handle it well. Accessing the WinRT APIs directly from Python will also require a set of Python projections for the API, which should be straightforward to generate from the WinRT metadata files. I know Dino Viehland did some work on that; not sure if he can elaborate or not though. Otherwise, IronPython would be the only option for writing Metro apps in Python - not that I'd be *horribly* upset at that :). IronPython is slowly growing Metro support, and it seems like most things will work, but the .NET framework shields it from a lot of the WinRT guts. - Jeff API RegCloseKey in advapi32.dll is not supported for this application type. python27.dll calls this API. API RegConnectRegistryA in advapi32.dll is not supported for this application type. python27.dll calls this API. API RegCreateKeyA in advapi32.dll is not supported for this application type. python27.dll calls this API. API RegCreateKeyExA in advapi32.dll is not supported for this application type. python27.dll calls this API. API RegDeleteKeyA in advapi32.dll is not supported for this application type. python27.dll calls this API. API RegDeleteValueA in advapi32.dll is not supported for this application type. python27.dll calls this API. API RegEnumKeyExA in advapi32.dll is not supported for this application type. python27.dll calls this API. API RegEnumValueA in advapi32.dll is not supported for this application type. python27.dll calls this API. API RegFlushKey in advapi32.dll is not supported for this application type. python27.dll calls this API. API RegLoadKeyA in advapi32.dll is not supported for this application type. python27.dll calls this API. API RegOpenKeyExA in advapi32.dll is not supported for this application type. python27.dll calls this API. API RegQueryInfoKeyA in advapi32.dll is not supported for this application type. python27.dll calls this API. API RegQueryValueA in advapi32.dll is not supported for this application type. python27.dll calls this API. API RegQueryValueExA in advapi32.dll is not supported for this application type. python27.dll calls this API. API RegSaveKeyA in advapi32.dll is not supported for this application type. python27.dll calls this API. API RegSetValueA in advapi32.dll is not supported for this application type. python27.dll calls this API. API RegSetValueExA in advapi32.dll is not supported for this application type. python27.dll calls this API. API CreateDirectoryA in kernel32.dll is not supported for this application type. python27.dll calls this API. API CreateEventA in kernel32.dll is not supported for this application type. python27.dll calls this API. API CreateFileA in kernel32.dll is not supported for this application type. python27.dll calls this API. API CreateFileMappingA in kernel32.dll is not supported for this application type. python27.dll calls this API. API CreateFileW in kernel32.dll is not supported for this application type. python27.dll calls this API. API CreatePipe in kernel32.dll is not supported for this application type. python27.dll calls this API. API CreateProcessA in kernel32.dll is not supported for this application type. python27.dll calls this API. API DeleteFileA in kernel32.dll is not supported for this application type. python27.dll calls this API. API ExpandEnvironmentStringsW in kernel32.dll is not
Re: [Python-Dev] Python as a Metro-style App
Martin wrote: > Does that hold for all versions of the C runtime (i.e. is msvcr80.dll also > exempt from the ban, or just the version that comes with VS 11)? Just the VS 11 CRT is allowed. > > > So to the extent that Python is just a C program the "port" should be > > pretty easy and mostly involve disabling functionality that isn't > > available at all to metro apps. > > See the start of the thread: I tried to create a "WinRT Component DLL", and > that failed, as VS would refuse to compile any C file in such a project. Not > sure whether this is triggered by defining WINAPI_FAMILY=2, or any other > compiler setting. > > I'd really love to use WINAPI_FAMILY=2, as compiler errors are much easier > to fix than verifier errors. Let me see if I can try this. Hopefully I still have my VM w/ this all setup and I can see if I can get it building this way. I can always ping some people on the C++ team and ask them for help if I run into issues. I'll give it a shot tomorrow and get back to you. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Python as a Metro-style App
Martin wrote: > See the start of the thread: I tried to create a "WinRT Component DLL", and > that failed, as VS would refuse to compile any C file in such a project. Not > sure whether this is triggered by defining WINAPI_FAMILY=2, or any other > compiler setting. > > I'd really love to use WINAPI_FAMILY=2, as compiler errors are much easier > to fix than verifier errors. I got the same errors as you - it seems like they're related to enabling the Immersive bit for the compile of the DLL. I'm not certain if that's necessary, when I did the run before to see if Python would pass the app store validation it didn't care that we didn't have the App Container bit set on the DLL (it did want NXCOMPAT and dynamic base set though). I was also able to just define WINAPI_FAMILY=2 in the .vcxproj file and I got the various expected errors when accessing banned APIs (it actually seems like a bunch were missing vs. what the validator reported, but maybe that's just an issue w/ the developer preview). Once I fixed those errors up I was able to get a DLL that successfully compiled. I'm going to ping some people on the windows team and see if the app container bit is or will be necessary for DLLs. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Python as a Metro-style App
Dino wrote: > Martin wrote: > > See the start of the thread: I tried to create a "WinRT Component > > DLL", and that failed, as VS would refuse to compile any C file in > > such a project. Not sure whether this is triggered by defining > > WINAPI_FAMILY=2, or any other compiler setting. > > > > I'd really love to use WINAPI_FAMILY=2, as compiler errors are much > > easier to fix than verifier errors. > > ... > > I'm going to ping some people on the windows team and see if the app > container bit is or will be necessary for DLLs. > I heard back from the Windows team and they are going to require the app container bit to be set on all PE files (although they don't currently enforce it). I was able to compile a simple .c file and pass /link /appcontainer and that worked, so I'm going to try and figure out if there's some way to get the .vcxproj to build a working command line that includes that. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] yield from, user defined iterators, and StopIteration w/ a value...
I'm trying to create an object which works like a generator and delegates to a
generator for its implementation, but can also participate in yield from using
3.3 beta. I want my wrapper object to be able to cache some additional
information - such as whether or not the generator has completed - as well as
have it provide some additional methods for interacting with the state of the
generator. But currently this doesn't seem possible because raising
StopIteration from a user defined iterator has its value ignored as far as
yield from is concerned. Here's a simplified example of the problem:
class C:
def __iter__(self): return self
def __next__(self):
raise StopIteration(100)
def g():
if False:
yield 100
return 100
def f(val):
x = yield from val
print('x', x)
print(list(f(C(
print(list(f(g(
Which outputs:
x None
[]
x 100
[]
So you can see for the C case the value I raise from StopIteration is ignored,
but the value from the generator is propagated out. From my reading of PEP 380
the behavior here is incorrect for the user defined iterator case.
next(iter(C())) raises StopIteration exception with a value and that should be
the resulting value of the yield from expression according to the formal
semantics.
Ok, looking at the implementation this seems to be because PyIter_Next clears
the exception which prevents it from being seen in the yield from code path.
So should yield from just be doing "(*iter->ob_type->tp_iternext)(iter);"
directly and avoid the error checking code? Or am I wrong and this is the
intended behavior?
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] More compact dictionaries with faster iteration
PJ wrote: > Actually, IronPython may already have ordered dictionaries by default; see: > > http://mail.python.org/pipermail/ironpython-users/2006- > May/002319.html > > It's described as an implementation detail that may change, perhaps that > could be changed to being unchanging. ;-) > I think this has changed since 2006. IronPython was originally using the .NET dictionary class and just locking while using it, but it now has a custom dictionary which is thread safe for multiple readers and allows 1 writer. But it doesn't do anything to preserve order of insertions. OTOH changing certain dictionaries in IronPython (such as keyword args) to be ordered would certainly be possible. Personally I just wouldn't want to see it be the default as that seems like unnecessary overhead when the specialized class exists. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] [IronPython] base64 module
Well, CPython at least still enforces the padding, even if it's ignoring the
invalid characters.
Here's Seo's repro 'simplified' to go straight to binascii (just to get to the
root API):
>>> import binascii
>>> binascii.a2b_base64('%')
''
And then sending a valid character, invalid padding:
>>> binascii.a2b_base64('A')
Traceback (most recent call last):
File "", line 1, in ?
binascii.Error: Incorrect padding
and then throwing in random invalid characters, and CPython ignores the invalid
characters:
>>> binascii.a2b_base64('ABC=')
'\x00\x10'
>>> binascii.a2b_base64('%%ABC=')
'\x00\x10'
>>> binascii.a2b_base64('[EMAIL PROTECTED]')
'\x00\x10'
>>> binascii.a2b_base64('[EMAIL PROTECTED]()')
'\x00\x10'
The documentation for binascii.a2b_base64 doesn't specify if it throws for
anything either.
I would suspect that there's a reason why CPython is ignoring the invalid
characters here. If this is the expected behavior then I'm happy to make
IronPython match this. And at the very least we HAVE to fix the exception that
gets thrown - I'm with Seo that it should be a ValueError but line between
ValueError and TypeError is blurry at times anyway, and TypeError is what's
documented.
Do you want to help develop Dynamic languages on CLR?
(http://members.microsoft.com/careers/search/details.aspx?JobID=6D4754DE-11F0-45DF-8B78-DC1B43134038)
-Original Message-
From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Sanghyeon Seo
Sent: Wednesday, April 05, 2006 10:45 PM
To: python-dev@python.org; users@lists.ironpython.com
Subject: [IronPython] base64 module
Hello,
base64 module documentation for b64decode function says, "TypeError is
raised if s were incorrectly padded or if there are non-alphabet
characters present in the string." But this doesn't seem to be the
case. Testcase:
import base64
base64.b64decode('%')
Since % is a non-alphabet character, this should raise TypeError (btw,
shouldn't this be ValueError instead?), but Python 2.4.3 silently
ignores.
I found this while experimenting with IronPython. IronPython 1.0 Beta 5 gives:
Traceback (most recent call last):
File base64, line unknown, in b64decode
SystemError: cannot decode byte: '%'
It's not TypeError, but it doesn't silently ignore either.
Seo Sanghyeon
___
users mailing list
users@lists.ironpython.com
http://lists.ironpython.com/listinfo.cgi/users-ironpython.com
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] [IronPython] [Mono-dev] IronPython Performance
On the recursion limits: Until beta 6 IronPython didn't have proper support for limiting recursion depth. There was some minor support there, but it wasn't right. In beta 6 we have full support for limiting recursion depth, but by default we allow infinite recursion. If the user explicitly sets the recursion limit then we'll go ahead and enforce it. But all the reasons you outline below are great explanations of the differences. Do you want to help develop Dynamic languages on CLR? (http://members.microsoft.com/careers/search/details.aspx?JobID=6D4754DE-11F0-45DF-8B78-DC1B43134038) -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Paolo Molaro Sent: Monday, April 24, 2006 2:53 AM To: Brent Fulgham Cc: [EMAIL PROTECTED]; python-dev@python.org; users@lists.ironpython.com; mono-devel-list@lists.ximian.com Subject: Re: [IronPython] [Mono-dev] IronPython Performance [I continued your large cross-post, even if I'm not subscribed to python-dev: hopefully the moderators will approve the post if needed.] On 04/21/06 Brent Fulgham wrote: > A while ago (nearly two years) an interesting paper was published by Jim > Hugunin > (http://www.python.org/pycon/dc2004/papers/9/) crowing about the significant > speed advantage observed in this interpreter running on top of Microsoft's > .NET > VM stack. I remember being surprised by these results, as Python has always > seemed fairly fast for an interpreted language. > > I've been working on the Programming Language Shootout for a few years now, > and > after growing tired of the repeated requests to include scores for IronPython > I > finally added the implementation this week. > > Comparison of IronPython (1.0 Beta 5) to Python 2.4.3 [1] and IronPython (1.0 > Beta 1) > to Python 2.4.2 [2] do not match the results reported in the 2004 paper. In > fact, > IronPython is consistenly 3 to 4 times slower (in some cases worse than > that), and > seemed to suffer from recursion/stack limitations. You're comparing different benchmarks, so it's not a surprise that you get different numbers. I only tried two benchmarks from the paper, pystone and the globals one (attached) and the results are largely equivalent to the ones from the paper: time mono IronPython-1.0-Beta6/IronPythonConsole.exe -O python-globals-bench2.py real 0m0.861s user 0m0.815s sys 0m0.043s (Note the above includes the startup time that I guess was not considered in the paper and is 0.620 seconds running test(1): just increasing the number of iterations will make the difference converge at about 10x faster mono vs python2.4). time python2.4 -O python-globals-bench2.py real 0m2.239s user 0m2.202s sys 0m0.025s python2.4 -O /usr/lib/python2.4/test/pystone.py Pystone(1.1) time for 5 passes = 1.4 This machine benchmarks at 35714.3 pystones/second mono IronPython-1.0-Beta6/IronPythonConsole.exe -O /usr/lib/python2.4/test/pystone.py Pystone(1.1) time for 5 passes = 0.989288 This machine benchmarks at 50541.4 pystones/second So IronPython is 40+% faster than CPython, like in the paper. The mono version I used is roughtly 1.1.14. As for the recursion limitations: what are they? I took the recursive benchmark from the shootout page and both CPython and IronPython need the call to setrecursionlimit (actually cpython needed it earlier for me, anyway). This might be a difference between IronPython beta 5 and the beta6 I'm using, though. It is also interesting because on my system running the attached python-ack.py with 8 as argument, mono/IronPython is 10% faster than CPython, while the shootout page reports CPython 2.4 50% faster. Feel free to investigate if this is because of improvements in mono or IronPython:) > I am aware of the following issues that might affect the results: > 1) Mono vs. Microsoft's CLR. This can certainly be a factor: some things in the MS runtime run faster and some slower than in the Mono runtime, so it depends which features are exercised the most in a particular benchmark. It is very likely that in most benchmarks the MS runtime will score better. > 2) Python 2.1 vs. Python 2.4 The python folks will have more details, but python2.4 seems to be faster than 2.1 and 2.3 in several areas. > 3) Changes in IronPython over the last two years. For some time the IronPython folks have focused on features and correctness issues, so I guess they'll start caring about performance in the future. It is also to note that IronPython now uses generics which have yet to see optimizations work in the mono runtime and libraries (they are a 2.0 feature). > I thought these findings might be interesting to some of the Python, > IronPython, > and Mono developers. Let the flames begin! ;-) Why flames? Hard numbers are just that: useful information. It's just your conclusions that might be incorrect;-) I think the summary to take away is: *) Sometimes IronPython on mono is faster than CPython *) Often CPython is faster than IronPyt
Re: [Python-Dev] __getslice__ usage in sre_parse
I've also opened a bug for supporting __getslice__ in IronPython. Do you want to help develop Dynamic languages on CLR? (http://members.microsoft.com/careers/search/details.aspx?JobID=6D4754DE-11F0-45DF-8B78-DC1B43134038) -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Guido van Rossum Sent: Sunday, April 30, 2006 8:31 AM To: Sanghyeon Seo Cc: Discussion of IronPython; python-dev@python.org Subject: Re: [Python-Dev] __getslice__ usage in sre_parse On 4/28/06, Sanghyeon Seo <[EMAIL PROTECTED]> wrote: > Hello, > > Python language reference 3.3.6 deprecates __getslice__. I think it's > okay that UserList.py has it, but sre_parse shouldn't use it, no? Well, as long as the deprecated code isn't removed, there's no reason why other library code shouldn't use it. So I disagree that technically there's a reason why sre_parse shouldn't use it. > __getslice__ is not implemented in IronPython and this breaks usage of > _sre.py, a pure-Python implementation of _sre, on IronPython: > http://ubique.ch/code/_sre/ > > _sre.py is needed for me because IronPython's own regex implementation > using underlying .NET implementation is not compatible enough for my > applications. I will write a separate bug report for this. > > It should be a matter of removing __getslice__ and adding > isinstance(index, slice) check in __getitem__. I would very much > appreciate it if this is fixed before Python 2.5. You can influence the fix yourself -- please write a patch (relative to Python 2.5a2 that was just released), submit it to Python's patch tracker on SourceForge (read python.org/dev first), and then sending an email here to alert the developers. This ought to be done well before the planned 2.5b1 release (see PEP 256 for the 2.5 release timeline). You should make sure that the patched Python 2.5 passes all unit tests before submitting your test. Good luck! -- --Guido van Rossum (home page: http://www.python.org/~guido/) ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/dinov%40microsoft.com ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] C99
Martin wrote:
> On 4 June 2016 at 06:11, Benjamin Peterson wrote:
> > PEP 7 requires CPython to use C code conforming to the venerable C89
> > standard. Traditionally, we've been stuck with C89 due to poor C
> > support in MSVC. However, MSVC 2013 and 2015 implement the key
> features of C99.
> > C99 does not offer anything earth-shattering; here are the features I
> > think we'd find most interesting:
> > - Variable declarations can be on any line: removes possibly the most
> > annoying limitation of C89.
> > - Inline functions: We can make Py_DECREF and Py_INCREF inline
> > functions rather than unpleasant macros.
> > - C++-style line comments: Not an killer feature but commonly used.
> > - Booleans
>
> My most-missed C99 feature would be designated initializers. Does MSVC
> support them? It might allow you to do away with those giant pasted slot
> tables, and just write the slots you need:
>
> PyTypeObject PyUnicodeIter_Type = {
> PyVarObject_HEAD_INIT(&PyType_Type, 0)
> .tp_name = "str_iterator",
> .tp_basicsize = sizeof(unicodeiterobject),
> .tp_dealloc = unicodeiter_dealloc,
> .tp_getattro = PyObject_GenericGetAttr,
> .tp_flags = Py_TPFLAGS_DEFAULT | Py_TPFLAGS_HAVE_GC,
> .tp_traverse = unicodeiter_traverse,
> .tp_iter = PyObject_SelfIter,
> .tp_iternext = unicodeiter_next,
> .tp_methods = unicodeiter_methods,
> };
I checked and VC++ does actually support this, and it looks like they support
// comments as well. I don't think it fully supports all of the C99 features
- it appears
They just cherry picked some stuff. The C99 standard library does appear to be
fully
supported with the exception of tgmath.h.
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] frame evaluation API PEP
Mark wrote: > > Dino and I thought of two potential alternatives, neither of which we > > have taken the time to implement and benchmark. One is to simply have > > a hash table of memory addresses to JIT data that is kept on the JIT > > side of things. Obviously it would be nice to avoid the overhead of a > > hash table lookup on every function call. This also doesn't help > > minimize memory when the code object gets GC'ed. > > Hash lookups aren't that slow. If you combine it with the custom flags > suggested by MRAB, then you would only suffer the lookup penalty when > actually entering the special interpreter. > You can use a weakref callback to ensure things get GC'd properly. > > Also, if there is a special extra field on code-object, then everyone will > want > to use it. How do you handle clashes? This is exactly what I've started prototyping and have mostly coded up, I've just been getting randomized and haven't gotten back to it yet. It may have some impact in the short-term but my expectation is that as the JIT gets better that this will become less and less important. Currently we're just JITing one method at a time and have no inlining support. But once we can start putting guards in place and inlining across multiple function calls we will start reducing the transitions from JIT -> Function Call -> JIT and get rid of those hash table lookups entirely. And if we can't succeed at inlining then I suspect the JIT won't end up offering the performance we'd hope for. ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] frame evaluation API PEP
On Mon, Jun 20, 2016 at 12:59 PM, Brett Cannon mailto:br...@python.org>> wrote: MRAB's response made me think of a possible approach: the co_extra field could be the very last field of the PyCodeObject struct and only present if a certain flag is set in co_flags. This is similar to a trick used by X11 (I know, it's long ago :-) But that doesn't resolve your memory worry, right? For a JIT you will have to access the memory regardless for execution count (unless Yury's patch to add caching goes in, in which case it will be provided by code objects already). If you make the code object constructor another function pointer in the interpreter struct, you could solve this quite well IMO. An interpreter with a JIT installed would always create code objects with the co_extra field. But interpreters without a JIT would have have code objects without it. This would mean the people who aren't using a JIT at all don't pay for co_extra. The flag would still be needed so the JIT can tell when you pass it a code object that was created before the JIT was installed (or belonging to a different interpreter). Would that work? Or is it important to be able to import a lot of code and then later import+install the JIT and have it benefit the code you already imported? That’s a pretty interesting idea. We actually load the JIT DLL before we execute any Python code so currently it wouldn’t have any issues with not having the full sized code objects created. But it could also let JITs store all of the info they need right there instead of having to create yet another place to track code data. And it fits in nicely with having the extra data being truly ephemeral that no one else should care about. It doesn’t help with the issue of potentially multiple consumers of that field that has been brought up before but I’m not sure how concerned we should be about that scenario anyway. I still want to check and see what the hash table overhead looks like but if that does end up looking bad I can definitely look at giving this a shot. ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Update on PEP 523 and adding a co_extra field to code objects
So I ran the tests with both a list and a tuple. They were about 5% slower on a handful of benchmarks, and then the difference between the tuple and list again had a few benchmarks that were around 5% slower. There was one benchmark where the tuple one significantly for some reason (mako_v2) which was 1.4x slower. It seems to me we should go with the tuple just because the common case will be having a single object and it'll be even less common to have these changing very frequently. -Original Message- From: Python-Dev [mailto:python-dev-bounces+dinov=microsoft@python.org] On Behalf Of Chris Angelico Sent: Tuesday, August 30, 2016 2:11 PM To: python-dev Subject: Re: [Python-Dev] Update on PEP 523 and adding a co_extra field to code objects On Wed, Aug 31, 2016 at 4:55 AM, Serhiy Storchaka wrote: > On 30.08.16 21:20, Antoine Pitrou wrote: >> >> On Tue, 30 Aug 2016 18:12:01 + >> Brett Cannon wrote: Why not make it always a list? List objects are reasonably cheap in memory and access time... (unlike dicts) >>> >>> >>> Because I would prefer to avoid any form of unnecessary performance >>> overhead for the common case. >> >> >> But the performance overhead of iterating over a 1-element list is >> small enough (it's just an array access after a pointer dereference) >> that it may not be larger than the overhead of the multiple tests and >> conditional branches your example shows. > > > Iterating over a tuple is even faster. It needs one pointer > dereference less. > > And for memory efficiency we can use just a raw array of pointers. Didn't all this kind of thing come up when function annotations were discussed? Insane schemes like dictionaries with UUID keys and so on. The decision then was YAGNI. The decision now, IMO, should be the same. Keep things simple. ChrisA ___ Python-Dev mailing list Python-Dev@python.org https://na01.safelinks.protection.outlook.com/?url=https%3a%2f%2fmail.python.org%2fmailman%2flistinfo%2fpython-dev&data=01%7c01%7cdinov%40microsoft.com%7c9d750b06b2134a2145c708d3d11a4ab0%7c72f988bf86f141af91ab2d7cd011db47%7c1&sdata=szub1gsDW2rdns3IQGV68J3tCqWiNcjqG77xYIfoORc%3d Unsubscribe: https://na01.safelinks.protection.outlook.com/?url=https%3a%2f%2fmail.python.org%2fmailman%2foptions%2fpython-dev%2fdinov%2540microsoft.com&data=01%7c01%7cdinov%40microsoft.com%7c9d750b06b2134a2145c708d3d11a4ab0%7c72f988bf86f141af91ab2d7cd011db47%7c1&sdata=TEzMSyJLmAe2BVZGPugXAh6bga2xN1WQw3bR0z0b%2fLg%3d ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Update on PEP 523 and adding a co_extra field to code objects
So it looks like both list and tuple are about within 5% of using co_extra directly. Using a tuple instead of a list is about a wash except for make_v2 where list is 1.4x slower for some reason (which I didn't dig into). I would say that using a tuple and copying the tuple on updates makes sense as we don't expect these to change very often and we don't expect collisions to happen very often. > -Original Message- > From: Python-Dev [mailto:python-dev- > bounces+dinov=microsoft@python.org] On Behalf Of Chris Angelico > Sent: Tuesday, August 30, 2016 2:11 PM > To: python-dev > Subject: Re: [Python-Dev] Update on PEP 523 and adding a co_extra field to > code objects > > On Wed, Aug 31, 2016 at 4:55 AM, Serhiy Storchaka > wrote: > > On 30.08.16 21:20, Antoine Pitrou wrote: > >> > >> On Tue, 30 Aug 2016 18:12:01 + > >> Brett Cannon wrote: > > Why not make it always a list? List objects are reasonably cheap > in memory and access time... (unlike dicts) > >>> > >>> > >>> Because I would prefer to avoid any form of unnecessary performance > >>> overhead for the common case. > >> > >> > >> But the performance overhead of iterating over a 1-element list is > >> small enough (it's just an array access after a pointer dereference) > >> that it may not be larger than the overhead of the multiple tests and > >> conditional branches your example shows. > > > > > > Iterating over a tuple is even faster. It needs one pointer > > dereference less. > > > > And for memory efficiency we can use just a raw array of pointers. > > Didn't all this kind of thing come up when function annotations were > discussed? Insane schemes like dictionaries with UUID keys and so on. > The decision then was YAGNI. The decision now, IMO, should be the same. > Keep things simple. > > ChrisA > ___ > Python-Dev mailing list > Python-Dev@python.org > https://na01.safelinks.protection.outlook.com/?url=https%3a%2f%2fmail.pyt > hon.org%2fmailman%2flistinfo%2fpython- > dev&data=01%7c01%7cdinov%40microsoft.com%7c9d750b06b2134a2145c70 > 8d3d11a4ab0%7c72f988bf86f141af91ab2d7cd011db47%7c1&sdata=szub1gs > DW2rdns3IQGV68J3tCqWiNcjqG77xYIfoORc%3d > Unsubscribe: > https://na01.safelinks.protection.outlook.com/?url=https%3a%2f%2fmail.pyt > hon.org%2fmailman%2foptions%2fpython- > dev%2fdinov%2540microsoft.com&data=01%7c01%7cdinov%40microsoft.co > m%7c9d750b06b2134a2145c708d3d11a4ab0%7c72f988bf86f141af91ab2d7c > d011db47%7c1&sdata=TEzMSyJLmAe2BVZGPugXAh6bga2xN1WQw3bR0z0b > %2fLg%3d ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
