[Python-Dev] PYC filename and optimization
Hi,
While working on the FAT Python project to try to optimize CPython, I
have an issue with the cached .pyc files. I would like to have one
.pyc without optimization when the FAT mode is disabled (default) and
one .pyc with optimization when the FAT mode is enabled (-F command
line option). Currently, I'm using this filenames:
- Lib/__pycache__/os.cpython-36.pyc: default mode
- Lib/__pycache__/os.cpython-36.fat-0.pyc: FAT mode
With -O:
- Lib/__pycache__/os.cpython-36.opt-1.pyc: default mode
- Lib/__pycache__/os.cpython-36.fat-1.pyc: FAT mode
With -OO:
- Lib/__pycache__/os.cpython-36.opt-2.pyc: default mode
- Lib/__pycache__/os.cpython-36.fat-2.pyc: FAT mode
Is it the "correct" way to name .pyc files? I had to modify unit tests
which expected an exact filename.
An alternative would be to add a new dot:
- Lib/__pycache__/os.cpython-36.fat.pyc
- Lib/__pycache__/os.cpython-36.fat.opt-1.pyc
- Lib/__pycache__/os.cpython-36.fat.opt-2.pyc
Or should I add a flag to the cache tag ("cpython-36")?
- Lib/__pycache__/os.cpython-36f.pyc
- Lib/__pycache__/os.cpython-36f.opt-1.pyc
- Lib/__pycache__/os.cpython-36f.opt-2.pyc
CPython adds the "d" suffix to the cache tag in debug build, but in my
case the flag would be added dynamically at Python startup. The FAT
mode is enabled by the -F command line option, not by a compilation
flag.
Victor
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Second milestone of FAT Python
Hi, 2015-11-04 23:20 GMT+01:00 Terry Reedy : > On 11/4/2015 3:50 AM, Victor Stinner wrote: >> (...) I reached the second milestone: >> it's now possible to run the full Python test suite with these >> optimizations enabled. It confirms that the optimizations don't break >> the Python semantic. > > Is the test suite complete enough to say this? (see below) > (...) > In particular, does the test suite have tests like this, to verify that > over-riding builtins works? I modified the AST optimizer to not emit guards (remove all guards), tests still pass. So: no, the optimized code doesn't override builtin functions. But I have dedicated unit tests for that in test_fat ;-) Victor ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PYC filename and optimization
On Nov 05, 2015, at 11:33 AM, Victor Stinner wrote: >- Lib/__pycache__/os.cpython-36.pyc: default mode >- Lib/__pycache__/os.cpython-36.fat-0.pyc: FAT mode > >With -O: > >- Lib/__pycache__/os.cpython-36.opt-1.pyc: default mode >- Lib/__pycache__/os.cpython-36.fat-1.pyc: FAT mode > > >With -OO: > >- Lib/__pycache__/os.cpython-36.opt-2.pyc: default mode >- Lib/__pycache__/os.cpython-36.fat-2.pyc: FAT mode > >Is it the "correct" way to name .pyc files? I had to modify unit tests >which expected an exact filename. I think this is the logical extension of PEP 488 for your use case. Cheers, -Barry ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PYC filename and optimization
On Thu, 5 Nov 2015 at 07:41 Barry Warsaw wrote: > On Nov 05, 2015, at 11:33 AM, Victor Stinner wrote: > > >- Lib/__pycache__/os.cpython-36.pyc: default mode > >- Lib/__pycache__/os.cpython-36.fat-0.pyc: FAT mode > > > >With -O: > > > >- Lib/__pycache__/os.cpython-36.opt-1.pyc: default mode > >- Lib/__pycache__/os.cpython-36.fat-1.pyc: FAT mode > > > > > >With -OO: > > > >- Lib/__pycache__/os.cpython-36.opt-2.pyc: default mode > >- Lib/__pycache__/os.cpython-36.fat-2.pyc: FAT mode > > > >Is it the "correct" way to name .pyc files? I had to modify unit tests > >which expected an exact filename. > > I think this is the logical extension of PEP 488 for your use case. > It will require tweaking importlib.util.cache_from_source() to no longer restriction to alphanumeric-only for the optimization level, but that can be tweaked to use `opt-` when given an alphanumeric and then to blindly use another value otherwise. ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Rationale behind lazy map/filter
On Thu, 05 Nov 2015 03:59:05 +, Michael Selik wrote: > > I'm not suggesting restarting at the top (I've elsewhere suggested that > > many such methods would be better as an *iterable* that can be restarted > > at the top by calling iter() multiple times, but that's not the same > > thing). I'm suggesting raising an exception other than StopIteration, so > > that this situation can be detected. If you are writing code that tries > > to resume iterating after the iterator has been exhausted, I have to > > ask: why? > > The most obvious case for me would be tailing a file. Loop over the lines > in the file, sleep, then do it again. There are many tasks analogous to > that scenario -- anything querying a shared resource. The 'file' iterator actually breaks the rules of iterators: it does *not* continue to raise StopIteration once it has been exhausted, if more input becomes available. Given that it is one of the most commonly used iterators (and I would not be surprised if other special-purpose iterators copied its design), this pattern does seem like a blocker for the proposal. --David ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
