Re: [Python-Dev] Issue #10348: concurrent.futures doesn't work on BSD
Le jeudi 30 décembre 2010 à 17:05 +0100, "Martin v. Löwis" a écrit : > > I really don't think it is our job to maintain a list of OS/versions > > which work and don't work. > > Of course not. I would propose a dynamic test: check how many POSIX > semaphores the installation supports, and fail if it's less than > 200 (say). I added informations about FreeBSD, NetBSD, Darwin and OpenBSD to the issue #10348: http://bugs.python.org/issue10348#msg125042 The maximum number of POSIX semaphores can be read with sysctl: - FreeBSD: "p1003_1b.sem_nsems_max" - NetBSD: "kern.posix.semmax" - Darwin: "kern.posix.sem.max" - OpenBSD: (no support) Victor ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Possible optimization for LOAD_FAST ?
Le mercredi 29 décembre 2010 à 14:20 +0100, "Martin v. Löwis" a écrit : > Am 28.12.2010 18:08, schrieb Lukas Lueg: > > Also, the > > load_fast in lne 22 to reference x could be taken out of the loop as x > > will always point to the same object > > That's not true; a debugger may change the value of x. That's why Python has the following option: -O : optimize generated bytecode slightly; also PYTHONOPTIMIZE=x I regulary recompile programs with gcc -O0 -g to debug them. It is very difficult to debug (with gdb) a program compiled with gcc -O2: many variables are stored in registers, and gdb doesn't support that correctly. Victor ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] [Python-checkins] r87603 - python/branches/py3k/Misc/NEWS
On Sat, 01 Jan 2011 16:07:02 -0500, Terry Reedy wrote: > On 1/1/2011 5:07 AM, georg.brandl wrote: > > Author: georg.brandl > > Date: Sat Jan 1 11:07:30 2011 > > New Revision: 87603 > > > > Log: > > Fix issue references. > > (add '#' to issue numbers). Whoops, two of those are mine. I am still > learning and will try to remember to include it in both log messages and > NEWS entries. Heh. I think two of them were mine, and I'm supposed to know better by now. -- R. David Murray www.bitdance.com ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] r87577 - in python/branches/py3k: Makefile.pre.in configure configure.in pyconfig.h.in
Am 30.12.2010 15:55, schrieb martin.v.loewis:
> Author: martin.v.loewis
> Date: Thu Dec 30 15:55:47 2010
> New Revision: 87577
>
> Log:
> Build and install libpython3.so.
> Modified: python/branches/py3k/configure.in
> ==
> --- python/branches/py3k/configure.in (original)
> +++ python/branches/py3k/configure.in Thu Dec 30 15:55:47 2010
> @@ -737,6 +738,10 @@
> BLDLIBRARY='-Wl,-R,$(LIBDIR) -L. -lpython$(LDVERSION)'
> RUNSHARED=LD_LIBRARY_PATH=`pwd`:${LD_LIBRARY_PATH}
> INSTSONAME="$LDLIBRARY".$SOVERSION
> + if test $with_pydebug == no
> + then
> + PY3LIBRARY=libpython3.so
> + fi
>;;
> Linux*|GNU*|NetBSD*|FreeBSD*|DragonFly*)
> LDLIBRARY='libpython$(LDVERSION).so'
> @@ -748,6 +753,11 @@
> ;;
> esac
> INSTSONAME="$LDLIBRARY".$SOVERSION
> + PY3LIBRARY=libpython3.so
> + if test $with_pydebug == no
> + then
> + PY3LIBRARY=libpython3.so
> + fi
> ;;
> hp*|HP*)
> case `uname -m` in
These changes do not work as written: if --with-pydebug is not given,
$with_pydebug is empty, not "no". Also, in the second case the
unconditional PY3LIBRARY assignment should probably be deleted.
cheers,
Georg
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Issue #10348: concurrent.futures doesn't work on BSD
> I added informations about FreeBSD, NetBSD, Darwin and OpenBSD to the
> issue #10348:
> http://bugs.python.org/issue10348#msg125042
>
> The maximum number of POSIX semaphores can be read with sysctl:
> - FreeBSD: "p1003_1b.sem_nsems_max"
> - NetBSD: "kern.posix.semmax"
> - Darwin: "kern.posix.sem.max"
> - OpenBSD: (no support)
I've been using os.sysconf("SC_SEM_NSEMS_MAX"), which seems to have
worked fine (it's also what POSIX mandates). See #10798.
Regards,
Martin
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Possible optimization for LOAD_FAST ?
On 1/2/2011 8:17 AM, Victor Stinner wrote: Le mercredi 29 décembre 2010 à 14:20 +0100, "Martin v. Löwis" a écrit : Am 28.12.2010 18:08, schrieb Lukas Lueg: Also, the load_fast in lne 22 to reference x could be taken out of the loop as x will always point to the same object That's not true; a debugger may change the value of x. That's why Python has the following option: -O : optimize generated bytecode slightly; also PYTHONOPTIMIZE=x I regulary recompile programs with gcc -O0 -g to debug them. It is very difficult to debug (with gdb) a program compiled with gcc -O2: many variables are stored in registers, and gdb doesn't support that correctly. Victor, you seem to be equating the gcc -O flag with the Python -O flag. They are described similarly, but can't be used the same way. In particular, there is no Python equivalent to gcc's -O0: there is no way to disable the Python peephole optimizer. --Ned. Victor ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/ned%40nedbatchelder.com ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Possible optimization for LOAD_FAST ?
Cesare Di Mauro gmail.com> writes: > > > 2010/12/28 Lukas Lueg googlemail.com> > > Consider the following code: > def foobar(x): > for i in range(5): > x[i] = i > The bytecode in python 2.7 is the following: > 2 0 SETUP_LOOP 30 (to 33) > 3 LOAD_GLOBAL 0 (range) > 6 LOAD_CONST 1 (5) > 9 CALL_FUNCTION 1 > 12 GET_ITER > >> 13 FOR_ITER 16 (to 32) > 16 STORE_FAST 1 (i) > 3 19 LOAD_FAST 1 (i) > 22 LOAD_FAST 0 (x) > 25 LOAD_FAST 1 (i) > 28 STORE_SUBSCR > 29 JUMP_ABSOLUTE 13 > >> 32 POP_BLOCK > >> 33 LOAD_CONST 0 (None) > 36 RETURN_VALUE > Can't we optimize the LOAD_FAST in lines 19 and 25 to a single load > and put the reference twice on the stack? There is no way that the > reference of i might change in between the two lines. Also, the > load_fast in lne 22 to reference x could be taken out of the loop as x will always point to the same object > > Yes, you can, but you need: > - a better AST evaluator (to mark symbols/variables with proper attributes); > - a better optimizer (usually located on compile.c) which has a "global vision" (not limited to single instructions and/or single expressions). > > It's not that simple, and the results aren't guaranteed to be good. > > Also, consider that Python, as a dynamic-and-not-statically-compiled language need to find a good trade-off between compilation time and execution. > > Just to be clear, a C program is usually compiled once, then executed, so you can spend even *hours* to better optimize the final binary code. > > With a dynamic language, usually the code is compiled and the executed as needed, in "realtime". So it isn't practical neither desirable having to wait too much time before execution begins (the "startup" problem). > > Python stays in a "gray area", because modules are usually compiled once > (when they are first used), and executed many times, but it isn't the only case. > > You cannot assume that optimization techniques used on other (static) languages can be used/ported in Python. > > Cesare > No, it's singularly impossible to prove that any global load will be any given value at compile time. Any optimization based on this premise is wrong. Alex ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Possible optimization for LOAD_FAST ?
On Sun, Jan 2, 2011 at 5:50 PM, Alex Gaynor wrote: > No, it's singularly impossible to prove that any global load will be any given > value at compile time. Any optimization based on this premise is wrong. True. My proposed way out of this conundrum has been to change the language semantics slightly so that global names which (a) coincide with a builtin, and (b) have no explicit assignment to them in the current module, would be fair game for such optimizations, with the understanding that the presence of e.g. "len = len" anywhere in the module (even in dead code!) would be sufficient to disable the optimization. But barring someone interested in implementing something based on this rule, the proposal has languished for many years. FWIW, this is reminiscent of Fortran's rules for "intrinsics" (its name for builtins), which have a similar optimization behavior (except there the potential overrides that the compiler doesn't need to take into account are load-time definitions). -- --Guido van Rossum (python.org/~guido) ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Possible optimization for LOAD_FAST ?
On 1/2/2011 10:18 PM, Guido van Rossum wrote:
My proposed way out of this conundrum has been to change the language
semantics slightly so that global names which (a) coincide with a
builtin, and (b) have no explicit assignment to them in the current
module, would be fair game for such optimizations, with the
understanding that the presence of e.g. "len = len" anywhere in the
module (even in dead code!) would be sufficient to disable the
optimization.
I believe this amounts to saying
1) Python code executes in three scopes (rather than two): global
builtin, modular (misleadingly call global), and local. This much is a
possible viewpoint today.
2) A name that is not an assignment target anywhere -- and that matches
a builtin name -- is treated as a builtin. This is the new part, and it
amounts to a rule for entire modules that is much like the current rule
for separating local and global names within a function. The difference
from the global/local rule would be that unassigned non-builtin names
would be left to runtime resolution in globals.
It would seem that this new rule would simplify the lookup of module
('global') names since if xxx in not in globals, there is no need to
look in builtins. This is assuming that following 'len=len' with 'del
len' cannot 'unmodularize' the name.
For the rule to work 'retroactively' within a module as it does within
functions would require a similar preliminary pass. So it could not work
interactively. Should batch mode main modules work the same as when
imported?
Interactive mode could work as it does at present or with slight
modification, which would be that builtin names within functions, if not
yet overriden, also get resolved when the function is compiled.
--
Terry Jan Reedy
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Possible optimization for LOAD_FAST ?
2011/1/1 Ned Batchelder
> On 12/31/2010 12:51 PM, Cesare Di Mauro wrote:
>
> "Aggressive" optimizations can be enabled with explicit options, in order
> to leave normal "debugger-prone" code.
>
> I wish the Python compiler would adopt a strategy of being able to disable
> optimizations. I wrote a bug about a "leaky abstraction" optimization
> messing up coverage testing 2.5 years ago, and it was closed as won't fix:
> http://bugs.python.org/issue2506. The debate there centered around, "but
> that line isn't executed, because it's been optimized away." It's common in
> sophisticated compilers (as in, any C compiler) to be able to choose whether
> you want optimizations for speed, or disabling optimizations for debugging
> and reasoning about the code. Python would benefit from the same choice.
>
> --Ned.
>
Command line parameters and/or environment variables are suitable for this,
but they aren't immediate and, also, have global effect.
I wish an explicit ("Explicit is better than implicit") and a finer control
over optimizations, with a per-module usage:
from __compiler__ import disable_peepholer, strict_syntax, static_builtins,
globals_as_fasts
Cesare
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Possible optimization for LOAD_FAST ?
2011/1/3 Alex Gaynor > No, it's singularly impossible to prove that any global load will be any > given > value at compile time. Any optimization based on this premise is wrong. > > Alex > That's your opinion, but I have very different ideas. Of course we can't leave the problem only on the compiler shoulders, but I think that can be ways to threat builtins as "static" variables, and globals like local (fast) variables too, taking into account changes on the builtins' and modules dictionaries. But it doesn't make sense to invest time in these things: JITs are becoming a good alternative, and may be they will be ready soon to take the CPython place as the "mainstream" implementation. Cesare ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Possible optimization for LOAD_FAST ?
On Mon, Jan 3, 2011 at 3:36 PM, Terry Reedy wrote:
> I believe this amounts to saying
>
> 1) Python code executes in three scopes (rather than two): global builtin,
> modular (misleadingly call global), and local. This much is a possible
> viewpoint today.
>
> 2) A name that is not an assignment target anywhere -- and that matches a
> builtin name -- is treated as a builtin. This is the new part, and it
> amounts to a rule for entire modules that is much like the current rule for
> separating local and global names within a function. The difference from the
> global/local rule would be that unassigned non-builtin names would be left
> to runtime resolution in globals.
>
> It would seem that this new rule would simplify the lookup of module
> ('global') names since if xxx in not in globals, there is no need to look in
> builtins. This is assuming that following 'len=len' with 'del len' cannot
> 'unmodularize' the name.
>
> For the rule to work 'retroactively' within a module as it does within
> functions would require a similar preliminary pass. So it could not work
> interactively. Should batch mode main modules work the same as when
> imported?
>
> Interactive mode could work as it does at present or with slight
> modification, which would be that builtin names within functions, if not yet
> overriden, also get resolved when the function is compiled.
This could potentially be handled by having the "exec" mode in
compile() assume it can see all the global assignments (and hence
assume builtin names refer to the builtins), while "single" would
assume this was not the case (and hence skip the optimisation). It may
also need an additional parameter to tell the compiler which names are
known to be visible in the current locals and globals (e.g. to allow
exec() to do the right thing)
This kind of issue is the reason Guido pointed out the idea really
needs someone else to pick it up and run with it - not just to figure
out the implementation details, but also to ferret out the full
implications for the language semantics and backwards compatibility.
Cheers,
Nick.
--
Nick Coghlan | ncogh...@gmail.com | Brisbane, Australia
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
