[issue37593] ast.arguments has confusing args/posonlyargs ordering

[Benjamin S Wolf]

New submission from Benjamin S Wolf :

Positional-only arguments come before position-or-keyword arguments.

def f(pos1, pos2, /, pos_or_kwd, *, kwd1, kwd2):

However, the posonlyargs are defined to come after args in the AST:

arguments = (arg* args, arg* posonlyargs, arg? vararg, arg* kwonlyargs,
 expr* kw_defaults, arg? kwarg, expr* defaults)

which results in confusing ast.dump output because they share defaults:

>>> r = ast.parse('lambda a=1,/,b=2:a+b', mode='eval')
>>> ast.dump(r.body.args)
"arguments(
args=[arg(arg='b', annotation=None, type_comment=None)],
posonlyargs=[arg(arg='a', annotation=None, type_comment=None)],
vararg=None, kwonlyargs=[], kw_defaults=[], kwarg=None,
defaults=[Constant(value=1, kind=None), Constant(value=2, kind=None)])"
[manually prettified]

Note how the ordering is 'args b', then 'posonlyargs a', but the defaults are 
still 1 then 2. This can be confusing to someone building an ast.arguments 
using keywords because the elements in 'defaults' have to be supplied in a 
specific order, but the keyword args 'args' and 'posonlyargs' do not, or to 
someone building an ast.arguments using positional arguments (because, maybe 
ironically, they're not keyword-only arguments) because 'posonlyargs' and 
'args' must be supplied in a different order than the ordering of elements in 
'defaults' would imply.

Potential solutions:
 1. Swap posonlyargs and args.
 2. Add a separate pos_defaults list.

--
messages: 347932
nosy: Benjamin.S.Wolf
priority: normal
severity: normal
status: open
title: ast.arguments has confusing args/posonlyargs ordering
type: behavior
versions: Python 3.8

___
Python tracker 
<https://bugs.python.org/issue37593>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue15230] runpy.run_path broken: Breaks scoping; pollutes sys.modules; doesn't set __package__

[Benjamin S Wolf]

New submission from Benjamin S Wolf :

(Python 3.2.3)

1. After discarding the module run_path used to run the code in, all references 
to variables from local scopes (even if they are references to global 
variables) are bound to None, preventing any code in functions from running 
properly.

/tmp/a.py --
FOO = 'bar'

def f():
print(FOO)

f()

/tmp/b.py --
# Hack needed for:
#  python3 /tmp/b.py,
#  python3 -m /tmp/b
#  runpy.run_path('/tmp/b.py')
from os.path import dirname
__path__ = [dirname(__file__)]
del dirname

# Hack needed for:
#  python3 -m /tmp/b
if __name__ == '__main__' and not __package__:
__package__ = '__main__'

from . import a

def g():
print(a.FOO)

g()


~$ python3
>>> import runpy
>>> d = runpy.run_module('/tmp/a')
bar
>>> d2 = runpy.run_path('/tmp/a.py')
bar
>>> d['f']

>>> d['FOO']
'bar'
>>> d['f']()
bar
>>> d2['f']

>>> d2['FOO']
'bar'
>>> d2['f']()
None
>>> d3 = runpy.run_path('/tmp/b.py')
bar
bar
>>> d3['g']

>>> d3['a']
.a' from '/tmp/a.py'>
>>> d3['g']()
Traceback (most recent call last):
  File "", line 1, in 
  File "/tmp/b.py", line 15, in g
print(a.FOO)
AttributeError: 'NoneType' object has no attribute 'FOO'

Notice that run_module gets this right, as d['f']() prints 'bar' but d2['f']() 
and d3['g']() do not.


2. run_path pollutes the module namespace when running a module that uses 
relative imports. This prevents any code that imports the same module in the 
same manner from running.

Continuing from #1 without having closed the interpreter:
>>> d4 = runpy.run_path('/tmp/b.py')
Traceback (most recent call last):
  File "", line 1, in 
  File "/usr/lib/python3.2/runpy.py", line 250, in run_path
return _run_module_code(code, init_globals, run_name, path_name)
  File "/usr/lib/python3.2/runpy.py", line 83, in _run_module_code
mod_name, mod_fname, mod_loader, pkg_name)
  File "/usr/lib/python3.2/runpy.py", line 73, in _run_code
exec(code, run_globals)
  File "/tmp/b.py", line 12, in 
from . import a
ImportError: cannot import name a
>>> '' in sys.modules
False
>>> '.a' in sys.modules
True
>>> d3['a'].f()
bar
>>> del sys.modules['.a']
>>> d4 = runpy.run_path('/tmp/b.py')
bar
bar
>>>

run_module, on the other hand, also alters sys.modules, but this does not 
prevent the module from being run, only from the secondary module from being 
re-imported:
[Create an empty file /tmp/__init__.py]
>>> sys.path = ['/'] + sys.path
>>> d5 = runpy.run_module('tmp.b')
bar
bar
>>> d6 = runpy.run_module('tmp.b')
bar
>>> d7 = runpy.run_module('tmp.b')
bar
>>> 'tmp' in sys.modules
True
>>> 'tmp.b' in sys.modules
False
>>> 'tmp.a' in sys.modules
True

[This was the only way I could get run_module to run /tmp/b.py, regardless of 
the presence or lack of the path and __package__ hacks at the top of the file, 
or any other changes I've experimented with. runpy.run_module('/tmp/b'), 
runpy.run_module('b') [with '/tmp' in sys.path] would generally result in:
  ValueError: Attempted relative import in non-package
and setting run_name='__main__' alongside any of the other changes would result 
in:
  ImportError: cannot import name a

python3 /tmp/b.py and python3 -m /tmp/b run fine.]


3. And finally, an examination of the run_path code shows that it doesn't, as 
the docs indicate, set __package__ to be run_name.rpartition('.')[0], but 
either the empty string or None: 
http://hg.python.org/cpython/file/3.2/Lib/runpy.py#l269

--
components: Library (Lib)
messages: 164437
nosy: Benjamin.S.Wolf
priority: normal
severity: normal
status: open
title: runpy.run_path broken: Breaks scoping; pollutes sys.modules; doesn't set 
__package__
type: behavior
versions: Python 3.2

___
Python tracker 
<http://bugs.python.org/issue15230>
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com