from:"\"p\""



New submission from Gregory P. Smith :

Now that we have a shiny new parser, can we finally get rid of this language 
wart:

assert thing, description  # works as intended

assert (thing, description)  # always True as non-empty tuples are Truthy

This most often happens when extending thing or description beyond a single 
line on assert statements as () are the natural way to do that and as it is 
with assert being a statement, knowing specifically where to place the ()s to 
not fall into the pit of snakes of unintentionally nerfing your assertion to be 
an always true tuple is hard for human authors.

This would obsolete the pylint error about tuple assertion and enable more 
natural assert use.

py.test framework users would presumably rejoice as well.


This parsing change would need a PEP.  I fail to see any obvious downsides 
though.

--
components: Parser
messages: 409101
nosy: gregory.p.smith, lys.nikolaou, pablogsal
priority: normal
severity: normal
status: open
title: Parse assert (x == y, "Descriptive text") as statement params instead of 
a tuple
versions: Python 3.11

___
Python tracker 
<https://bugs.python.org/issue46167>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46167] Parse assert (x == y, "Descriptive text") as statement params instead of a tuple



Gregory P. Smith  added the comment:

It's not about an advantage, it's about removing the problem of what edit to 
make when working on


assert 
thing_that_has_a_meaningful_name.methods_have_good_names(value_from_somewhere) 
== other_thing_that_is_meaningful, "Description of what the issue is if this 
fails for humans, as if the names weren't enough"

and making that fit within whatever line length limit your codebase has.

put () in the wrong place and it triggers the long standing Python wart or 
parsing as a tuple.

rather than warn about the syntax wart, we should just do the thing code 
authors want in the first place.

--

___
Python tracker 
<https://bugs.python.org/issue46167>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46150] test_pathlib assumes "fakeuser" does not exist as user



Change by Gregory P. Smith :


--
assignee:  -> gregory.p.smith
nosy: +gregory.p.smith

___
Python tracker 
<https://bugs.python.org/issue46150>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46150] test_pathlib assumes "fakeuser" does not exist as user



Change by Gregory P. Smith :


--
pull_requests: +28466
pull_request: https://github.com/python/cpython/pull/30244

___
Python tracker 
<https://bugs.python.org/issue46150>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46150] test_pathlib assumes "fakeuser" does not exist as user



Gregory P. Smith  added the comment:


New changeset b8de8b7039cee47465b2af9950b0b9ed2d3f2903 by Nikita Sobolev in 
branch 'main':
bpo-46150: ensure `fakeuser` does not exist in `PosixPathTest.test_expanduser` 
(GH-30240)
https://github.com/python/cpython/commit/b8de8b7039cee47465b2af9950b0b9ed2d3f2903


--

___
Python tracker 
<https://bugs.python.org/issue46150>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue10202] ftplib doesn't check close status after sending file

Gregory P. Smith added the comment:

[pasting in my investigation that I replied with on a mailing list:]

The possible problem does appear real but it likely not frequently observed and
is something most people reading the code wouldn't see as it's rather esoteric:

https://blog.netherlabs.nl/articles/2009/01/18/the-ultimate-so_linger-page-or-why-is-my-tcp-not-reliable
via
https://stackoverflow.com/questions/8874021/close-socket-directly-after-send-unsafe
describes the scenario.

So does this apply in Python?

Apparently. Our socket module close() is simply calling close as expected.
https://github.com/python/cpython/blob/main/Modules/socketmodule.c#L3132 ->
https://github.com/python/cpython/blob/main/Modules/socketmodule.c#L443

Our ftplib code that implicitly calls close on the data connection when exiting
the
`with self.transfercmd(cmd, rest) as conn: ` context manager
https://github.com/python/cpython/blob/main/Lib/ftplib.py#L498`

Triggers a close() without a prior shutdown(socket.SHUT_RDWR) + read() -> EOF
or timeout dance.

In most protocols this isn't a big deal. ftp being so old as to rely solely on
the TCP connection itself is the outlier.

How often does this happen in practice? I don't know. I haven't heard of it
happening, but how many people honestly use ftplib to send data between
operating systems where a socket close triggering a TCP RST rather than FIN on
the sender might be more likely in the past 20 years? I suspect not many.

The code can be improved to prevent it. But I don't believe it'll be feasible
to construct a real world not-mock-filled regression test that would fail
without it.

Potential solution:
Do the shutdown+recv EOF dance as the final step inside of the storbinary and
storlines `with self.transfercmd as conn` blocks.

Technically speaking socket.socket.shutdown() is conditionally compiled but I
don't know if any platforms lacking the socket shutdown API exist anymore (my
guess is that conditional compilation for shutdown is a leftover from the
1990s). If so, a "if hasattr(conn, 'shutdown'):" conditional before the added
logic would suffice.

Looking at various ftp client source code (netkit ftp & netbsd ftp - both bsd
derivatives) as well as a modern gonuts golang one I do not see them explicitly
doing the shutdown dance. (I didn't dive in to figure out if it is hidden
within their flclose() or conn.Close() implementations as that'd be surprising)

--
nosy: +gregory.p.smith

___
Python tracker
<https://bugs.python.org/issue10202>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue10202] ftplib doesn't check close status after sending file



Change by Gregory P. Smith :


--
versions: +Python 3.10, Python 3.11, Python 3.9 -Python 2.7, Python 3.1, Python 
3.2

___
Python tracker 
<https://bugs.python.org/issue10202>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue10202] ftplib doesn't check close status after sending file



Change by Gregory P. Smith :


--
stage: test needed -> needs patch

___
Python tracker 
<https://bugs.python.org/issue10202>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46120] Add note to `typing.Union` that it is recommended to use `|` instead



Gregory P. Smith  added the comment:


New changeset 1b30660c3b40da557050948ac8635f2dc50c4ee2 by Nikita Sobolev in 
branch 'main':
bpo-46120: State that `|` is preferred over `Union` (GH-30222)
https://github.com/python/cpython/commit/1b30660c3b40da557050948ac8635f2dc50c4ee2


--
nosy: +gregory.p.smith

___
Python tracker 
<https://bugs.python.org/issue46120>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46150] test_pathlib assumes "fakeuser" does not exist as user

2021-12-27 Thread Gregory P. Smith



Change by Gregory P. Smith :


--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 
<https://bugs.python.org/issue46150>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46339] PEG parser segfault from ast.literal_eval

2022-01-10 Thread Gregory P. Smith



New submission from Gregory P. Smith :

cpython/b$ ./python 
Python 3.11.0a3+ (heads/main-dirty:081a214008, Jan 11 2022, 02:48:22) [GCC 
11.2.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import ast
>>> ast.literal_eval('''F"""
... 
... 
... {6
... 0="""''')
Segmentation fault

this was discovered by oss-fuzz.

This may be a recent regression?  It didn't happen on my build from a few weeks 
ago.  worth testing on older versions.

--
assignee: pablogsal
components: Parser
messages: 410270
nosy: gregory.p.smith, lys.nikolaou, pablogsal
priority: normal
severity: normal
stage: needs patch
status: open
title: PEG parser segfault from ast.literal_eval
type: crash
versions: Python 3.11

___
Python tracker 
<https://bugs.python.org/issue46339>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46349] inspect.getdoc() should append parent class method docs when encountering a local one line docstring

2022-01-11 Thread Gregory P. Smith



New submission from Gregory P. Smith :

Today `inspect.getdoc()` is quite simple. If a method has any docstring, it 
returns it.

There is one idiom where this is not very useful to our users.  A """See base 
class.""" docstring on a method.

Rather than having to repeat the canonical base class documentation in all 
methods or argue with tooling about not having a docstring in this common 
scenario, a single line docstring being used as a hint to just bring in the 
parent's docstring would be more helpful to the user.

Today's inspect.getdoc() could would turn from:
https://github.com/python/cpython/blob/main/Lib/inspect.py#L850

into something like:

```
...
if doc is None or (isinstance(doc, str) and '\n' not in doc.strip()):
  try:
  parent_doc = _finddoc(object)
  if doc is None:
  doc = parent_doc
  elif isinstance(parent_doc, str):
  doc = f'{doc}\nParent class MRO doc:\n{parent_doc}'
  except (AttributeError, TypeError):
  return doc
   ...
```

Why not just tell people to omit docstrings when they want parent docs?  
Because not all coding styles and linter tooling allows for this as they aim to 
enforce that documentation _is_ written.  Source analysis tooling and IDEs may 
not have a whole transitive dependency view of where the base classes even come 
from and are thus incapable of reliably analyzing whether a parent MRO method 
even exists or has a docstring.  This allows for trivial one line docstrings in 
that situation while still providing better information to the help() user.

This would add value to when using 
help(typical_subclass_of_abstract_thing.method) in notebooks.

--
components: Library (Lib)
messages: 410335
nosy: gregory.p.smith
priority: normal
severity: normal
stage: needs patch
status: open
title: inspect.getdoc() should append parent class method docs when 
encountering a local one line docstring
type: enhancement
versions: Python 3.11

___
Python tracker 
<https://bugs.python.org/issue46349>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46349] inspect.getdoc() should append parent class method docs when encountering a local one line docstring

2022-01-11 Thread Gregory P. Smith



Gregory P. Smith  added the comment:

>From the context of help(typical_subclass_of_abstract_thing.method), the user 
>isn't told what the base classes are or which one might have documentation.  
>So it could become an exercise in frustration.

and LOL yes the parameter name is silly but that's a separate module wide 
cleanup that could be done - inspect is full of public APIs that shadow object 
as a parameter name.  it is often quite old code.

--

___
Python tracker 
<https://bugs.python.org/issue46349>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46347] memory leak in PyEval_EvalCodeEx

2022-01-11 Thread Gregory P. Smith



Gregory P. Smith  added the comment:

when was the regression introduced?

I wouldn't necessarily rush an urgent release out unless this was just 
introduced in the 3.10.1 patch release.  Could it wait for 3.10.2 already 
scheduled for four weeks from now?

--
nosy: +gregory.p.smith

___
Python tracker 
<https://bugs.python.org/issue46347>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27273] subprocess.run(cmd, input='text') should pass universal_newlines=True to Popen

2022-01-12 Thread Gregory P. Smith



Gregory P. Smith  added the comment:

agreed.

--
resolution:  -> rejected
stage: patch review -> resolved
status: open -> closed

___
Python tracker 
<https://bugs.python.org/issue27273>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40255] Fixing Copy on Writes from reference counting

2022-01-14 Thread Gregory P. Smith



Gregory P. Smith  added the comment:

[data] I finally dug up the old YouTube doc at work with their findings. Mostly 
just posting this here for future public reference if anyone wants. Nothing 
surprising.

When youtube experimented with a modified 2.7 adding "eternal refcounts" in 
2015, they saw a 3-5% CPU performance regression. (not the 10% I had in my mind)

Their version of this simply set a high bit on the refcount as the indicator 
and added the obvious conditional into the Py_INCREF/Py_DECREF macros.

Unsurprisingly in line with what others have since found. For their preforked 
server and decision of what to mark eternal before forking, it saved them 10% 
ram (fewer copy on writes). The -ram vs +cpu +maintenance cost tradeoff wound 
up not being worthwhile to them though. Their motivation for trying was 
entirely COW memory savings.

=== CPython 2.7 object.h modification they used:

```
+#ifdef GOOGLE_ETERNAL_REFCOUNT_SUPPORT
+
+#define PY_ETERNAL_REFCOUNT (PY_SSIZE_T_MAX / 2)
+
+#define Py_IS_ETERNAL(op) (   \
+  ((PyObject*)(op))->ob_refcnt >= PY_ETERNAL_REFCOUNT)
+
+#define Py_SET_ETERNAL(op)\
+  do {\
+  ((PyObject*)(op))->ob_refcnt = PY_ETERNAL_REFCOUNT; \
+  if (PyObject_IS_GC(op)) {   \
+PyObject_GC_UnTrack(op);  \
+  }   \
+  } while (0)
+
+#define Py_INCREF(op) (   \
+  Py_IS_ETERNAL(op)   \
+?  PY_ETERNAL_REFCOUNT\
+:  (_Py_INC_REFTOTAL  _Py_REF_DEBUG_COMMA \
+   ((PyObject*)(op))->ob_refcnt++)\
+  )
+
+#define Py_DECREF(op)   \
+do {\
+if (Py_IS_ETERNAL(op)) break;   \
+if (_Py_DEC_REFTOTAL  _Py_REF_DEBUG_COMMA   \
+--((PyObject*)(op))->ob_refcnt != 0)\
+_Py_CHECK_REFCNT(op)\
+else\
+_Py_Dealloc((PyObject *)(op));  \
+} while (0)
+
+#else
```

--

___
Python tracker 
<https://bugs.python.org/issue40255>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46406] optimize int division



New submission from Gregory P. Smith :

Based on a python-dev thread, we've come up with faster int division code for 
CPython's bignums.

https://mail.python.org/archives/list/python-...@python.org/thread/ZICIMX5VFCX4IOFH5NUPVHCUJCQ4Q7QM/#NEUNFZU3TQU4CPTYZNF3WCN7DOJBBTK5

filing this issue for starters to attach a PR to.  details forthcoming.

--
messages: 410735
nosy: gregory.p.smith
priority: normal
severity: normal
stage: needs patch
status: open
title: optimize int division
type: performance
versions: Python 3.11

___
Python tracker 
<https://bugs.python.org/issue46406>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46406] optimize int division



Change by Gregory P. Smith :


--
keywords: +patch
pull_requests: +28829
stage: needs patch -> patch review
pull_request: https://github.com/python/cpython/pull/30626

___
Python tracker 
<https://bugs.python.org/issue46406>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46406] optimize int division



Gregory P. Smith  added the comment:

The PR was directly inspired by Mark Dickinson's code in the email thread 
directly using __asm__ to get the instruction he wanted.  There is usually a 
way to make the compiler actually do what you intend.  This appears to be it.

Interestingly, experimenting with small code snippets rather than the entire 
longobject.c on gotbolt.org to check various compilers output does not always 
yield as nice of a result.  (clang 11+ showed promise there, but this change 
benefits gcc equally as well in real world CPython microbenchmark timeit 
tests).  https://godbolt.org/z/63eWPczjx was my playground code.

```
$ ./b-clang13/python -m timeit -n 150 -s 'x = 10**1000; r=x//10; assert r 
== 10**999, r' 'x//17'
150 loops, best of 5: 450 nsec per loop
$ ./b-clang13-new-basic-divrem1/python -m timeit -n 150 -s 'x = 10**1000; 
r=x//10; assert r == 10**999, r' 'x//17'
150 loops, best of 5: 375 nsec per loop
$ ./b-gcc9/python -m timeit -n 150 -s 'x = 10**1000; r=x//10; assert r == 
10**999, r' 'x//17'
150 loops, best of 5: 448 nsec per loop
$ ./b-gcc9-new-basic-divrem1/python -m timeit -n 150 -s 'x = 10**1000; 
r=x//10; assert r == 10**999, r' 'x//17'
150 loops, best of 5: 370 nsec per loop
```

That's on an AMD zen3 (x86_64).  Also tested with other divisors, 17 is not 
specialized by the compiler.  These were not --enable-optimizations builds, 
though the results remain similar on those for non-specialized values as x//10 
turns into when using -fprofile-values on gcc9.

Performance tests using other architectures forthcoming.

A pyperformance suite run on a benchmark-stable host is worthwhile. I don't 
actually expect this to show up as significant in most things there; we'll see.

The new code is not any more difficult to maintain than the previous code 
regardless.

--

___
Python tracker 
<https://bugs.python.org/issue46406>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46406] optimize int division



Change by Gregory P. Smith :


--
nosy: +mark.dickinson

___
Python tracker 
<https://bugs.python.org/issue46406>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46406] optimize int division