[issue37712] Exception frames from unittest.TestCase.fail dependent on nesting

[Karthikeyan Singaravelan]

Change by Karthikeyan Singaravelan :


--
nosy: +ezio.melotti, michael.foord, rbcollins

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37717] argparse subcommand docs has non-existent parameter "action"

[retnikt]

New submission from retnikt :

In the library documentation for argparse, the section for 
ArgumentParser.add_subparsers ( 
https://docs.python.org/3/library/argparse.html#sub-commands ) states that 
there is a parameter for 'action' with the description 'the basic type of 
action to be taken when this argument is encountered at the command line'. 
However, no such parameter actually exists, and passing it to the function 
causes very strange behaviour:

Traceback (most recent call last):
  File "", line 1, in 
  File "/usr/lib/python3.6/argparse.py", line 1716, in add_subparsers
action = parsers_class(option_strings=[], **kwargs)
TypeError: __init__() got an unexpected keyword argument 'parser_class'

This line should be removed from the documentation. It is present in versions 
3.4+ and 2.7

--
assignee: docs@python
components: Documentation
messages: 348718
nosy: docs@python, retnikt
priority: normal
severity: normal
status: open
title: argparse subcommand docs has non-existent parameter "action"
type: behavior
versions: Python 2.7, Python 3.5, Python 3.6, Python 3.7, Python 3.8, Python 3.9

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36670] test suite broken due to cpu usage feature on win 10/ german

[Sebastian Koslowski]

Sebastian Koslowski  added the comment:

I ran into this issue on a Win10 German box running the tests for version 3.7.4

Your changes fixes the issue for me.

--
nosy: +skoslowski
versions: +Python 3.7 -Python 3.8

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37717] argparse subcommand docs has non-existent parameter "action"

[Karthikeyan Singaravelan]

Change by Karthikeyan Singaravelan :


--
nosy: +paul.j3

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37718] 2to3 exception handling

[Xinmeng Xia]

New submission from Xinmeng Xia :

we run the converted Python3 code, the following error will happen:
Traceback (most recent call last):
  File "/home/xxm/Desktop/instrument/datasetpy3/Labeled-LDA-Python/example.py", 
line 50, in 
llda_model.save_model_to_dir(save_model_dir)
  File 
"/home/xxm/Desktop/instrument/datasetpy3/Labeled-LDA-Python/model/labeled_lda.py",
 line 697, in save_model_to_dir
LldaModel._write_object_to_file(save_model_path, save_model.__dict__)
  File 
"/home/xxm/Desktop/instrument/datasetpy3/Labeled-LDA-Python/model/labeled_lda.py",
 line 650, in _write_object_to_file
print(("%s\n\t%s" % (message, e.message)))
AttributeError: 'TypeError' object has no attribute 'message'

it seems that attributes change between Python2 and Python3. However 2to3 lack 
for this fix when dealing with exception fix.

--
components: 2to3 (2.x to 3.x conversion tool)
files: labeled_lda.py
messages: 348720
nosy: xxm
priority: normal
severity: normal
status: open
title: 2to3  exception handling
versions: Python 3.7
Added file: https://bugs.python.org/file48516/labeled_lda.py

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37717] argparse subcommand docs has non-existent parameter "action"

[Karthikeyan Singaravelan]

Karthikeyan Singaravelan  added the comment:

Seems related : issue23487 . kwargs is manipulated before passing to 
parsers_class causing the error message and also the discussion in issue23487 
notes this to be a documentation issue over usage of action argument.

--
nosy: +xtreak

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37716] 2to3 Accuracy of calculation

[Mark Dickinson]

Mark Dickinson  added the comment:

This is essentially a duplicate of #12831. There's nothing 2to3 can reasonably 
do here, since it isn't in a position to guess whether true division or floor 
division was intended. As noted in the duplicate, the "-3" and the "-Q" command 
line options can help detect these cases.

--
nosy: +mark.dickinson
resolution:  -> duplicate
stage:  -> resolved
status: open -> closed
superseder:  -> 2to3 and integer division

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37474] Should Py_Initialize() control the floating point mode?

[Mark Dickinson]

Mark Dickinson  added the comment:

> Should I understand that fedisableexcept(FE_OVERFLOW) is useless since 
> FreeBSD 6?

That's my understanding, yes. And since it was only introduced in FreeBSD 6, 
it's been useless forever. IOW, I think it's true that this line of code, in 
its current form, hasn't ever had any effect.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37474] Should Py_Initialize() control the floating point mode?

[Mark Dickinson]

Mark Dickinson  added the comment:

> And since it was only introduced 

Sorry, that was unclear. "it" refers to "fedisableexcept" in the above.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37474] Should Py_Initialize() control the floating point mode?

[STINNER Victor]

STINNER Victor  added the comment:

Mark: Do you think that it's worth it to convert attached fp_except.c into 
tests run by test_float, to check floating point exceptions in Python?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37714] 2to3 tab Problems

[Mark Dickinson]

Mark Dickinson  added the comment:

Duplicate of #15332.

--
nosy: +mark.dickinson
resolution:  -> duplicate
stage:  -> resolved
status: open -> closed
superseder:  -> 2to3 should fix bad indentation (or warn about it)

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37474] Should Py_Initialize() control the floating point mode?

[Mark Dickinson]

Mark Dickinson  added the comment:

> Mark: Do you think that it's worth it to convert attached fp_except.c into
> tests run by test_float, to check floating point exceptions in Python?

Sure, it wouldn't harm. I'd expect that all these cases are already being 
tested indirectly by some part of test_math, but direct tests are better.

Note that Python still can't/won't assume IEEE 754 floating-point, so if the 
tests use special values like infinities and nans, or other IEEE 754 
assumptions, then they should be guarded by a suitable "skipIf" decorator.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37719] addSubtest not calling addFailure and addError

[Xavier Dollé]

New submission from Xavier Dollé :

addSubTest from TestResult is appending elements to failures and errors without 
using addFailure or addError, making the extend of this class more difficult.

suggestion:

def addSubTest(self, test, subtest, err):
...
if issubclass(err[0], test.failureException):
self.addFailure(subtest, err)
else:
self.addError(subtest, err)

The suggested change would make it more concise and easier to extend.

--
components: Tests
messages: 348728
nosy: Xavier Dollé
priority: normal
severity: normal
status: open
title: addSubtest not calling addFailure and addError
type: enhancement
versions: Python 3.9

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37719] addSubtest not calling addFailure and addError

[Karthikeyan Singaravelan]

Karthikeyan Singaravelan  added the comment:

See also issue29152 which seems to raise same concerns over this behvavior.

--
nosy: +ezio.melotti, michael.foord, rbcollins, xtreak

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37631] EXTRA_CFLAGS get overrided by CFLAGS_NODIST

[Miro Hrončok]

Change by Miro Hrončok :


--
keywords: +patch
pull_requests: +14781
stage:  -> patch review
pull_request: https://github.com/python/cpython/pull/15020

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37631] EXTRA_CFLAGS get overrided by CFLAGS_NODIST

[Miro Hrončok]

Change by Miro Hrončok :


--
nosy: +hroncok

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37720] Crash of python3.7 with virt-manager

[Hans Peter]

New submission from Hans Peter :

Hi!

A few days ago, I upgraded to UbuntuMate 19.04.

I can't run 'virt-manager' because of this:

# virt-manager
Output: Segmentation fault


kernel: [ 2003.888116] virt-manager[16014]: segfault at 32d0 ip 
32d0 sp 7ffeb09ac658 error 14 in python3.7[40+21000]

kernel: [ 2003.888124] Code: Bad RIP value.

Look at "crash_and_packages.log" I added to attachment.

Please help, to fix that. Thanks!

--
components: Installation
files: crash_and_packages.log
messages: 348730
nosy: mother-earth40
priority: normal
severity: normal
status: open
title: Crash of python3.7 with virt-manager
type: crash
versions: Python 3.7
Added file: https://bugs.python.org/file48517/crash_and_packages.log

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37720] Crash of python3.7 with virt-manager

[STINNER Victor]

STINNER Victor  added the comment:

Try to run virt-manager with faulthandler enabled to get the Python traceback 
where the bug occurs:

PYTHONDEVMODE=1 virt-manager --no-fork

I expect that the bug comes from a third-party C extension (no part of Python 
standard library). My notes to debug a Python crash:
https://pythondev.readthedocs.io/debug_tools.html

--
nosy: +vstinner

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37720] Crash of python3.7 with virt-manager

[Hans Peter]

Hans Peter  added the comment:

PYTHONDEVMODE=1 virt-manager --no-fork
Fatal Python error: Segmentation fault

Current thread 0x7f835bafd740 (most recent call first):
  File "", line 219 in _call_with_frames_removed
  File "", line 1043 in create_module
  File "", line 583 in module_from_spec
  File "", line 670 in _load_unlocked
  File "", line 967 in _find_and_load_unlocked
  File "", line 983 in _find_and_load
  File "/usr/lib/python3.7/lzma.py", line 27 in 
  File "", line 219 in _call_with_frames_removed
  File "", line 728 in exec_module
  File "", line 677 in _load_unlocked
  File "", line 967 in _find_and_load_unlocked
  File "", line 983 in _find_and_load
  File "/usr/lib/python3.7/shutil.py", line 29 in 
  File "", line 219 in _call_with_frames_removed
  File "", line 728 in exec_module
  File "", line 677 in _load_unlocked
  File "", line 967 in _find_and_load_unlocked
  File "", line 983 in _find_and_load
  File "/usr/lib/python3.7/tempfile.py", line 44 in 
  File "", line 219 in _call_with_frames_removed
  File "", line 728 in exec_module
  File "", line 677 in _load_unlocked
  File "", line 967 in _find_and_load_unlocked
  File "", line 983 in _find_and_load
  File "/usr/lib/python3/dist-packages/apport/report.py", line 12 in 
  File "", line 219 in _call_with_frames_removed
  File "", line 728 in exec_module
  File "", line 677 in _load_unlocked
  File "", line 967 in _find_and_load_unlocked
  File "", line 983 in _find_and_load
  File "/usr/lib/python3/dist-packages/apport/__init__.py", line 5 in 
  File "", line 219 in _call_with_frames_removed
  File "", line 728 in exec_module
  File "", line 677 in _load_unlocked
  File "", line 967 in _find_and_load_unlocked
  File "", line 983 in _find_and_load
  File "", line 219 in _call_with_frames_removed
  File "", line 953 in _find_and_load_unlocked
  File "", line 983 in _find_and_load
  File "/usr/lib/python3/dist-packages/apport_python_hook.py", line 63 in 
apport_excepthook
Segmentation fault

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37720] Crash of python3.7 with virt-manager

[Hans Peter]

Hans Peter  added the comment:

How to fix that? See above post.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37631] EXTRA_CFLAGS get overrided by CFLAGS_NODIST

[STINNER Victor]

STINNER Victor  added the comment:

Python has way too many variables to control compiler flags:
---
# Compiler options
OPT=@OPT@
BASECFLAGS= @BASECFLAGS@
BASECPPFLAGS=   @BASECPPFLAGS@
CONFIGURE_CFLAGS=   @CFLAGS@
# CFLAGS_NODIST is used for building the interpreter and stdlib C extensions.
# Use it when a compiler flag should _not_ be part of the distutils CFLAGS
# once Python is installed (Issue #21121).
CONFIGURE_CFLAGS_NODIST=@CFLAGS_NODIST@
# LDFLAGS_NODIST is used in the same manner as CFLAGS_NODIST.
# Use it when a linker flag should _not_ be part of the distutils LDFLAGS
# once Python is installed (bpo-35257)
CONFIGURE_LDFLAGS_NODIST=@LDFLAGS_NODIST@
CONFIGURE_CPPFLAGS= @CPPFLAGS@
CONFIGURE_LDFLAGS=  @LDFLAGS@
# Avoid assigning CFLAGS, LDFLAGS, etc. so users can use them on the
# command line to append to these values without stomping the pre-set
# values.
PY_CFLAGS=  $(BASECFLAGS) $(OPT) $(CONFIGURE_CFLAGS) $(CFLAGS) 
$(EXTRA_CFLAGS)
PY_CFLAGS_NODIST=$(CONFIGURE_CFLAGS_NODIST) $(CFLAGS_NODIST) 
-I$(srcdir)/Include/internal
# Both CPPFLAGS and LDFLAGS need to contain the shell's value for setup.py to
# be able to build extension modules using the directories specified in the
# environment variables
PY_CPPFLAGS=$(BASECPPFLAGS) -I. -I$(srcdir)/Include $(CONFIGURE_CPPFLAGS) 
$(CPPFLAGS)
PY_LDFLAGS= $(CONFIGURE_LDFLAGS) $(LDFLAGS)
PY_LDFLAGS_NODIST=$(CONFIGURE_LDFLAGS_NODIST) $(LDFLAGS_NODIST)
NO_AS_NEEDED=   @NO_AS_NEEDED@
SGI_ABI=@SGI_ABI@
CCSHARED=   @CCSHARED@
# LINKFORSHARED are the flags passed to the $(CC) command that links
# the python executable -- this is only needed for a few systems
LINKFORSHARED=  @LINKFORSHARED@
ARFLAGS=@ARFLAGS@
# Extra C flags added for building the interpreter object files.
CFLAGSFORSHARED=@CFLAGSFORSHARED@
# C flags used for building the interpreter object files
PY_STDMODULE_CFLAGS= $(PY_CFLAGS) $(PY_CFLAGS_NODIST) $(PY_CPPFLAGS) 
$(CFLAGSFORSHARED)
PY_BUILTIN_MODULE_CFLAGS= $(PY_STDMODULE_CFLAGS) -DPy_BUILD_CORE_BUILTIN
PY_CORE_CFLAGS= $(PY_STDMODULE_CFLAGS) -DPy_BUILD_CORE
# Linker flags used for building the interpreter object files
PY_CORE_LDFLAGS=$(PY_LDFLAGS) $(PY_LDFLAGS_NODIST)
# Strict or non-strict aliasing flags used to compile dtoa.c, see above
CFLAGS_ALIASING=@CFLAGS_ALIASING@
---

I'm not sure which variables are "standard" and supposed to be overriden or 
not, because there is no documentation. The priority because these variables is 
not documented at all. For example:

* What the priority between BASECFLAGS, CFLAGS, OPT and EXTRA_FLAGS?
* Which flags are used to build Python core?
* Which flags are used to build modules of the standard library?
* Which flags are used to build third-party modules?

Linker flags are not documented neither which caused a regression for a short 
time (issue ith BLDSHARED if I recall correctly).

Examples of variables related to linker flags:

---
# LDFLAGS_NODIST is used in the same manner as CFLAGS_NODIST.
# Use it when a linker flag should _not_ be part of the distutils LDFLAGS
# once Python is installed (bpo-35257)
CONFIGURE_LDFLAGS_NODIST=@LDFLAGS_NODIST@
CONFIGURE_LDFLAGS=  @LDFLAGS@

PY_LDFLAGS= $(CONFIGURE_LDFLAGS) $(LDFLAGS)
PY_LDFLAGS_NODIST=$(CONFIGURE_LDFLAGS_NODIST) $(LDFLAGS_NODIST)
PY_CORE_LDFLAGS=$(PY_LDFLAGS) $(PY_LDFLAGS_NODIST)

LDSHARED=   @LDSHARED@ $(PY_LDFLAGS)
BLDSHARED=  @BLDSHARED@ $(PY_CORE_LDFLAGS)
---

--
nosy: +vstinner

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37268] Deprecate the parser module

[miss-islington]

miss-islington  added the comment:


New changeset 9211e2fd81fe1db6f73ded70752b144cc9691ab6 by Miss Islington (bot) 
(Pablo Galindo) in branch 'master':
bpo-37268: Add deprecation notice and a DeprecationWarning for the parser 
module (GH-15017)
https://github.com/python/cpython/commit/9211e2fd81fe1db6f73ded70752b144cc9691ab6


--
nosy: +miss-islington

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37720] Crash of python3.7 with virt-manager

[STINNER Victor]

STINNER Victor  added the comment:

That's a crash in the apport project itself which is specific to Ubuntu, and 
unrelated to Python standard library. I close this issue.

Please report issues specific to Ubuntu to the Ubuntu bug tracker.

--
resolution:  -> third party
stage:  -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue15817] Misc/gdbinit: Expose command documentation to gdb help

[Florian Bruhin]

Change by Florian Bruhin :


--
pull_requests: +14782
pull_request: https://github.com/python/cpython/pull/15021

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37268] Deprecate the parser module

[Pablo Galindo Salgado]

Change by Pablo Galindo Salgado :


--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37631] EXTRA_CFLAGS get overrided by CFLAGS_NODIST

[Charalampos Stratakis]

Charalampos Stratakis  added the comment:

I agree that documenting the flags is quite important, I've had a hard time 
trying to figure out how to implement the LDFLAGS_NODIST, and the change still 
broke macos builds (luckily it was fixed swiftly).

Nevertheless, this is still a bug which should be addressed.

On another note, where would be the best place to start documenting those 
flags? Makefile, the python docs, somewhere else?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37709] CSVReader ignores dialect.lineterminator

[Benjamin Schollnick]

Benjamin Schollnick  added the comment:

This is tested under python 3...

filename = "csv_Sample.csv"
from csv import DictReader
datafile = open(filename, 'r')
data = csv.DictReader(datafile, lineterminator = '\x07', delimiter='\x06')
print(next(data))
OrderedDict([('Field1', 'A'), ('Field2', 'B'), ('Field3', 'C'), ('Field4', 
'D'), ('Field5', 'E'), ('Field6', 'F'), ('Field7', 'G'), ('Field8', 'H'), 
('Field9', 'I'), ('Field10\x07', 'J\x07')])
print(ord(data.reader.dialect.lineterminator))

So it's untested under python 2, since I've stopped developing under Py2.  

I noticed the note in the CSV reader documentation, *AFTER* I diagnosed the 
issue with the CSV reader...  Which is why I opened the bug / feature 
enhancement request, since this is an very odd edge case.

I agree 90+% of all CSVs are going to be \n line terminated, but if we offer it 
for writing, we should offer it for reading.

The main emphasis here is this code will not working in the real world, eg.

filename = "csvFile.csv"
from csv import DictReader, DictWriter
import csv
with open(filename, mode='w') as output_file:
outcsv = csv.writer(output_file, delimiter=',', lineterminator=";")
outcsv.writerow(['John Cleese', 'CEO', 'March'])
outcsv.writerow(['Graham Chapman', 'CFO', 'November'])
outcsv.writerow(['Terry Jones', 'Animation', 'March'])
outcsv.writerow(['Eric Idle', 'Laugh Track', 'November'])
outcsv.writerow(['Michael Palin', 'Snake Wrangler', 'March'])

with open(filename, mode='r') as input_file:
csv_reader = csv.reader(input_file, delimiter=',', lineterminator=";")
for row in csv_reader:
print(row)

--
Added file: https://bugs.python.org/file48518/CSV_SAMPLE.CSV

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37544] Test hang causes --enable-optimizations build to hang

[Jarek Zgoda]

Change by Jarek Zgoda :


Added file: https://bugs.python.org/file48519/test.pythoninfo.libressl-2.9.2.txt

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37631] EXTRA_CFLAGS get overrided by CFLAGS_NODIST

[STINNER Victor]

STINNER Victor  added the comment:

IMHO comments in Makefile.pre.in is a good start.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37681] warning: ‘no_sanitize_thread’

[Ronald Oussoren]

Ronald Oussoren  added the comment:

- Which python version do you try to build?
- What platform are you on?
- Which compiler and compiler version do you use?

--
nosy: +ronaldoussoren

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29636] Specifying indent in the json.tool command

[Daniel Himmelstein]

Daniel Himmelstein  added the comment:

Since opening this issue, I've encountered several additional instances where 
indentation control would have been nice. I don't agree that jq is a sufficient 
substitute:

1. jq is generally not pre-installed on systems. For projects where users are 
guaranteed to have Python installed (but may be on any operating system), it is 
most straightforward to be able to just use a python command and not have to 
explain how to install jq on 3 different OSes.

2. jq does a lot more than prettifying JSON. The simple use case of 
reformatting JSON (to/from a file or stdin/stdout) can be challenging.

3. json.tool describes itself as a "simple command line interface to ... 
pretty-print JSON objects". Indentation is an essential aspect of pretty 
printing. Why even have this CLI if we're unwilling to add essential basic 
functionality that is already well supported by the underlying json.dump API?

Python excels at APIs that match user needs. It seems like we're a small change 
away from making json.tool flexible enough to satisfy real-world needs.

So I'm in favor of merging PR 9765 or PR 345. I'm happy to do the work on 
either to get them mergeable based on whatever specification we can agree on 
here.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37704] Remove Tools/scripts/h2py.py

[STINNER Victor]

STINNER Victor  added the comment:

Lib/plat-*/ directories has been removed in Python 3.6 by:

commit 20737cc91e823b6b17fff16574283c034c99d641
Author: Zachary Ware 
Date:   Thu Sep 8 11:38:46 2016 -0700

Issue #28027: Remove Lib/plat-* files

> Python 2 standard library had platform specific modules like CDROM which were 
> generated by Tools/scripts/h2py.py. These modules were removed, but h2py.py 
> is still around.

Example of Python 2.7:

$ head -n1 Lib/plat-linux2/IN.py 
# Generated by h2py from /usr/include/netinet/in.h
...

$ tail -n1 Lib/plat-linux2/regen 
...
h2py -i '(u_long)' /usr/include/sys/types.h /usr/include/netinet/in.h 
/usr/include/dlfcn.h

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37587] JSON loads performance improvement for long strings

[Nick Coghlan]

Nick Coghlan  added the comment:


New changeset 8a758f5b99c5fc3fd32edeac049d7d4a4b7cc163 by Nick Coghlan (Marco 
Paolini) in branch 'master':
bpo-37587: Make json.loads faster for long strings (GH-14752)
https://github.com/python/cpython/commit/8a758f5b99c5fc3fd32edeac049d7d4a4b7cc163


--
nosy: +ncoghlan

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37587] JSON loads performance improvement for long strings

[miss-islington]

Change by miss-islington :


--
pull_requests: +14783
pull_request: https://github.com/python/cpython/pull/15022

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37587] JSON loads performance improvement for long strings

[Nick Coghlan]

Nick Coghlan  added the comment:

I went ahead and merged the minimal PR and flagged it for backporting to 3.8 - 
it's an obviously beneficial change, that clearly does less work on each pass 
through the loop.

Even if you are doing non-strict parsing of a string that consists entirely of 
invalid characters, you'll get the same level of performance that the previous 
code offered for all strict parsing.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37587] JSON loads performance improvement for long strings

[miss-islington]

miss-islington  added the comment:


New changeset 9265a877426af4fa5c44cc8482e0198806889350 by Miss Islington (bot) 
in branch '3.8':
bpo-37587: Make json.loads faster for long strings (GH-14752)
https://github.com/python/cpython/commit/9265a877426af4fa5c44cc8482e0198806889350


--
nosy: +miss-islington

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37246] http.cookiejar.DefaultCookiePolicy should use current timestamp instead of last updated timestamp value for checking expiry

[Ngalim Siregar]

Change by Ngalim Siregar :


--
keywords: +patch
pull_requests: +14784
stage:  -> patch review
pull_request: https://github.com/python/cpython/pull/15023

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37587] JSON loads performance improvement for long strings

[Inada Naoki]

Inada Naoki  added the comment:

Wait... there is no benchmark for the "minimum change".

I tested 4 compilers, and provide much better patch in 
https://bugs.python.org/issue37587#msg348114

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue9216] FIPS support for hashlib

[STINNER Victor]

STINNER Victor  added the comment:

FYI the multiprocessing module uses hmac.new(authkey, message, 'md5').digest(). 
I don't think that using MD5 even for HMAC is allowed in FIPS mode.

--
nosy: +vstinner

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37587] JSON loads performance improvement for long strings

[Steve Dower]

Steve Dower  added the comment:

While you're testing patches, can you try this version too?

Py_UCS4 c = 0, minc = 0x20;
for (next = end; next < len; next++) {
c = PyUnicode_READ(kind, buf, next);
if (c == '"' || c == '\\') {
break;
}
minc = c < minc ? c : minc;
}
if (strict && minc <= 0x1f) {
raise_errmsg("Invalid control character at", pystr, next);
goto bail;
}

When we tried this, the conditional expression became a "cmovl" operator which 
removed 3-4 branches from within the loop entirely, and it was 18% better than 
the baseline (which has now moved...)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37587] JSON loads performance improvement for long strings

[Steve Dower]

Steve Dower  added the comment:

Oh, we also need to capture "next"... but then again, since the success case is 
far more common, I'd be okay with scanning again to find it.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36670] test suite broken due to cpu usage feature on win 10/ german

[Steve Dower]

Change by Steve Dower :


--
stage:  -> needs patch
versions: +Python 3.8, Python 3.9

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37587] JSON loads performance improvement for long strings

[Marco Paolini]

Marco Paolini  added the comment:

@steve.dower yes, that's what made me discard that experiment we did during the 
sprint.

Ok will test your new patch soon

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37587] JSON loads performance improvement for long strings

[Inada Naoki]

Inada Naoki  added the comment:

Since scope of "c" is very wide, and there is even `&c` in the scope, compiler 
stores the `c` to stack every time on:

  c = PyUnicode_READ(kind, buf, next);

That is the bottleneck.  `if (strict && ...)` is not the bottleneck.

My patch used a new variable with tight scope so compiler can bypass store it 
to the stack.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37721] smtpd: staling connect

[cherusk]

New submission from cherusk :

>From PR:

Following script stals/hangs. According to my tracing done, in the smtpd 
component. The concept of the script is baseline of my unit testing for [1], so 
not hypothetical:

import logging
import sys
logging.basicConfig(level=logging.DEBUG, stream=sys.stdout)

from smtpd import DebuggingServer as TestMailServer
import threading
from notifiers import get_notifier
import time
import itertools as it

mail_server = threading.Thread(target=TestMailServer, args=(("localhost", 
2500), None))
mail_server.daemon = True
mail_server.start()

time.sleep(2)


for msg in it.repeat('copy', 100):
print(msg)
get_notifier('email').notify(from_='cherusk@localhost',
 to='root@localhost',
 message=msg,
 subject='TST',
 host='localhost',
 port=2500,
 tls=False,
 ssl=False,
 html=False,
 username="some",
 password='pw'
 )

I evaluated that on several platforms(FED29, UBUNTU-Latest), same symptoms. 
Also, on network stack I saw a clean TCP handshake happening, also cleanly 
ACKed by the smptd. So, presumption is that it's something in the dispatcher.

[1] 
https://github.com/cherusk/tw_timeliness/blob/master/test/time_engine_test.py

--
components: email
messages: 348752
nosy: barry, cherusk, r.david.murray
priority: normal
pull_requests: 14785
severity: normal
status: open
title: smtpd: staling connect
type: behavior
versions: Python 3.7

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37587] JSON loads performance improvement for long strings

[Steve Dower]

Steve Dower  added the comment:

> compiler stores the `c` to stack every time

The disassembly we looked at didn't do this, so it may just be certain 
compilers. Perhaps we can actually use the register keyword to help them out? :)


Here's a slightly altered one that doesn't require rescanning for the sake of 
the error message:

Py_UCS4 c = 0, minc = strict ? 0x20 : 0x00;
for (next = end; next < len; next++) {
c = PyUnicode_READ(kind, buf, next);
if (c == '"' || c == '\\' || c < minc) {
break;
}
}
if (c < minc) {
raise_errmsg("Invalid control character at", pystr, next);
goto bail;
}

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37704] Remove Tools/scripts/h2py.py

[STINNER Victor]

STINNER Victor  added the comment:


New changeset 27eeaf0f2c9bd53a0fbdfdf38ee759e971221f8d by Victor Stinner in 
branch 'master':
bpo-37704: Remove Tools/scripts/h2py.py (GH-15000)
https://github.com/python/cpython/commit/27eeaf0f2c9bd53a0fbdfdf38ee759e971221f8d


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue13032] h2py.py can fail with UnicodeDecodeError

[STINNER Victor]

STINNER Victor  added the comment:

I just removed Tools/scripts/h2py.py in bpo-37704.

--
resolution:  -> wont fix
stage:  -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37704] Remove Tools/scripts/h2py.py

[STINNER Victor]

Change by STINNER Victor :


--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue15459] ctypes Structures with subclassed types in the _fields_ list

[Tommy Sparber]

Tommy Sparber  added the comment:

This bug is still present using Python 3.7.
I'm also not sure how to fix the leakage of the internal bitfield size encoding.

--
nosy: +tsparber
versions: +Python 3.5, Python 3.6, Python 3.7

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37681] warning: ‘no_sanitize_thread’

[hai shi]

hai shi  added the comment:

I want to build the cpython of master.
My platform and compiler info: Linux version 3.10.0-957.1.3.el7.x86_64 
(mockbu...@kbuilder.bsys.centos.org) (gcc version 4.8.5 20150623 (Red Hat 
4.8.5-36) (GCC) ) #1 SMP Thu Nov 29 14:49:43 UTC 2018

Looks not all platform get this info.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37587] JSON loads performance improvement for long strings

[Inada Naoki]

Inada Naoki  added the comment:

I tested before, after, Steve's patch, and my patch with gcc 8.3.0 and PGO+LTO.
https://gist.github.com/methane/f6077bd1b0b04d40a9c790d9ed670a44#file-gcc-8-3-0-pgo-md

Surprisingly, there is no difference.  Even my patch didn't help register 
allocation when PGO is enabled.

I will run same test with other compilers & PGO (enabled|disabled).

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37587] JSON loads performance improvement for long strings

[Inada Naoki]

Inada Naoki  added the comment:

I'm sorry, I was wrong.  PGO did very nice job on all cases.
gcc allocates `c` to register in the hot loop.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34313] Tkinter crashes with Tk-related error on macOS with ActiveTcl 8.6

[Ned Deily]

Ned Deily  added the comment:

@royf, Without more information, we can only speculate what kind of crash you 
are seeing.  But based on past experience, it is almost certainly a crash in Tk 
itself and, if so, you need to work with your supplier of Tk, presumably the 
conda project, to get a working version of Tk 8.6.x. Our most recent experience 
with building Tk on macOS for the python.org installers found that using the 
released Tk 8.6.8 source tarball 
(ftp://ftp.tcl.tk/pub/tcl//tcl8_6/tk8.6.8-src.tar.gz) was the least problematic 
at the moment.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37717] argparse subcommand docs has non-existent parameter "action"

[paul j3]

paul j3  added the comment:

As discussed in https://bugs.python.org/issue23487, `action` works if the class 
is compatible with argparse._SubParsersAction.

these commands all do the same thing:

parser.add_subparsers()   # default
parser.add_subparsers(action='parsers')  
parser.add_subparsers(action=argparse._SubParsersAction) 

Your example using 

parser.add_subparsers(action='store')

raises the error because the argparse._StoreAction class cannot handle the 
`parser_class` parameter that add_subparsers has added to the kwargs.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37587] JSON loads performance improvement for long strings

[Inada Naoki]

Inada Naoki  added the comment:

This issue is very compiler sensitive.
Please don't report performance without compiler version and PGO option.

Now I'm facing strange behavior. pyperf reports slower time (1ms) for PGO 
builds, although disasm looks good.
But it's 2:30am already...  Please wait for a few days.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37707] Skip individual unit tests that are expensive for the PGO task

[Neil Schemenauer]

Neil Schemenauer  added the comment:


New changeset 52a48e62c6a94577152f9301bbe5f3bc806cfcf1 by Neil Schemenauer in 
branch 'master':
bpo-37707: Exclude expensive unit tests from PGO task (GH-15009)
https://github.com/python/cpython/commit/52a48e62c6a94577152f9301bbe5f3bc806cfcf1


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37707] Skip individual unit tests that are expensive for the PGO task

[miss-islington]

Change by miss-islington :


--
pull_requests: +14786
pull_request: https://github.com/python/cpython/pull/15024

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36044] PROFILE_TASK for PGO build is not a good workload

[Neil Schemenauer]

Neil Schemenauer  added the comment:

Closing as I think PR 14702 mostly resolves this.  The gc_collect() issue has 
not been resolved but based on some investigation, I think it is not worth the 
effort to try to reduce the time spend in that function.  Reducing the number 
of unit tests being run had a huge benefit and did not significantly impact the 
optimization.

--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37707] Skip individual unit tests that are expensive for the PGO task

[Neil Schemenauer]

Change by Neil Schemenauer :


--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34313] Tkinter crashes with Tk-related error on macOS with ActiveTcl 8.6

[Terry J. Reedy]

Terry J. Reedy  added the comment:

Is this issue open for some specific action in a repository covered by this 
tracker, or should it be closed?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37681] warning: ‘no_sanitize_thread’

[Ronald Oussoren]

Ronald Oussoren  added the comment:

A GCC export should be able to tell more, but the warning should be harmless.

Objects/obmalloc.c says that the thread sanitizer is available from GCC 4.8, 
but for some reason it isn't on your system. This seems to be a bug in 
obmalloc.c:  The GCC 4.8.5 docs don't mention this attribute [1], while the 5.5 
docs do [2]

[1] 
https://gcc.gnu.org/onlinedocs/gcc-4.8.5/gcc/Function-Attributes.html#Function-Attributes
[2] 
https://gcc.gnu.org/onlinedocs/gcc-5.5.0/gcc/Function-Attributes.html#Function-Attributes

There's a couple of releases between 4.8.5 and 5.5.0, I don't know which one 
introduces this function attribute. Once that version is know it should be 
fairly easy to provide a patch that only uses the attribute for new enough 
compilers.

--
stage:  -> needs patch
type:  -> compile error
versions: +Python 3.8, Python 3.9

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37707] Skip individual unit tests that are expensive for the PGO task

[Neil Schemenauer]

Neil Schemenauer  added the comment:


New changeset 382cb85401bb010ead411c0532499ffe16c3cf27 by Neil Schemenauer 
(Miss Islington (bot)) in branch '3.8':
bpo-37707: Exclude expensive unit tests from PGO task (GH-15009) (#15024)
https://github.com/python/cpython/commit/382cb85401bb010ead411c0532499ffe16c3cf27


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37705] winerror_to_errno implementation

[Eryk Sun]

Eryk Sun  added the comment:

Special casing of Winsock error codes should be handled in winerror_to_errno 
rather than in oserror_parse_args, so it's consistently applied everywhere. And 
it should use the proper range for Winsock, 1-11999 -- not 
1-2147483647. Codes above the Winsock range should map to EINVAL. It's not 
a non-issue since this range has the following blocks allocated:

IPSEC 13000-13999
SXS   14000-14999
EVT   15000-15079
EC15080-15099
MUI   15100-15199
MCA   15200-15249
SYSPART   15250-15299
VORTEX15300-15320
GPIO  15321-15340
RUNLEVEL  15400-15500
COMTASK   15501-15510
APPX  15600-15699
APPMODEL  15700-15720
APPXSTATE 15800-15840
API   15841-15860
STORE 15861-15880

Another thing winerror_to_errno should do is unwrap HRESULT codes for Win32 
errors (0x8007-0x8007). In other words, 0x80070005 is effectively the 
same as ERROR_ACCESS_DENIED (5). It should be mapped to EACCES and raise 
PermissionError.

For example:

int
winerror_to_errno(int winerror)
{
/* Unwrap FACILITY_WIN32 HRESULT errors. */
if ((winerror & 0x) == 0x8007) {
winerror &= 0x;
}

/* Winsock error codes (1-11999) are errno values. */
if (winerror >= 1 && winerror < 12000) {
return winerror;
}

switch(winerror) {

/* ... */

}
}

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37722] imaplib crashes when trying to read a letter from an imap server imaps.почта.рус

[Иван Михайлов]

Change by Иван Михайлов :


--
components: Library (Lib)
files: imaplib.png
nosy: Иван Михайлов
priority: normal
severity: normal
status: open
title: imaplib crashes when trying to read a letter from an imap server 
imaps.почта.рус
type: crash
versions: Python 3.7
Added file: https://bugs.python.org/file48520/imaplib.png

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36084] Threading: add builtin TID attribute to Thread objects

[STINNER Victor]

STINNER Victor  added the comment:


New changeset 84846b0187919551b1b08dca447658b710b1 by Victor Stinner (Jake 
Tesler) in branch 'master':
bpo-36084: Add threading Native ID information to What's New documentation 
(GH-14845)
https://github.com/python/cpython/commit/84846b0187919551b1b08dca447658b710b1


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36084] Threading: add builtin TID attribute to Thread objects

[miss-islington]

Change by miss-islington :


--
pull_requests: +14787
pull_request: https://github.com/python/cpython/pull/15028

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36084] Threading: add builtin TID attribute to Thread objects

[miss-islington]

miss-islington  added the comment:


New changeset 7026737d77836657c3b713000c3154cfdb7451db by Miss Islington (bot) 
in branch '3.8':
bpo-36084: Add threading Native ID information to What's New documentation 
(GH-14845)
https://github.com/python/cpython/commit/7026737d77836657c3b713000c3154cfdb7451db


--
nosy: +miss-islington

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37723] important performance regression on regular expression parsing

[yannvgn]

New submission from yannvgn :

On complex cases, parsing regular expressions takes much, much longer on Python 
>= 3.7

Example (ipython):

In [1]: import re
In [2]: char_list = ''.join([chr(i) for i in range(0x)])
In [3]: long_char_list = char_list * 10
In [4]: pattern = f'[{re.escape(long_char_list)}]'
In [5]: %time compiled = re.compile(pattern)

The test was run on Amazon Linux AMI 2017.03.

On Python 3.6.1, the regexp compiled in 2.6 seconds:
CPU times: user 2.59 s, sys: 30 ms, total: 2.62 s
Wall time: 2.64 s

On Python 3.7.3, the regexp compiled in 15 minutes (~350x increase in this 
case):
CPU times: user 15min 6s, sys: 240 ms, total: 15min 7s
Wall time: 15min 9s

Doing some profiling with cProfile shows that the issue is caused by 
sre_parse._uniq function, which does not exist in Python <= 3.6

The complexity of this function is on average O(N^2) but can be easily reduced 
to O(N).

The issue might not be noticeable with simple regexps, but programs like text 
tokenizers - which use complex regexps - might really be impacted by this 
regression.

--
components: Regular Expressions
messages: 348771
nosy: ezio.melotti, mrabarnett, yannvgn
priority: normal
severity: normal
status: open
title: important performance regression on regular expression parsing
type: performance
versions: Python 3.7, Python 3.8, Python 3.9

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37723] important performance regression on regular expression parsing

[yannvgn]

Change by yannvgn :


--
keywords: +patch
pull_requests: +14788
stage:  -> patch review
pull_request: https://github.com/python/cpython/pull/15030

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37724] [[Errno 17] File exists: ] # Try create directories that are not part of the archive with

[Tal Cohen]

Change by Tal Cohen <3talco...@gmail.com>:


--
components: Library (Lib)
nosy: Tal Cohen, lars.gustaebel
priority: normal
severity: normal
status: open
title: [[Errno 17] File exists: ] # Try create directories that are not part of 
the archive with
versions: Python 2.7

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37724] [[Errno 17] File exists: ] # Try create directories that are not part of the archive with

[Tal Cohen]

New submission from Tal Cohen <3talco...@gmail.com>:

_extract_member, raised an exception in case of more than one process, This 
happens because "not os.path.exists(upperdirs" return True in case of 
multiprocess


CODE:
.
if upperdirs and not os.path.exists(upperdirs):
# Create directories that are not part of the archive with
# default permissions.
os.makedirs(upperdirs)



I solved the problem by creating the root path before like:
try:
os.makedirs(path)
except OSError as e:
if e.errno != os.errno.EEXIST:
raise
pass

there is an option to add :  
if e.errno != os.errno.EEXIST: in your code?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37725] "make clean" should remove PGO task data

[Neil Schemenauer]

New submission from Neil Schemenauer :

I find it annoying and surprising that "make clean" does not remove the PGO 
data.  If you change a source file, running "make clean" and "make" should 
result in a newly built executable, IMHO.  As it is now, you usually get a 
confusing build failure (PGO data is out of date).

The fix is fairly easy.  Make a new target that does what "clean" currently 
does.  Have the PGO build call that when it needs to preserve the PGO data.  
Introduce a new "clean" target that does what the old clean did and also 
removes the PGO data.

Changing the build system is fraught with danger but I think this is a fairly 
safe change.  The current behavior is quite annoying, IMHO.

--
components: Build
messages: 348773
nosy: nascheme
priority: normal
severity: normal
stage: patch review
status: open
title: "make clean" should remove PGO task data
type: enhancement

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37725] "make clean" should remove PGO task data

[Neil Schemenauer]

Change by Neil Schemenauer :


--
keywords: +patch
pull_requests: +14790
pull_request: https://github.com/python/cpython/pull/15033

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37726] Tutorial should not recommend getopt

[Guido van Rossum]

New submission from Guido van Rossum :

I read on python-ideas that the tutorial recommends getopt as the simple 
argument parsing choice, and argparse as advanced. This is silly. We should 
only promote argparse (unless you want to use raw sys.argv[1:]).

--
assignee: docs@python
components: Documentation
keywords: newcomer friendly
messages: 348774
nosy: docs@python, gvanrossum
priority: normal
severity: normal
stage: needs patch
status: open
title: Tutorial should not recommend getopt
versions: Python 2.7, Python 3.7, Python 3.8, Python 3.9

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37727] error past 15 places

[Whitequill Riclo]

New submission from Whitequill Riclo :

I have found that when doing the calculation when running:

pow(1 + 1/100, 100) = 2.716110034087023

pow(1 + 1/1000, 1000) = 3.03503520654962

This is incorrect as the following is a well known way to calculate Euler's 
number.
I do not know what component is the cause so I just picked one.

--
components: Library (Lib)
messages: 348775
nosy: Whitequill Riclo
priority: normal
severity: normal
status: open
title: error past 15 places
type: behavior
versions: Python 3.6

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37727] error past 15 places

[Whitequill Riclo]

Whitequill Riclo  added the comment:

This error happens in Google's search too which means that Google is using 
Python server-side and some how this hasn't been addressed.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34162] idlelib/NEWS.txt for 3.8.0 (and backports)

[Terry J. Reedy]

Change by Terry J. Reedy :


--
pull_requests: +14791
pull_request: https://github.com/python/cpython/pull/15034

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37681] warning: ‘no_sanitize_thread’

[hai shi]

hai shi  added the comment:

thanks, Ronald. Let me check the process by the upstairs refs.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33822] IDLE subsection of What's New 3.8

[Terry J. Reedy]

Change by Terry J. Reedy :


--
keywords: +patch
pull_requests: +14792
stage:  -> patch review
pull_request: https://github.com/python/cpython/pull/15035

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33821] IDLE subsection of What's New 3.7

[Terry J. Reedy]

Change by Terry J. Reedy :


--
pull_requests: +14793
pull_request: https://github.com/python/cpython/pull/15036

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37709] CSVReader ignores dialect.lineterminator

[Skip Montanaro]

Skip Montanaro  added the comment:

Looking at your sample file, it seems stranger than you first indicated. Your 
line terminator actually appears to be '\x07\r\n', not just '\x07'.  Opening 
your file in text mode will leave you with '\x07' as the last character of the 
last cell in each row. I've attached two files, bell.csv, which has just '\x07' 
as the line terminator, and lfmapper.py, which provides a class (suboptimally 
named LFMapper) which takes a file object opened in binary mode and optional 
line_terminator and encoding args, and performs the necessary slicing of the 
input bytes, decoding them and returning strings.

Unless Python grows a way for you to tell the open() function what string to 
use as the line terminator in text mode, I don't think your example is ever 
going to work without some sort of shim class.

--
Added file: https://bugs.python.org/file48521/bell.csv

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37709] CSVReader ignores dialect.lineterminator

[Skip Montanaro]

Change by Skip Montanaro :


Added file: https://bugs.python.org/file48522/lfmapper.py

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34162] idlelib/NEWS.txt for 3.8.0 (and backports)

[miss-islington]

Change by miss-islington :


--
pull_requests: +14795
pull_request: https://github.com/python/cpython/pull/15038

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34162] idlelib/NEWS.txt for 3.8.0 (and backports)

[miss-islington]

Change by miss-islington :


--
pull_requests: +14794
pull_request: https://github.com/python/cpython/pull/15037

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34162] idlelib/NEWS.txt for 3.8.0 (and backports)

[Terry J. Reedy]

Terry J. Reedy  added the comment:


New changeset fff5cb21ae270d8572741e18030765580c7ae361 by Terry Jan Reedy in 
branch 'master':
bpo-34162: Add missing items to idlelib/NEWS.txt. (GH-15034)
https://github.com/python/cpython/commit/fff5cb21ae270d8572741e18030765580c7ae361


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33822] IDLE subsection of What's New 3.8

[Terry J. Reedy]

Terry J. Reedy  added the comment:


New changeset a72ca90eb9f13ee2abc7e19b669974d2d0b3d63e by Terry Jan Reedy in 
branch 'master':
bpo-33822: Add IDLE section of What's New 3.8 (#15035)
https://github.com/python/cpython/commit/a72ca90eb9f13ee2abc7e19b669974d2d0b3d63e


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33821] IDLE subsection of What's New 3.7

[Terry J. Reedy]

Terry J. Reedy  added the comment:


New changeset 5982b7201b84bfd24a1c2b2836401afee1cad8a7 by Terry Jan Reedy in 
branch 'master':
bpo-33821: Update IDLE section of What's New 3.7 (#15036)
https://github.com/python/cpython/commit/5982b7201b84bfd24a1c2b2836401afee1cad8a7


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33822] IDLE subsection of What's New 3.8

[miss-islington]

Change by miss-islington :


--
pull_requests: +14796
pull_request: https://github.com/python/cpython/pull/15039

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33821] IDLE subsection of What's New 3.7

[miss-islington]

Change by miss-islington :


--
pull_requests: +14798
pull_request: https://github.com/python/cpython/pull/15041

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33821] IDLE subsection of What's New 3.7

[miss-islington]

Change by miss-islington :


--
pull_requests: +14797
pull_request: https://github.com/python/cpython/pull/15040

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33822] IDLE subsection of What's New 3.8

[miss-islington]

miss-islington  added the comment:


New changeset c7236b498fa2fda2240308064605a8ab5eb0fdf8 by Miss Islington (bot) 
in branch '3.8':
bpo-33822: Add IDLE section of What's New 3.8 (GH-15035)
https://github.com/python/cpython/commit/c7236b498fa2fda2240308064605a8ab5eb0fdf8


--
nosy: +miss-islington

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34162] idlelib/NEWS.txt for 3.8.0 (and backports)

[miss-islington]

miss-islington  added the comment:


New changeset 34de5dcf218badfece4ef78587733bbb903ab7b9 by Miss Islington (bot) 
in branch '3.8':
bpo-34162: Add missing items to idlelib/NEWS.txt. (GH-15034)
https://github.com/python/cpython/commit/34de5dcf218badfece4ef78587733bbb903ab7b9


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37727] error past 15 places

[Zachary Ware]

Zachary Ware  added the comment:

This is a limitation of binary floating point, please see 
https://docs.python.org/3/tutorial/floatingpoint.html

--
nosy: +zach.ware
resolution:  -> not a bug
stage:  -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34162] idlelib/NEWS.txt for 3.8.0 (and backports)

[miss-islington]

miss-islington  added the comment:


New changeset ac7cb7b9beaf907a4a484991e467913813bb4971 by Miss Islington (bot) 
in branch '3.7':
bpo-34162: Add missing items to idlelib/NEWS.txt. (GH-15034)
https://github.com/python/cpython/commit/ac7cb7b9beaf907a4a484991e467913813bb4971


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33821] IDLE subsection of What's New 3.7

[miss-islington]

miss-islington  added the comment:


New changeset 9e044ddefd5e02cf0ffac52bcbae6a21a677d257 by Miss Islington (bot) 
in branch '3.8':
bpo-33821: Update IDLE section of What's New 3.7 (GH-15036)
https://github.com/python/cpython/commit/9e044ddefd5e02cf0ffac52bcbae6a21a677d257


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33821] IDLE subsection of What's New 3.7

[miss-islington]

miss-islington  added the comment:


New changeset 8c6e35d2d21928d3e611c39399099e69532952a9 by Miss Islington (bot) 
in branch '3.7':
bpo-33821: Update IDLE section of What's New 3.7 (GH-15036)
https://github.com/python/cpython/commit/8c6e35d2d21928d3e611c39399099e69532952a9


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37727] error past 15 places

[Tim Peters]

Tim Peters  added the comment:

You'll see much the same in every programming language that supports your 
computer's floating-point hardware.  Start by reading this gentle introduction:

https://docs.python.org/3/tutorial/floatingpoint.html

This bug tracker isn't a place for tutorials, though, and understanding 
floating-point deeply is a massive undertaking.

--
nosy: +tim.peters

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32178] Some invalid email address groups cause an IndexError instead of a HeaderParseError

[Abhilash Raj]

Change by Abhilash Raj :


--
pull_requests: +14799
stage: resolved -> patch review
pull_request: https://github.com/python/cpython/pull/15044

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37723] important performance regression on regular expression parsing

[Serhiy Storchaka]

Change by Serhiy Storchaka :


--
nosy: +serhiy.storchaka

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com