[issue23458] [2.7] random: make the file descriptor non-inheritable (on POSIX)

[Ned Deily]

Ned Deily added the comment:

FD_CLOEXEC is first support on OS X 10.5.  Here's a patch to skip the test on 
earlier systems: tested on 10.4, 10.5, and 10.10.

--
nosy: +ned.deily
stage:  -> commit review
Added file: http://bugs.python.org/file38516/issue23458_tiger.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23458] [2.7] random: make the file descriptor non-inheritable (on POSIX)

[STINNER Victor]

STINNER Victor added the comment:

Ned Deily added the comment:
> FD_CLOEXEC is first support on OS X 10.5.  Here's a patch to skip the test on 
> earlier systems: tested on 10.4, 10.5, and 10.10.

What do you mean by "first support"? Does it mean that fcntl(fd, F_SETFD, 
FD_CLOEXEC) is simply a no-op?

Does Python 3.4 with the PEP 446 works on OS X 10.4? Can you test the following 
example?

Python 3.4.1 (default, Nov  3 2014, 14:38:10)
[GCC 4.9.1 20140930 (Red Hat 4.9.1-11)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import os
>>> fd=os.open('.', os.O_RDONLY)
>>> os.get_inheritable(fd)
False
>>> os.set_inheritable(fd, True)
>>> os.get_inheritable(fd)
True

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23625] load_module() docs: zipped eggs are not loaded.

[Thomas Guettler]

Thomas Guettler added the comment:

In this case I am wearing newbee user glasses.

And with this glasses on my nose, I don't care for implementation.

I am confused that imp module does not work like "import foo".

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23679] SSL Ciphers RC4

[mogli]

mogli added the comment:

That was fast, great job!

For the record: The SSLv3 issue I also wrote about was a false positive because 
the test only works with Javascript. Python 2.7.9 has SSLv3 disabled by default 
as it should.

urllib2.urlopen("https://sslv3.dshield.org";)  # fails as it should

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23646] PEP 475: handle EINTR in the time module, retry sleep()

[Roundup Robot]

Roundup Robot added the comment:

New changeset 4accc35cbfcf by Victor Stinner in branch 'default':
Revert changeset d927047b1d8eb87738676980a24930d053ba2150
https://hg.python.org/cpython/rev/4accc35cbfcf

--
nosy: +python-dev

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23646] PEP 475: handle EINTR in the time module, retry sleep()

[STINNER Victor]

STINNER Victor added the comment:

I commited sleep_eintr.patch by mistake. After this change, test_socket started 
to fail on Windows. I don't understand why.

http://buildbot.python.org/all/builders/AMD64%20Windows7%20SP1%203.x/builds/5836/steps/test/logs/stdio

==
ERROR: testClose (test.test_socket.TCPCloserTest)
--
Traceback (most recent call last):
  File "C:\buildbot.python.org\3.x.kloth-win64\build\lib\test\test_socket.py", 
line 3723, in testClose
read, write, err = select.select([sd], [], [], 1.0)
TypeError: argument must be an int, or have a fileno() method.

==
FAIL: testRecv (test.test_socket.NonBlockingTCPTests)
--
Traceback (most recent call last):
  File "C:\buildbot.python.org\3.x.kloth-win64\build\lib\test\test_socket.py", 
line 3883, in testRecv
self.fail("Error trying to do non-blocking recv.")
AssertionError: Error trying to do non-blocking recv.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23625] load_module() docs: zipped eggs are not loaded.

[Paul Moore]

Paul Moore added the comment:

Egg files are a format defined by setuptools. If you look in the setuptools 
documentation it notes that egg files are simply zipfiles with a particular 
structure and naming convention. So from a core Python perspective, you can use 
eggs just like any other zipfile. See the zipimport documentation for further 
details if you need them.

To further confuse the issue, setuptools includes some pretty complex 
mechanisms for adding eggs to syst.path so that their contents can be imported 
(there are .pth files and multi-version support, for example). Those mechanisms 
are part of setuptools, not of core Python. From a core Python perspective, the 
path handling is something you do manually.

So while I understand your confusion, I think you should understand that it is 
a confusion over the setuptools implementation, and how it interacts with the 
mechanisms defined by core Python (which are defined in terms of zipfiles). So 
if you feel there are any documentation issues, they should probably be 
addressed by setuptools rather than in the core docs.

On a peripherally related point, your comment "zipped egg files in sys.path are 
not found" doesn't really explain your problem very well. When reporting to 
setuptools, I'd suggest that you need to clarify your problem a bit better - 
what steps did you take, did they match the steps that the setuptools docs tell 
you to use, and how did the failure show itself? But as I say, that's for a 
setuptools bug report.

--
nosy: +paul.moore

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23685] Fix usage of PyMODINIT_FUNC

[STINNER Victor]

New submission from STINNER Victor:

Attached patch fixes the usage of the PyMODINIT_FUNC macro.

My patch is based on Thomas Wouters's patch of the issue #11410.

I don't understand why Modules/pyexpat.c redefined PyMODINIT_FUNC if not 
defined. In which case PyMODINIT_FUNC was not defined?

I'm not sure that PC/config.c should use PyMODINIT_FUNC instead of use 
"PyObject*".

@Steve.Dower: Does my change to PC/config.c look correct or not?

--
components: Build, Windows
files: PyMODINIT_FUNC.patch
keywords: patch
messages: 238272
nosy: haypo, loewis, pitrou, steve.dower, tim.golden, twouters, zach.ware
priority: normal
severity: normal
status: open
title: Fix usage of PyMODINIT_FUNC
versions: Python 3.5
Added file: http://bugs.python.org/file38517/PyMODINIT_FUNC.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue11410] Use GCC visibility attrs in PyAPI_*

[STINNER Victor]

STINNER Victor added the comment:

I extracted the changes on the PyMODINIT_FUNC macro and I opened the issue 
#23685.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue11410] Use GCC visibility attrs in PyAPI_*

[STINNER Victor]

STINNER Victor added the comment:

+#if defined(__GNUC__) && __GNUC__ >= 4
+#   define HAVE_ATTRIBUTE_VISIBILITY
+#endif

Clang now also supports __attribute__((visibility("..."))). I don't know since 
which version.

I'm not sure because I don't see it:
http://clang.llvm.org/docs/AttributeReference.html

I see it there:
http://llvm.org/docs/LangRef.html#visibility-styles

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue11410] Use GCC visibility attrs in PyAPI_*

[STINNER Victor]

STINNER Victor added the comment:

> The getargs.c change *is* necessary, although it doesn't have to be that 
> exact change.

Why not moving these declarations to Include/modsupport.h where 
_PyArg_Parse...() are already used?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23625] load_module() docs: zipped eggs are not loaded.

[Thomas Guettler]

Thomas Guettler added the comment:

The docs should be where new users look.

I don't speak about several hundret words 

Where do you think new users look for documentation if they want a method which 
does find a module?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue18814] Add codecs.convert_surrogateescape to "clean" surrogate escaped strings

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

I uploaded the patch just before your comment Nick.

Here is updated patch. Functions are renamed as Nick suggested, added two more 
functions: decompose_astrals() and compose_surrogate_pairs(). They are mainly 
for example here, they can be committed in other issue.

I hesitate about the rehandle_surrogatepass name. This function handles 
surrogates than can be created not only with the "surrogatepass" handler, but 
also with different ways, e.g. with the "surrogateescape" handler, with chr(), 
handle_astral() or decompose_astrals(). Actually it checks that the string is 
valid Unicode (not containing surrogates) and handle errors if found with 
specified error handler.

May be there is a time for wider discussion on Python-Dev. I especially want to 
hear opinions of Ezio and Martin.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23644] g++ module compile fails with ‘_Atomic’ does not name a type

[STINNER Victor]

STINNER Victor added the comment:

Hum, I'm lost with the problem with C++ :-( What is your use case? Do you want 
to compile CPython with C++? Or compile a third-party extension with C++ and 
this extension includes "Python.h" which includes "pyatomic.h".

For third-party code, pyatomic.h is only needed by PyThreadState_GET() in 
pystate.h. Maybe we should hide completly pyatomic.h. Currently, pyatomic.h is 
not really used if Py_LIMITED_API is defined.

By the way, can you try to compile the extension with Py_LIMITED_API? It should 
just work.


C++ 11 atomic:
---
#include 

int main()
{
std::atomic_uintptr_t current;
current.store(0, std::memory_order_relaxed);
return 0;
}
---


C++ 11 atomic used with C functions:
---
#include 

int main()
{
std::atomic current(0);
std::atomic_store_explicit(¤t, (uintptr_t)0, 
std::memory_order_relaxed);
return 0;
}
---

I didn't find how to use std::atomic_store_explicit with std::atomic_uintptr_t, 
so I used std::atomic instead.

"std::atomic_store_explicit(¤t, 0, std::memory_order_relaxed);" doesn't 
work: 0 must be explicitly cast to uintptr_t :-(

I tried to hack pyatomic.h to use std::atomic template: see attached 
pyatomic_cpp.patch.

But I don't understand: when PyThreadState_GET() is compiled in C++, should it 
use exactly the same functions of CPython (compiled with C)? It should be 
strange to to use a std::atomic C++ template to access an atomic C variable.

--
Added file: http://bugs.python.org/file38518/pyatomic_cpp.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23644] g++ module compile fails with ‘_Atomic’ does not name a type

[STINNER Victor]

STINNER Victor added the comment:

> For third-party code, pyatomic.h is only needed by PyThreadState_GET() in 
> pystate.h. Maybe we should hide completly pyatomic.h. Currently, pyatomic.h 
> is not really used if Py_LIMITED_API is defined.

pystate_cplusplus.patch: disable completly pyatomic.h on C++, and modify 
pystate.h: PyThreadState_GET() is just an alias to PyThreadState_Get() with 
compiled with C++.

IMO pystate_cplusplus.patch makes more sense.

I don't understand the purpose of atomicfix.patch or pyatomic_cpp.patch, 
PyThreadState_GET() will not work with these patches.

--
Added file: http://bugs.python.org/file38519/pystate_cplusplus.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23686] Upgrade copy of OpenSSL bundled with Python

[Alex Gaynor]

New submission from Alex Gaynor:

On Thursday OpenSSL will disclose some security issues and issue new releases: 
https://mta.openssl.org/pipermail/openssl-announce/2015-March/20.html

When that happens, Python's that bundle an OpenSSL should be upgraded.

--
keywords: security_issue
messages: 238280
nosy: alex
priority: normal
severity: normal
status: open
title: Upgrade copy of OpenSSL bundled with Python

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue18814] Add codecs.convert_surrogateescape to "clean" surrogate escaped strings

[Serhiy Storchaka]

Changes by Serhiy Storchaka :


Added file: http://bugs.python.org/file38520/codecs_convert_escapes_2.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue22585] os.urandom() should use getentropy() of OpenBSD 5.6

[Roundup Robot]

Roundup Robot added the comment:

New changeset 371deb4a5fd1 by Ned Deily in branch '3.4':
Issue #22585: make URandomFDTests test case actually run
https://hg.python.org/cpython/rev/371deb4a5fd1

New changeset c89f7c34e356 by Ned Deily in branch 'default':
Issue #22585: null merge
https://hg.python.org/cpython/rev/c89f7c34e356

New changeset 961efafe9497 by Ned Deily in branch '2.7':
Issue #22585, #23115: make URandomFDTests test case actually run
https://hg.python.org/cpython/rev/961efafe9497

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23115] Backport #22585 -- getentropy for urandom to Python 2.7

[Roundup Robot]

Roundup Robot added the comment:

New changeset 961efafe9497 by Ned Deily in branch '2.7':
Issue #22585, #23115: make URandomFDTests test case actually run
https://hg.python.org/cpython/rev/961efafe9497

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23458] [2.7] random: make the file descriptor non-inheritable (on POSIX)

[Ned Deily]

Ned Deily added the comment:

$ sw_vers
ProductName:Mac OS X
ProductVersion: 10.4.11
BuildVersion:   8S165
$ ./python
Python 3.4.3+ (3.4:910a7a540a31, Mar 17 2015, 03:33:01)
[GCC 4.0.1 (Apple Computer, Inc. build 5370)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import os
>>> fd=os.open('.', os.O_RDONLY)
>>> os.get_inheritable(fd)
False
>>> os.set_inheritable(fd, True)
>>> os.get_inheritable(fd)
True

But if I port test_urandom_fd_non_inheritable from 2.7, it fails on 3.4 as well.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23686] Upgrade copy of OpenSSL bundled with Python

[Ned Deily]

Changes by Ned Deily :


--
components: +Build
nosy: +ned.deily, steve.dower, zach.ware
stage:  -> needs patch
versions: +Python 2.7, Python 3.4, Python 3.5

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23686] Update Windows and OS X installer OpenSSL to 1.0.2a

[Ned Deily]

Changes by Ned Deily :


--
title: Upgrade copy of OpenSSL bundled with Python -> Update Windows and OS X 
installer OpenSSL to 1.0.2a

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23593] Update Windows and OS X installer OpenSSL to 1.0.2

[Ned Deily]

Ned Deily added the comment:

Superseded by Issue23686, OpenSSL 1.0.2a

--
resolution:  -> out of date
stage: patch review -> resolved
status: open -> closed
superseder:  -> Update Windows and OS X installer OpenSSL to 1.0.2a

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue18814] Add codecs.convert_surrogateescape to "clean" surrogate escaped strings

[Nick Coghlan]

Nick Coghlan added the comment:

I'd wondered about that with respect to rehandle_surrogatepass.

The current implementation looks like it processes *all* surrogates (even valid 
surrogate pairs), so "handle_surrogates" might be a suitable name.

If the intent is for it to be "handle_lone_surrogates", I'm not sure the 
current implementation achieves that, as a valid surrogate pair will match 
re.compile('[\ud800-\uefff]+').

The rest looks OK to me, including the decompose_astrals() and 
compose_surrogate_pairs() functions. Regardless of any practical utility, the 
latter two seem useful for *educational* purposes when it comes to unicode, by 
making it clear how to switch between the single code point and dual code point 
representations of the astrals.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue18814] Add codecs.convert_surrogateescape to "clean" surrogate escaped strings

[Nick Coghlan]

Nick Coghlan added the comment:

Oh, and yes, I agree a python-dev discussion would be a good idea.

>From my perspective, "rehandle_surrogateescape" is the key function for making 
>it easier to check for malformed input data from operating system interfaces.

The other items I don't personally have a use case for, but they seem 
potentially valuable in make some key Unicode concepts a bit more discoverable.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue1553375] Add traceback.print_full_exception()

[Nick Coghlan]

Nick Coghlan added the comment:

Adding Robert Collins to the nosy list to see if the recent traceback changes 
make it easier to implement this one correctly.

Robert, for context, the general idea here is to be able to stitch the 
traceback for a caught exception together with the stack trace for the current 
frame in order to give a full stack trace for the caught exception, rather than 
just the stack trace up to the frame where it was caught.

--
nosy: +rbcollins

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue22698] Add constants for ioctl request codes

[M. Levinson]

M. Levinson added the comment:

In addition to ioctl constants from Linux, it would also be very helpful
to add the constants from NetBSD et al. that are described in their ioctl(2) 
man page: http://man.netbsd.org/man/ioctl+2+NetBSD-current

Thank you!

--
nosy: +mrl

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23644] g++ module compile fails with ‘_Atomic’ does not name a type

[Joshua J Cogliati]

Joshua J Cogliati added the comment:

>Hum, I'm lost with the problem with C++ :-( What is your use case? Do >you 
>want to compile CPython with C++? Or compile a third-party >extension with C++ 
>and this extension includes "Python.h" which >includes "pyatomic.h".

My use case is I have C++ code that I want to use from a extension.  The 
extension includes Python.h, which includes pyatomic.h.  Thank you for looking 
into this.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23687] Stacktrace identifies wrong line in multiline list comprehension

[Edward]

New submission from Edward:

This code:

z = [
["Y" for y in None
] for x in range(4)
]

produces this stacktrace in both Python 2.7 and 3.4:

Traceback (most recent call last):
  File "/Users/edwsmith/dev/untitled4/test.py", line 7, in 
] for x in range(4)
  File "/Users/edwsmith/dev/untitled4/test.py", line 7, in 
] for x in range(4)
TypeError: 'NoneType' object is not iterable

Of course my code was slightly more complex, but I lost a fair amount of time 
troubleshooting how the 'for x in range(4)' was evaluating to None, when 
really, it was the inner comprehension that was failing.

Ideally the stack trace would say:

Traceback (most recent call last):
  File "/Users/edwsmith/dev/untitled4/test.py", line 6, in 
["Y" for y in None
  File "/Users/edwsmith/dev/untitled4/test.py", line 6, in 
["Y" for y in None
TypeError: 'NoneType' object is not iterable

--
components: Interpreter Core
messages: 238290
nosy: ers81239
priority: normal
severity: normal
status: open
title: Stacktrace identifies wrong line in multiline list comprehension
type: behavior
versions: Python 2.7, Python 3.4

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue2292] Missing *-unpacking generalizations

[Brett Cannon]

Changes by Brett Cannon :


--
priority: normal -> release blocker

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23458] [2.7] random: make the file descriptor non-inheritable (on POSIX)

[STINNER Victor]

STINNER Victor added the comment:

Ok, so Python 3.4 works as expected on Mac OS X 10.4. It's probably because 
Python 3.4 uses ioctl() if available. Maybe ioctl() works, but not fcntl().

os.urandom() only *tries* to make the file descriptor non-inheritable. It would 
be possible to backport some features of the PEP 446 but the code would be much 
more complex.

Please apply issue23458_tiger.patch, it's ok to skip the test.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23644] g++ module compile fails with ‘_Atomic’ does not name a type

[STINNER Victor]

STINNER Victor added the comment:

@Joshua: Can you please try to compile your extension with Py_LIMITED_API 
defined? Ex: "#define Py_LIMITED_API 0x0303" at the top of your C file, or 
g++ -DPy_LIMITED_API=0x0303.

And can you also please try to patch Python with pystate_cplusplus.patch?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23688] unnecessary copying of memoryview in gzip.GzipFile.write ?

[Wolfgang Maier]

New submission from Wolfgang Maier:

I thought I'd go back to work on a test patch for issue21560 today, but now I'm 
puzzled by the explicit handling of memoryviews in gzip.GzipFile.write.
The method is defined as:

def write(self,data):
self._check_closed()
if self.mode != WRITE:
import errno
raise OSError(errno.EBADF, "write() on read-only GzipFile object")

if self.fileobj is None:
raise ValueError("write() on closed GzipFile object")

# Convert data type if called by io.BufferedWriter.
if isinstance(data, memoryview):
data = data.tobytes()

if len(data) > 0:
self.size = self.size + len(data)
self.crc = zlib.crc32(data, self.crc) & 0x
self.fileobj.write( self.compress.compress(data) )
self.offset += len(data)

return len(data)

So for some reason, when it gets passed data as a meoryview it will first copy 
its content to a bytes object and I do not understand why.
zlib.crc32 and zlib.compress seem to be able to deal with memoryviews so the 
only sepcial casing that seems required here is in determining the byte length 
of the data, which I guess needs to use memoryview.nbytes. I've prepared a 
patch (overlapping the one for issue21560) that avoids copying the data and 
seems to work fine.

Did I miss something about the importance of the tobytes conversion ?

--
components: Library (Lib)
messages: 238294
nosy: wolma
priority: normal
severity: normal
status: open
title: unnecessary copying of memoryview in gzip.GzipFile.write ?
type: resource usage
versions: Python 3.5

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23688] unnecessary copying of memoryview in gzip.GzipFile.write ?

[Wolfgang Maier]

Changes by Wolfgang Maier :


--
keywords: +patch
Added file: http://bugs.python.org/file38521/memoryview_write.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23684] urlparse() documentation does not account for default scheme

[Demian Brecht]

Demian Brecht added the comment:

+1 to the change, but I'd propose maybe "value of *scheme*" given the tables 
are inline with both urlsplit and urlparse docs?

--
nosy: +demian.brecht

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23688] unnecessary copying of memoryview in gzip.GzipFile.write ?

[STINNER Victor]

STINNER Victor added the comment:

The patch looks good to be me, but it lacks an unit test. Can you please add a 
simple unit test to ensure that it's possible to memoryview to write(), and 
that the result is correct? (ex: uncompress and ensure that you get the same 
content)

--
nosy: +haypo

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23688] unnecessary copying of memoryview in gzip.GzipFile.write ?

[STINNER Victor]

Changes by STINNER Victor :


--
nosy: +serhiy.storchaka
type: resource usage -> performance

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue22931] cookies with square brackets in value

[Demian Brecht]

Demian Brecht added the comment:

Thanks for the report Mark, updating this patch to be more backwards compatible 
was on my to-do list. I've attached a new patch that simply adds the new 
characters to the legal value set.

It does look like that's the commit that introduced this issue, but the change 
was made for good reason.

--
Added file: http://bugs.python.org/file38522/issue22931_1.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23625] load_module() docs: zipped eggs are not loaded.

[Mark Lawrence]

Mark Lawrence added the comment:

https://pythonhosted.org/setuptools/

--
nosy: +BreamoreBoy

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue12855] linebreak sequences should be better documented

[SMRUTI RANJAN SAHOO]

SMRUTI RANJAN SAHOO added the comment:

i think in this, "line \fone\nline two\n" ,the space after line taking some 
garbage value or you can say hex value of "\".  so that's why that is showing 
some hex value. if you write "\n " instead of"\" then you can't find that hex 
value. i attached my idle image here.

--
nosy: +SMRUTI RANJAN SAHOO
Added file: http://bugs.python.org/file38523/python.JPG

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue22931] cookies with square brackets in value

[Demian Brecht]

Changes by Demian Brecht :


Removed file: http://bugs.python.org/file38522/issue22931_1.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue22931] cookies with square brackets in value

[Demian Brecht]

Changes by Demian Brecht :


Added file: http://bugs.python.org/file38525/issue22931_1.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue2211] Cookie.Morsel interface needs update

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

Here is a patch with extended and unified tests. Also fixed one bug. I have 
left comments about some changes on Rietveld.

--
Added file: http://bugs.python.org/file38524/issue2211_6.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23688] unnecessary copying of memoryview in gzip.GzipFile.write ?

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

Better way is data = data.cast('B').

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23688] unnecessary copying of memoryview in gzip.GzipFile.write ?

[STINNER Victor]

STINNER Victor added the comment:

> Better way is data = data.cast('B').

Why is this cast required? Can you please elaborate? If some memoryview must be 
rejected, again, we need more unit tests.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23685] Fix usage of PyMODINIT_FUNC

[Steve Dower]

Steve Dower added the comment:

I don't think we should be using PyMODINIT_FUNC for builtin modules, since that 
will make the init functions publicly available from python35.dll. That said, I 
do like being able to be consistent here... can we define PyMODINIT_FUNC 
differently when building pythoncore? That at least would keep the definitions 
in the same place if we ever change it (and since there's an open PEP regarding 
extension modules, it doesn't seem impossible).

The change for pyexpat looks right, since that's an external module. I have no 
ideas why PyMODINIT_FUNC wouldn't always be defined there.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23685] Fix usage of PyMODINIT_FUNC

[STINNER Victor]

STINNER Victor added the comment:

"I don't think we should be using PyMODINIT_FUNC for builtin modules, since 
that will make the init functions publicly available from python35.dll."

Do you mean that my change on PC/config.c is wrong?

For example, Modules/arraymodule.c already contains:

"PyMODINIT_FUNC PyInit_array(void)".

Is the PyInit_array symbol exported today or not on Windows?

If you think that the PyInit_array symbol should be private, maybe there is 
already an issue in Modules/arraymodule.c?

"can we define PyMODINIT_FUNC differently when building pythoncore?"

We can add a different macro for builtin modules. Using the issue #11410, we 
may use it to hide the symbols: __attribute__((visibility("hidden"))).

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23688] unnecessary copying of memoryview in gzip.GzipFile.write ?

[Wolfgang Maier]

Wolfgang Maier added the comment:

Here is a patch with memoryview tests.
Are tests and code patches supposed to go in one file or separate ones ?

--
Added file: http://bugs.python.org/file38526/test_memoryview_write.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23688] unnecessary copying of memoryview in gzip.GzipFile.write ?

[Wolfgang Maier]

Wolfgang Maier added the comment:

@Serhiy:
Why would data = data.cast('B') be required ? When would the memoryview not be 
in 'B' format already ?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23685] Fix usage of PyMODINIT_FUNC

[Steve Dower]

Steve Dower added the comment:

Just had a look in Include/pyport.h and we're already defining PyMODINIT_FUNC 
differently for building core, so all your changes should be fine. The 
redefinition you removed from pyexpat.c was clearly never meant to be used for 
builtin modules.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23688] unnecessary copying of memoryview in gzip.GzipFile.write ?

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

memoryview is converted to bytes because len() for memoryview returns a size of 
first dimension (a number of items for one-dimension view), not a number of 
bytes.

>>> m = memoryview(array.array('I', [1, 2, 3]))
>>> len(m)
3
>>> len(m.tobytes())
12
>>> len(m.cast('B'))
12

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23688] unnecessary copying of memoryview in gzip.GzipFile.write ?

[Wolfgang Maier]

Wolfgang Maier added the comment:

> memoryview is converted to bytes because len() for memoryview returns a size 
> of first dimension (a number of items for one-dimension view), not a number 
> of bytes.
> 
 m = memoryview(array.array('I', [1, 2, 3]))
 len(m)
> 3
 len(m.tobytes())
> 12
 len(m.cast('B'))
> 12

Right, I was aware of this. But are you saying that my proposed solution (using 
memoryview.nbytes) is wrong ? If so, then cast is certainly an option and 
should still outperform tobytes.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue18814] Add codecs.convert_surrogateescape to "clean" surrogate escaped strings

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

Note that provided Python implementations are rather a proof of concept. After 
discussion I'll provide more efficient C implementations, that should be 1-2 
orders faster (and infinitely fast for common case of ASCII strings).

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23685] Fix usage of PyMODINIT_FUNC

[STINNER Victor]

STINNER Victor added the comment:

builtin_modules.patch: add _PyBUILTIN_MODINIT_FUNC macro and use it on modules 
which are builtins on POSIX.

I checked: it's not necessary to modify Modules/config.c to make the symbol 
hidden: only the C code in Modules/ need to set the attribute on the function.

So Modules/config.c, Modules/config.c.in, Modules/makesetup, PC/config.c, 
Tools/freeze/makeconfig.py, ... are unchanged by the patch.

--

On Linux with GCC >= 4.0, you can test manually to hide PyInit_xxx symbols 
using:

#define _PyBUILTIN_MODINIT_FUNC __attribute__((visibility("hidden"))) PyObject*

--
Added file: http://bugs.python.org/file38527/builtin_modules.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23685] Fix usage of PyMODINIT_FUNC

[Roundup Robot]

Roundup Robot added the comment:

New changeset 22a0c925a7c2 by Victor Stinner in branch 'default':
Issue #23685: Fix usage of PyMODINIT_FUNC in _json, _scproxy, nis, pyexpat
https://hg.python.org/cpython/rev/22a0c925a7c2

--
nosy: +python-dev

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23685] Fix usage of PyMODINIT_FUNC

[STINNER Victor]

STINNER Victor added the comment:

I commited PyMODINIT_FUNC.patch without the useless change on PC/config.c.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23689] Memory leak in Modules/sre_lib.h

[Evgeny Kapun]

New submission from Evgeny Kapun:

In Modules/sre_lib.h on line 882 [1], a block of memory is allocated. If 
SRE(match) function later terminates abruptly, either because of a signal or 
because subsequent memory allocation fails, this block is never released.

[1] https://hg.python.org/cpython/file/c89f7c34e356/Modules/sre_lib.h#l882

--
components: Regular Expressions
messages: 238313
nosy: abacabadabacaba, ezio.melotti, mrabarnett
priority: normal
severity: normal
status: open
title: Memory leak in Modules/sre_lib.h
type: resource usage
versions: Python 3.4

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23688] unnecessary copying of memoryview in gzip.GzipFile.write ?

[STINNER Victor]

STINNER Victor added the comment:

> Are tests and code patches supposed to go in one file or separate ones ?

It's more convinient to have a single patch with both changes.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue12319] [http.client] HTTPConnection.putrequest not support "chunked" Transfer-Encodings to send data

[Demian Brecht]

Demian Brecht added the comment:

Updated patch changes the following:

+ Removes support for trailers in requests as they're not supported
+ If Transfer-Encoding is explicitly set by the client, it's assumed that the 
caller will handle all encoding (backwards compatibility)
+ Fixed a bug where chunk size was being sent as decimal instead of hex

--
Added file: http://bugs.python.org/file38528/issue12319_2.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue21076] Turn signal.SIG* constants into enums

[Ethan Furman]

Ethan Furman added the comment:

Working on issue23673 I saw this in the new signal.py:

+def _enum_to_int(value):
+"""Convert an IntEnum member to a numeric value.
+If it's not a IntEnum member return the value itself.
+"""
+try:
+return int(value)
+except (ValueError, TypeError):
+return value

The SIG, etc, constants are based on IntEnum, so they are already numeric 
values, and this function is unnecessary.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue21076] Turn signal.SIG* constants into enums

[Ethan Furman]

Ethan Furman added the comment:

Removing the 'enum_to_int' function would also take care of the accepting 
inappropriate types problem.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23685] Fix usage of PyMODINIT_FUNC

[Steve Dower]

Steve Dower added the comment:

Sounds good. Wasn't quite sure if you were after any effect or just consistency 
:)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23690] re functions never release GIL

[Evgeny Kapun]

New submission from Evgeny Kapun:

Looks like function in re module (match, fullmatch and so on) don't release 
GIL, even though these operations can take much time. As a result, other 
threads can't run while a pattern is being matched, and thread switching 
doesn't happen as well.

--
components: Regular Expressions
messages: 238316
nosy: abacabadabacaba, ezio.melotti, mrabarnett
priority: normal
severity: normal
status: open
title: re functions never release GIL
type: resource usage
versions: Python 3.4

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23689] Memory leak in Modules/sre_lib.h

[STINNER Victor]

STINNER Victor added the comment:

There is maybe a bug. Can you show an example of regex and a text where the 
memory leak occurs? You can use the tracemalloc module to check if there is a 
memory leak. Or use sys.getcounts() if you compiled Python in debug mode.

sre_lib.h is very complex, it uses the C instruction "goto" with regex 
bytecodes.

"DO_JUMP(JUMP_REPEAT, jump_repeat, ctx->pattern+ctx->pattern[0]);" calls "goto 
entrace" to execute following bytecodes, but later it comes back after 
DO_JUMP() with the "jump_repeat:" label:

https://hg.python.org/cpython/file/c89f7c34e356/Modules/sre_lib.h#l1180

--
nosy: +haypo

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23690] re functions never release GIL

[STINNER Victor]

STINNER Victor added the comment:

Supporting to release the GIL would require to redesign the _sre module.

For example, the getstring() gets a "view" of a Python string, it doesn't copy 
the string. So we must hold the GIL, otherwise the Python string can be 
modified by other threads. Copying a very long string may be slower than just 
match the pattern :-/

During the pattern matching, other Python functions are called, these functions 
require the GIL to be hold. Example: PyObject_Malloc().

--
nosy: +haypo
type: resource usage -> performance

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue11410] Use GCC visibility attrs in PyAPI_*

[STINNER Victor]

STINNER Victor added the comment:

In the issue #23685, I proposed a patch to add _PyBUILTIN_MODINIT_FUNC for 
builtin modules. It makes possible to hide PyInit_xxx symbols of builtin 
symbols with __attribute__((visibility("hidden"))). It also avoids to export 
these privates symbols on Windows in the Python DLL (and on Linux in the Python 
.so library).

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23691] re.finditer iterator is not reentrant, but doesn't protect against nested calls to __next__

[Evgeny Kapun]

New submission from Evgeny Kapun:

Iterator returned by re.finditer includes a SRE_STATE value, which is not 
designed to be used concurrently. However, it is possible to call __next__ on 
such iterator while another such call is in progress, e.g. from a signal 
handler. This may result in corruption of SRE_STATE structure.

--
components: Regular Expressions
messages: 238323
nosy: abacabadabacaba, ezio.melotti, mrabarnett
priority: normal
severity: normal
status: open
title: re.finditer iterator is not reentrant, but doesn't protect against 
nested calls to __next__
type: crash
versions: Python 3.4

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23689] Memory leak in Modules/sre_lib.h

[Evgeny Kapun]

Evgeny Kapun added the comment:

Memory leak only happens if match operation terminates abruptly, e.g. because 
of SIGINT. In this case, DO_JUMP doesn't come back.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23690] re functions never release GIL

[Evgeny Kapun]

Evgeny Kapun added the comment:

Aren't Python strings immutable?

Also, match functions still permit execution of signal handlers, which can 
execute any Python code.

If GIL is needed during matching, can it be released temporarily to permit 
thread switching?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23689] Memory leak in Modules/sre_lib.h

[Evgeny Kapun]

Evgeny Kapun added the comment:

Tracemalloc code:

import re
import signal
import tracemalloc

class AlarmError(Exception):
pass
def handle_alarm(signal, frame):
raise AlarmError
signal.signal(signal.SIGALRM, handle_alarm)

s1 = tracemalloc.take_snapshot()
for _ in range(20):
try:
signal.alarm(1)
re.match('(?:a|a|(?=b)){1000}', 'a'*999)
raise RuntimeError
except AlarmError:
pass
s2 = tracemalloc.take_snapshot()
res = s2.compare_to(s1, 'lineno')
for e in res[:10]:
print(e)

For me, it shows almost 3 MiB allocated in re.py.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23689] Memory leak in Modules/sre_lib.h

[Serhiy Storchaka]

Changes by Serhiy Storchaka :


--
nosy: +serhiy.storchaka

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23325] Turn SIG_DFL and SIG_IGN into functions

[Ethan Furman]

Ethan Furman added the comment:

A private method is being added to Enum to better support Enum replacement of 
constants, part of which includes changing __reduce_ex__ to return the string 
of the name.

These changes answer points 1 and 4.

Point 2 would be nice, but seems somewhat less important if Enums are being 
used.

Point 3 -- I don't see how 'Signal(xxx, yyy)' is more complicated than 
'Signal(xxx, zzz)'?

--
nosy: +barry, eli.bendersky

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23689] Memory leak in Modules/sre_lib.h

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

May be this patch helps.

--
keywords: +patch
stage:  -> patch review
versions: +Python 2.7, Python 3.5
Added file: http://bugs.python.org/file38529/sre_clean_repeat_data.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue433030] SRE: Atomic Grouping (?>...) is not supported

[Evgeny Kapun]

Changes by Evgeny Kapun :


--
nosy: +abacabadabacaba

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23688] unnecessary copying of memoryview in gzip.GzipFile.write ?

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

You patch is correct Wolfgang, but with cast('B') the patch would be smaller 
(no need to replace len(data) to nbytes).

While we are here, it is possible to add the support of general byte-like 
objects.

if not isinstance(data, bytes):
data = memoryview(data).cast('B')

isinstance() check is just for optimization, it can be omitted if doesn't 
affect a performance.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue9134] sre bug: lastmark_save/restore

[Evgeny Kapun]

Changes by Evgeny Kapun :


--
nosy: +abacabadabacaba

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23692] Undocumented feature prevents re module from finding certain matches

[Evgeny Kapun]

New submission from Evgeny Kapun:

This pattern matches:

re.match('(?:()|(?(1)()|z)){2}(?(2)a|z)', 'a')

But this doesn't:

re.match('(?:()|(?(1)()|z)){0,2}(?(2)a|z)', 'a')

The difference is that {2} is replaced by {0,2}. This shouldn't prevent the 
pattern from matching anywhere where it matched before.

The reason for this misbehavior is a feature which is designed to protect re 
engine from infinite loops, but in fact it sometimes prevents patterns from 
matching where they should. I think that this feature should be at least 
properly documented, by properly I mean that it should be possible to 
reconstruct the exact behavior from documentation, as the implementation is not 
particularly easy to understand.

--
components: Regular Expressions
messages: 238330
nosy: abacabadabacaba, ezio.melotti, mrabarnett
priority: normal
severity: normal
status: open
title: Undocumented feature prevents re module from finding certain matches
type: behavior
versions: Python 3.4

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue2211] Cookie.Morsel interface needs update

[Demian Brecht]

Demian Brecht added the comment:

Thanks for the updates Serhiy. All look good to me.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23644] g++ module compile fails with ‘_Atomic’ does not name a type

[Joshua J Cogliati]

Joshua J Cogliati added the comment:

> @Joshua: Can you please try to compile your extension with  Py_LIMITED_API 
> defined? Ex: "#define Py_LIMITED_API 0x0303" at the top of your C file, 
> or g++ -DPy_LIMITED_API=0x0303.

It fails in that case, because SWIG is using functions that are not part of the 
Py_LIMITED_API.  

> And can you also please try to patch Python with pystate_cplusplus.patch?

With the pystate_cplusplus.patch I was able to compile both min_example.tar.gz 
and my actual extension.  So I with your patch, it does work.  Thank you.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue8027] distutils fail to determine c++ linker with unixcompiler if using ccache

[Joshua J Cogliati]

Joshua J Cogliati added the comment:

This bug is still in Python 3.5.0a2 (but first issue 23644 needs to be fixed 
before g++ can be used at all)

Attached is a patch for Python 3.5.0.

--
versions: +Python 3.5
Added file: http://bugs.python.org/file38530/fix-distutils-350.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23644] g++ module compile fails with ‘_Atomic’ does not name a type

[Joshua J Cogliati]

Joshua J Cogliati added the comment:

Once this is fixed, maybe issue 8027 can be fixed as well in 3.5.0.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue1553375] Add traceback.print_full_exception()

[Robert Collins]

Robert Collins added the comment:

That should be straightforward - its just sequence suffix/prefix overlap 
detection, and FrameSummary (unlike frames) can be compared with ==. So yes, I 
think it makes it easier. It's not on my immediate itch-scratching though, but 
if someone were to poke at it and need any feedback / thoughts I'd be delighted 
to do so.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23631] 3.5 (a2) traceback regression snarls Idle

[Robert Collins]

Robert Collins added the comment:

Closing, though ideally Terry can confirm it is fully fixed for him.

--
resolution:  -> fixed
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue18983] Specify time unit for timeit CLI

[Roundup Robot]

Roundup Robot added the comment:

New changeset 1ebf8d5b7d60 by Robert Collins in branch 'default':
Issue #18983: Allow selection of output units in timeit.
https://hg.python.org/cpython/rev/1ebf8d5b7d60

--
nosy: +python-dev

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue18983] Specify time unit for timeit CLI

[Roundup Robot]

Roundup Robot added the comment:

New changeset ed34dd00405e by Robert Collins in branch 'default':
Fix patch attribution for issue 18983.
https://hg.python.org/cpython/rev/ed34dd00405e

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue18983] Specify time unit for timeit CLI

[Robert Collins]

Changes by Robert Collins :


--
nosy: +rbcollins
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23631] 3.5 (a2) traceback regression snarls Idle

[Berker Peksag]

Changes by Berker Peksag :


--
stage: needs patch -> resolved

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23606] ctypes.util.find_library("c") no longer makes sense

[eryksun]

eryksun added the comment:

Say I need to use ctypes to call _wsopen_s to open a file without write 
sharing. If I read you correctly, you're saying I'll need to know it's exported 
by api-ms-win-crt-stdio-l1-1-0.dll? Does the 'l1-1-0' suffix reflect a version 
number that will be incremented over time? If so, how about adding an API that 
maps 'stdio' to the version that Python is linked against?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue2211] Cookie.Morsel interface needs update

[Demian Brecht]

Changes by Demian Brecht :


--
stage: patch review -> commit review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23183] timeit CLI best of 3: undocumented output format

[Robert Collins]

Changes by Robert Collins :


--
nosy: +rbcollins

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23183] timeit CLI best of 3: undocumented output format

[Robert Collins]

Robert Collins added the comment:

Here is a patch with some prose - feedback appreciated!

--
keywords: +patch
Added file: http://bugs.python.org/file38531/issue-23183-1.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23183] timeit CLI best of 3: undocumented output format

[Robert Collins]

Changes by Robert Collins :


--
stage:  -> patch review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23690] re functions never release GIL

[STINNER Victor]

STINNER Victor added the comment:

> Aren't Python strings immutable?

Yes. But the re module supports more types than just str and bytes. For 
example, bytearray is also accepted:

>>> re.match(b'^abc', b'abc')
<_sre.SRE_Match object; span=(0, 3), match=b'abc'>
>>> re.match(b'^abc', bytearray(b'abc'))
<_sre.SRE_Match object; span=(0, 3), match=b'abc'>

> Also, match functions still permit execution of signal handlers, which can 
> execute any Python code.

Correct, signal handlers are called. If you mutate the string currently used in 
the pattern matching, you can probably crash Python. I hope that nobody does 
such ugly things in Python signal handlers :-)

> If GIL is needed during matching, can it be released temporarily to permit 
> thread switching?

It's possible to modify the _sre module to release the GIL in some cases. It's 
possible to release the GIL for immutables string, and keep the GIL for mutable 
strings. To do this, you have to audit the source code. First, ensure that no 
global variable is used. For example, the "state" must not be shared (it's ok, 
it's allocated on the stack, thread stacks are not shared).

If you start to release the GIL, you have to search for all functions which 
must be called with the GIL hold. For example, memory allocators, but also all 
functions manipulating Python objects. Hint: seach "PyObject*". For example, 
getslice() must be called with the GIL hold.

Since the GIL is a lock, you should benchmark to ensure that sequences of 
acquire/release the GIL doesn't kill performances with a single thread, and 
with multiple threads. Anyway, a benchmark will be needed.

To be clear: I'm *not* interested to optimize the _sre module to release the 
GIL (to support parallel executions).

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23552] Have timeit warn about runs that are not independent of each other

[Robert Collins]

Robert Collins added the comment:

I think for PyPI its actually important here - the JIT'd state of the code is 
essentially global state being mutated - you can't assess how fast the code is 
without first warming up the JIT, and if it warms up half way through your 
fastest run, you're still not actually finding out what you might want to find 
out.

E.g. do you want to know:
 - how fast is this unjitted [e.g. CLI's]
 - how fast will this be once its hot [e.g. services]

Personally, I think as a first approximation, warning about massive variance is 
a good thing. We could add an option to turn it off, and we could also look at 
hooking properly into the jit to allow detection of stable state and benchmark 
only from there on in. But those extra things don't detract from the utility of 
warning about suspect runs.

--
nosy: +rbcollins

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23689] Memory leak in Modules/sre_lib.h

[STINNER Victor]

STINNER Victor added the comment:

Oh cool, you wrote a script to reproduce the issue! And Serhiy wrote a patch, 
great! Great job guys.

sre_clean_repeat_data.patch looks good to me.

@Serhiy: Can you try the example to ensure that it fixes the issue? If yes, go 
ahead!

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23688] unnecessary copying of memoryview in gzip.GzipFile.write ?

[STINNER Victor]

STINNER Victor added the comment:

> While we are here, it is possible to add the support of general byte-like 
> objects.

With and without the patch, write() accepts bytes, bytearray and memoryview. 
Which other byte-like types do you know?

writeframesraw() method of aifc, sunau and wave modules use this pattern:

if not isinstance(data, (bytes, bytearray)):
data = memoryview(data).cast('B')

We can maybe reuse it in gzip module?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue6422] timeit called from within Python should allow autoranging

[Robert Collins]

Changes by Robert Collins :


--
nosy: +rbcollins
stage: needs patch -> patch review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23552] Have timeit warn about runs that are not independent of each other

[Robert Collins]

Robert Collins added the comment:

Reviewed on rietvald.

--
stage:  -> patch review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23644] g++ module compile fails with ‘_Atomic’ does not name a type

[Roundup Robot]

Roundup Robot added the comment:

New changeset cb05b6d7aacd by Victor Stinner in branch 'default':
Issue #23644: Fix issues with C++ when compiling Python extensions
https://hg.python.org/cpython/rev/cb05b6d7aacd

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23644] g++ module compile fails with ‘_Atomic’ does not name a type

[STINNER Victor]

STINNER Victor added the comment:

"With the pystate_cplusplus.patch I was able to compile both min_example.tar.gz 
and my actual extension.  So I with your patch, it does work.  Thank you."

Cool! I applied this simple patch instead of trying to write an ugly glue in 
pyatomic.h between C and C++.

C and C++ simply look to be incompatible for handling atomic types :-/ The good 
news is that we don't them atomic types for Python extensions, only to compile 
Python core.

Thanks for the report. I prefer to detect such annoying issue before a ".0" 
release (3.5.0). FYI it's a regression introduced by the issue #22038 which 
started to use  (but it's now fixed).

--
resolution:  -> fixed
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com