[issue7978] SocketServer doesn't handle syscall interruption
Changes by Kevin Chen : -- nosy: +kchen ___ Python tracker <http://bugs.python.org/issue7978> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue38571] Segfault with StopIteration
New submission from Kevin Chen :
Unfortunately, I don't currently have a simple test case, but after migrating
an app from python 3.6 to python 3.7, python frequently (but not always)
segfaults when running tests for my app in code related to StopIteration
(perhaps somehow related to the implementation of PEP 479?).
I'm running a self-built (from python.org sources) unpatched build of Python
3.7.5 on Ubuntu Xenial.
(The output below is edited to redact some path names.)
Core was generated by `./python'.
Program terminated with signal SIGSEGV, Segmentation fault.
#0 0x0044cc40 in PyException_GetContext (self=self@entry=0x441cbe0)
at Objects/exceptions.c:338
338 Py_XINCREF(context);
(gdb) py-bt
Traceback (most recent call first):
File "lib/requests/models.py", line 750, in generate
for chunk in self.raw.stream(chunk_size, decode_content=True):
File "lib/requests/models.py", line 828, in content
self._content = b''.join(self.iter_content(CONTENT_CHUNK_SIZE)) or b''
File "lib/requests/sessions.py", line 686, in send
r.content
File "lib/requests/sessions.py", line 533, in request
resp = self.send(prep, **send_kwargs)
File "lib/requests/sessions.py", line 546, in get
return self.request('GET', url, **kwargs)
[trace continues with other files specific to my app]
(gdb) bt
#0 0x0044cc40 in PyException_GetContext (
self=self@entry=) at Objects/exceptions.c:338
#1 0x0050b76d in PyErr_SetObject (
exception=, value=StopIteration())
at Python/errors.c:126
#2 0x0050ba17 in PyErr_SetNone (exception=)
at Python/errors.c:163
#3 0x005d254d in gen_send_ex (gen=0x4351b00, arg=,
exc=, closing=) at Objects/genobject.c:241
#4 0x00429406 in _PyEval_EvalFrameDefault (f=,
throwflag=) at Python/ceval.c:2809
#5 0x005d20cb in gen_send_ex (gen=0x457c1b0, arg=,
exc=, closing=) at Objects/genobject.c:221
#6 0x00459679 in list_extend (self=self@entry=0x445a590,
iterable=iterable@entry=)
at Objects/listobject.c:897
#7 0x0045de55 in _PyList_Extend (self=self@entry=0x445a590,
iterable=iterable@entry=)
at Objects/listobject.c:937
#8 0x005b76fc in PySequence_List (
v=v@entry=) at Objects/abstract.c:1905
#9 0x005b867d in PySequence_Fast (v=,
m=m@entry=0x603715 "can only join an iterable") at Objects/abstract.c:1935
#10 0x00438e02 in stringlib_bytes_join (sep=b'',
iterable=) at Objects/stringlib/join.h:22
#11 0x00445b53 in _PyMethodDef_RawFastCallKeywords (
method=0x8b0cc0 , self=self@entry=b'',
args=args@entry=0x4322f20, nargs=nargs@entry=1, kwnames=kwnames@entry=0x0)
at Objects/call.c:648
#12 0x005cfabf in _PyMethodDescr_FastCallKeywords (
descrobj=,
args=args@entry=0x4322f18, nargs=nargs@entry=2, kwnames=kwnames@entry=0x0)
at Objects/descrobject.c:288
#13 0x0042df85 in call_function (kwnames=0x0, oparg=2,
pp_stack=) at Python/ceval.c:4593
#14 _PyEval_EvalFrameDefault (f=, throwflag=)
at Python/ceval.c:3110
#15 0x004211c5 in function_code_fastcall (co=co@entry=0x3597c30,
args=, args@entry=0x3ce1f08, nargs=nargs@entry=1,
globals=globals@entry={'__name__': 'requests.models', '__doc__':
'\nrequests.models\n~~~\n\nThis module contains the primary objects
that power Requests.\n', '__package__': 'requests', '__loader__':
, '__spec__': , origin='lib/requests/models.py', loader_state=None,
submodule_search_locations=None, _set_fileattr=True,
_cached='lib/requests/__pycache__/models.cpython-37.pyc', _initializing=False)
at remote 0x354cd70>, '__file__': 'lib/requests/models.py', '__cached__':
'lib/requests/__pycache__/models.cpython-37.pyc', '__builtins__': {'__name__':
'builtins', '__doc__': "Built-in functions, exceptions, and other
objects.\n\nNoteworthy: N...(truncated)) at Objects/call.c:283
#16 0x00445180 in _PyFunction_FastCallDict (
func=, args=0x3ce1f08, nargs=1, kwargs=0x0)
at Objects/call.c:322
[remainder of trace redacted]
Sometimes the C backtrace is slightly different, but very similar:
Core was generated by `./python'.
Program terminated with signal SIGSEGV, Segmentation fault.
#0 PyErr_SetObject (exception=0x8b7c00 <_PyExc_StopIteration>, value=0x0)
at Python/errors.c:101
101 Py_INCREF(exc_value);
(gdb) bt
#0 PyErr_SetObject (exception=, value=0x0)
at Python/errors.c:101
#1 0x0050ba17 in PyErr_SetNone (exception=)
at Python/errors.c:163
#2 0x005d254d in gen_send_ex (gen=0x3c71b10, arg=,
exc=, closing=) at Objects/genobject.c:241
[r
[issue38571] Segfault with StopIteration
Kevin Chen added the comment: Sorry for the noise. I upgraded my gevent (1.3.5 to 1.4.0) and greenlet (0.4.13 to 0.4.15) libraries, and that appears to have resolved the issue. I assume there was an incompatibility in one or both of the older versions of these libraries with python 3.7, but I had not previously realized that they were causing the issue. -- resolution: -> not a bug stage: -> resolved status: open -> closed ___ Python tracker <https://bugs.python.org/issue38571> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue42912] enum.Flag ~ bitwise negation is very slow
Change by Kevin Chen : -- nosy: aspin2 priority: normal severity: normal status: open title: enum.Flag ~ bitwise negation is very slow versions: Python 3.8 ___ Python tracker <https://bugs.python.org/issue42912> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue42912] enum.Flag ~ bitwise negation is very slow
New submission from Kevin Chen : Ignore this, opened issue by accident -- stage: -> resolved status: open -> closed ___ Python tracker <https://bugs.python.org/issue42912> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue42915] enum.Flag ~ bitwise negation is very slow and can't be defined as a Flag value
New submission from Kevin Chen :
Here's a code sample:
```
import time
from enum import Flag, auto
class MyFlag(Flag):
NONE = 0
FLAG_1 = auto()
FLAG_2 = auto()
FLAG_3 = auto()
FLAG_4 = auto()
FLAG_5 = auto()
FLAG_6 = auto()
#
# NOT_FLAG_1_OR_2 = ~FLAG_1 & ~FLAG_2
def test_flag():
f = MyFlag.NONE
inverted = (
~MyFlag.FLAG_1
& ~MyFlag.FLAG_2
& ~MyFlag.FLAG_3
& ~MyFlag.FLAG_4
& ~MyFlag.FLAG_5
& ~MyFlag.FLAG_6
)
return f & inverted
INVERTED = (
~MyFlag.FLAG_1
& ~MyFlag.FLAG_2
& ~MyFlag.FLAG_3
& ~MyFlag.FLAG_4
& ~MyFlag.FLAG_5
& ~MyFlag.FLAG_6
)
def test_flag_cached():
f = MyFlag.NONE
return f & INVERTED
if __name__ == "__main__":
start_time = time.time()
for _ in range(10_000):
test_flag()
elapsed = time.time() - start_time
print(f"Took normal {elapsed:2f} seconds.")
start_time = time.time()
for _ in range(10_000):
test_flag_cached()
elapsed = time.time() - start_time
print(f"Took cached {elapsed:2f} seconds.")
```
And its outputs:
```
Took normal 1.799731 seconds.
Took cached 0.009488 seconds.
```
Basically, bitwise negation is very very slow. From what I can tell, it seems
that a lot of time is spent here computing powers of two. I've read elsewhere
that flag values are cached, and it looks like negated Flag values can't be
cached? This seems related to the second issue, which is that any negated Flag
value being defined results in `RecursionError: maximum recursion depth
exceeded` as it searches for a good name for Flag.
Obviously, the simple workaround is just to define a constant variable
elsewhere with the negated value, but it isn't very obvious anywhere that this
is necessary, and I wanted to raise this to see if anyone has knowledge of the
implementation details of Flag for possibly resolving this in the class itself.
--
messages: 384983
nosy: aspin2
priority: normal
severity: normal
status: open
title: enum.Flag ~ bitwise negation is very slow and can't be defined as a Flag
value
versions: Python 3.8
___
Python tracker
<https://bugs.python.org/issue42915>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue42915] enum.Flag ~ bitwise negation is very slow and can't be defined as a Flag value
Kevin Chen added the comment: Awesome thanks! Does the rewrite fix the issue with creating negated flags as well? -- ___ Python tracker <https://bugs.python.org/issue42915> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue31062] socket.makefile does not handle line buffering
New submission from Kevin Chen:
File objects generated with socket.makefile and that attempt to use line
buffering appear to not actually use line buffering, at least for writing. In
this example, the string does not appear to be written until the flush call.
First, set up a socket:
$ nc -l -U /tmp/foo
Then:
Python 3.6.2 (default, Jul 26 2017, 01:41:27)
[GCC 4.8.4] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import socket
>>> s = socket.socket(socket.AF_UNIX, socket.SOCK_STREAM)
>>> s.connect("/tmp/foo")
>>> f = s.makefile("rw", buffering=1)
>>> f.write("asdf\n")
5
>>> f.flush()
The following patch appears to fix the problem:
--- socket.py.orig 2017-07-25 21:41:39.974554944 -0400
+++ socket.py 2017-07-27 17:02:58.223353418 -0400
@@ -253,7 +253,11 @@
buffer = io.BufferedWriter(raw, buffering)
if binary:
return buffer
-text = io.TextIOWrapper(buffer, encoding, errors, newline)
+line_buffering = False
+if buffering == 1:
+line_buffering = True
+text = io.TextIOWrapper(buffer, encoding, errors, newline,
+line_buffering)
text.mode = mode
return text
--
components: Library (Lib)
messages: 299351
nosy: kchen
priority: normal
severity: normal
status: open
title: socket.makefile does not handle line buffering
type: behavior
versions: Python 3.6
___
Python tracker
<http://bugs.python.org/issue31062>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue20891] PyGILState_Ensure on non-Python thread causes fatal error
Change by Kevin Chen : -- nosy: +kchen ___ Python tracker <https://bugs.python.org/issue20891> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue6074] .pyc files created readonly if .py file is readonly, python won't overwrite
Kevin Chen added the comment:
I propose a fix:
static FILE *
open_exclusive(char *filename, mode_t mode)
{
#if defined(O_EXCL)&&defined(O_CREAT)&&defined(O_WRONLY)&&defined(O_TRUNC)
/* Use O_EXCL to avoid a race condition when another process tries to
write the same file. When that happens, our open() call fails,
which is just fine (since it's only a cache).
XXX If the file exists and is writable but the directory is not
writable, the file will never be written. Oh well.
*/
int fd;
(void) unlink(filename);
fd = open(filename, O_EXCL|O_CREAT|O_WRONLY|O_TRUNC
#ifdef O_BINARY
|O_BINARY /* necessary for Windows */
#endif
#ifdef __VMS
, mode, "ctxt=bin", "shr=nil"
#elif defined(MS_WINDOWS)
, mode | _S_IWRITE
#else
, mode
#endif
);
if (fd < 0 )
return NULL;
return fdopen(fd, "wb");
#else
/* Best we can do -- on Windows this can't happen anyway */
return fopen(filename, "wb");
#endif
}
--
so doesn't matter what the .py file permission is under windows, the .pyc file
will always have both read and write permissions.
--
nosy: +lowlifer123
___
Python tracker
<http://bugs.python.org/issue6074>
___
___
Python-bugs-list mailing list
Unsubscribe:
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue6074] .pyc files created readonly if .py file is readonly, python won't overwrite
Kevin Chen added the comment: This is essentially the way things were done in version 2.5.4 of Python. The unlink() always succeeded, because the created .pyc file permission is always set to 0666 in the fd=open() function. This means the .pyc will never be created as read-only, and as long as they are never set to read-only manually by the user, everything will be okay. You might say this will be a problem if someone accidentally set the it to read-only. Well we have been using Python 2.5 for many years, and we have managed quite well, so it is not really going to be a big issue, and can be fixed in the future. Whereas at the moment we have a PROBLEM!! With the current state of the Python 2.6, 2.7 and 3.2 interpreters, many Windows users cannot even think about upgrade from Python 2.5 because source control tools like Perforce will set all .py files to read-only, and so ALL created .pyc files will become read-only every time you run Python. I cannot stress how much pain this causes. PLEASE MAKE IT 2.5 EQUIVALENT ASAP! THANK YOU!! even just this will do the trick: - fd = open(filename, O_EXCL|O_CREAT|O_WRONLY|O_TRUNC #ifdef O_BINARY |O_BINARY /* necessary for Windows */ #endif #ifdef __VMS , mode, "ctxt=bin", "shr=nil" #elif defined(MS_WINDOWS) , 0666 #else , mode #endif ); --- And a side note, for the VC8 build for PC, the file random.c was left out from the visual studio project file for pythoncore project. It gives these errors when I try recompile: Error 2 error LNK2019: unresolved external symbol __PyRandom_Init referenced in function _Py_Main main.obj Error 3 error LNK2001: unresolved external symbol __PyRandom_Init pythonrun.obj Error 4 error LNK2019: unresolved external symbol __PyOS_URandom referenced in function _posix_urandom posixmodule.obj This is my first time using bug tracker, so please point me to the appropriate place to put this. Thank you. -- ___ Python tracker <http://bugs.python.org/issue6074> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue6074] .pyc files created readonly if .py file is readonly, python won't overwrite
Changes by Kevin Chen : -- nosy: +ncoghlan ___ Python tracker <http://bugs.python.org/issue6074> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue6074] .pyc files created readonly if .py file is readonly, python won't overwrite
Kevin Chen added the comment: Hi I made the following patch for this issue. It addresses the #2051 security bug. So the .pyc and .pyo files are created using the same permission as .py The MS_WINDOWS version will chmod the permission of .pyc and .pyo files to write accessible before deleting them. I have also created a test for it. This test needs cleaning up, because it is the first time I have written a test for cpython. I would love to have a look at the final committed test so I can learn how to make it better next time. I will also be happy to keep extending that test to make sure it covers more cases. A great big thanks to Nick for your help! -- keywords: +patch versions: +Python 3.1 Added file: http://bugs.python.org/file26912/issue6074.patch ___ Python tracker <http://bugs.python.org/issue6074> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue6074] .pyc files created readonly if .py file is readonly, python won't overwrite
Kevin Chen added the comment: Hi Eric, I am happy to install Python 3.3 on Windows and help you test this. Can you please give me some ideas on what I need to do? Do I sync to the latest Python 3.3 branch and compile? Then run some unit-test code? Which test files should I run? What should I be expecting? Also, do you know if anyone is working on adding the fix and test to Python 2.7? -- ___ Python tracker <http://bugs.python.org/issue6074> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue6074] .pyc files created readonly if .py file is readonly, python won't overwrite
Kevin Chen added the comment: Hi Nick, Thank you so much for rolling out this change. I think your implementation of the fix is more elegant. I have learned a few things from you. I would love to keep helping to test/fix bugs for Python on Windows. Please contact me if there is anything you would like me to take a look at. There will be many Windows users who will feel very grateful for this change you have implemented. This will actually give many the chance to upgrade to the latest 2.7 when it is released. -- ___ Python tracker <http://bugs.python.org/issue6074> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue6074] .pyc files created readonly if .py file is readonly, python won't overwrite
Kevin Chen added the comment: Okay just tried it. This is under Windows 7 x64. I created a .py file with read-only permission. When I import the .py module, the created .pyc doesn't have read-only permission. So it works as intended under Windows OS. As long as under posix system the created .pyc files have the same permissions as the .py file, then everything is good. -- ___ Python tracker <http://bugs.python.org/issue6074> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue15880] os.path.split() and long UNC names
Changes by Kevin Chen : -- nosy: +kevin.chen ___ Python tracker <http://bugs.python.org/issue15880> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue28671] SSL server requesting client certificates should send CA list
New submission from Kevin Chen: When a Python HTTPS server requests client certificates, it should send a CA list so the client knows which certificates are acceptable. It looks like right now Python calls SSL_CTX_load_verify_locations, so once the client certificate is sent, Python can verify whether the client against the specify CAs. However, it looks like Python should also call SSL_CTX_set_client_CA_list so the client knows which certificates to send. -- assignee: christian.heimes components: SSL messages: 280620 nosy: christian.heimes, kchen priority: normal severity: normal status: open title: SSL server requesting client certificates should send CA list type: behavior ___ Python tracker <http://bugs.python.org/issue28671> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
