[issue43805] multiprocessing.Queue hangs when process on other side dies
New submission from Marko :
When child process dies unexpectedly Queue.get waits indefinitely.
Here is example:
import os
import signal
import multiprocessing
def child_func(qa, qb):
input = qa.get()
print('Child received: ', input)
os.kill(os.getpid(), signal.SIGTERM)
qb.put('B')
exit(0)
qa = multiprocessing.Queue()
qb = multiprocessing.Queue()
process = multiprocessing.Process(target=child_func, args=(qa, qb))
process.start()
qa.put('A')
try:
input = qb.get()
print('Parent received: ', input)
except Exception as ex:
print(ex)
process.join()
exit(0)
--
components: Library (Lib)
messages: 390774
nosy: kormang
priority: normal
severity: normal
status: open
title: multiprocessing.Queue hangs when process on other side dies
versions: Python 3.8
___
Python tracker
<https://bugs.python.org/issue43805>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue22393] multiprocessing.Pool shouldn't hang forever if a worker process dies unexpectedly
Marko added the comment: I've created issue43805. I think it would be better to have universal solution. And not specific ones, like in issue9205. Haven't checked the PRs, though. -- nosy: +kormang ___ Python tracker <https://bugs.python.org/issue22393> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue43805] multiprocessing.Queue hangs when process on other side dies
Marko added the comment: Possible duplicate of issue22393 -- ___ Python tracker <https://bugs.python.org/issue43805> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue43806] asyncio.StreamReader hangs when reading from pipe and other process exits unexpectedly
New submission from Marko :
When using asyncio to read from pipe, if process on the other side of pipe
crashes read operation hangs indefinitely.
Example:
import asyncio
import contextlib
import os
import signal
import time
prx, ctx = os.pipe()
read_transport = None
read_stream = None
async def _connect_read(loop):
global read_transport
global read_stream
stream_reader = asyncio.StreamReader()
def protocol_factory():
return asyncio.StreamReaderProtocol(stream_reader)
rpipe = os.fdopen(prx, mode='r')
transport, _ = await loop.connect_read_pipe(protocol_factory, rpipe)
read_transport = transport
read_stream = stream_reader
def close():
read_transport.close()
@contextlib.asynccontextmanager
async def connect():
try:
loop = asyncio.get_event_loop()
await _connect_read(loop)
yield
finally:
close()
cpid = os.fork()
if cpid == 0:
os.kill(os.getpid(), signal.SIGKILL)
os.write(ctx, b'A')
time.sleep(10.0)
else:
async def read_from_child():
async with connect():
input = await read_stream.read(1)
print('Parent received: ', input)
asyncio.run(read_from_child())
--
components: Library (Lib)
messages: 390778
nosy: kormang
priority: normal
severity: normal
status: open
title: asyncio.StreamReader hangs when reading from pipe and other process
exits unexpectedly
versions: Python 3.8
___
Python tracker
<https://bugs.python.org/issue43806>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue43805] multiprocessing.Queue hangs when process on other side dies
Marko added the comment: Somewhat related issue43806 with asyncio.StreamReader -- ___ Python tracker <https://bugs.python.org/issue43805> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue22393] multiprocessing.Pool shouldn't hang forever if a worker process dies unexpectedly
Marko added the comment: Somewhat related issue43806 with asyncio.StreamReader -- ___ Python tracker <https://bugs.python.org/issue22393> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue43806] asyncio.StreamReader hangs when reading from pipe and other process exits unexpectedly
Marko added the comment: @jaswdr Hm, this is was somewhat unexpected for me, since when reading directly from pipe, and EOF is sent. Timeout was somewhat awkward in my case, since I don't know when other process will start sending, and how long it would take. On the other hand, I use asyncio loop, and can do this asynchronously, so I get notified when child process dies, by other means, and close the stream. So there are plenty of possible workarounds, but I'm not sure it is impossible to solve the problem on the library level yet. It would take more digging into implementation from my side, however. -- ___ Python tracker <https://bugs.python.org/issue43806> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue43806] asyncio.StreamReader hangs when reading from pipe and other process exits unexpectedly
Marko added the comment: Thanks, takluyver! You are right. Synchronous code that I was comparing it to had os.close(ctx), but I forgot to add it in the async example, so I thought it was a bug. Closing this issue. -- resolution: -> not a bug stage: -> resolved status: open -> closed ___ Python tracker <https://bugs.python.org/issue43806> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue43805] multiprocessing.Queue hangs when process on other side dies
Marko added the comment: Yes, something like that would indeed be really helpful. How likely is that something like that gets implemented? -- ___ Python tracker <https://bugs.python.org/issue43805> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue44899] tarfile: add support for creating an archive of potentially changing files
Change by Marko Tuononen : -- keywords: +patch pull_requests: +28610 stage: -> patch review pull_request: https://github.com/python/cpython/pull/30402 ___ Python tracker <https://bugs.python.org/issue44899> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue44899] tarfile: add support for creating an archive of potentially changing files
Change by Marko Tuononen : -- pull_requests: +28632 pull_request: https://github.com/python/cpython/pull/30426 ___ Python tracker <https://bugs.python.org/issue44899> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue13430] Add a curry function to the functools module
New submission from Marko Nervo : I think it would be very usefull to add a curry function to the functools module. It should be like this. def curry(func, *args, **kwargs): if (len(args) + len(kwargs)) >= func.__code__.co_argcount: return func(*args, **kwargs) return (lambda *x, **y: curry(func, *(args + x), **dict(kwargs, **y))) This function allows you to create curried functions or methods in two main ways: (1) >>> def adder(x, y, z): ... return (x + y + z) >>> adder = curry(adder) (2) >>> @curry ... def adder(x, y, z): ... return (x + y + z) Curried functions could be used as follow. >>> adder(2, 3, 4) >>> adder(2, 3)(4) >>> adder(2)(3)(4) >>> adder(z = 4)(2, 3) # etc, etc, etc... Best regards, Marco. -- components: Library (Lib) messages: 147882 nosy: collinwinter, eric.araujo, gregory_p, markonervo, rhettinger, serprex priority: normal severity: normal status: open title: Add a curry function to the functools module type: feature request versions: Python 2.7, Python 3.4 ___ Python tracker <http://bugs.python.org/issue13430> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue13430] Add a curry function to the functools module
Marko Nervo added the comment: In [1]: import functools In [2]: def adder(x, y, z): ...: return (x + y + z) ...: In [3]: adder = functools.partial(adder) In [4]: adder(2)(3)(4) --- TypeError Traceback (most recent call last) /home/marko/ in () TypeError: adder() takes exactly 3 arguments (1 given) No, it can't be replaced using functools.partial. -- ___ Python tracker <http://bugs.python.org/issue13430> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue13430] Add a curry function to the functools module
Marko Nervo added the comment: I totally disagree with the syntax for curried function, but I think a curry function is perfect for the functools module and I also think it could be useful at least as functools.partial. In addition, a lot of languages support currying, such as Haskell, Scala, Javascript... However, here an example less confusional >>> adder = curry(lambda (x, y): (x + y)) >>> adder3 = adder(3) >>> adder3(4) 7 >>> adder3(5) 8 Currying let you defining new functions on other functions. This is a common practice in functional programming (point-free style). So, why not? -- ___ Python tracker <http://bugs.python.org/issue13430> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue13430] Add a curry function to the functools module
Marko Nervo added the comment: > But so does functools.partial. So the question is, what use case does it > help that functools.partial doesn't? Sure, it's common `defining new functions on other functions`... more times. Here a stupid example with fold (our reduce). @curry def fold(function, start, sequence): if len(sequence) == 0: return start else: return fold(function, function(start, sequence[0]), sequence[1:]) Now, someone could be define a generic summer function by fold. import operator as op summer = fold(op.add) Now, an other programmer could be use summer for defining listsummer (a function which sum only list), as follow. listsummer = summer([]) In addition, curry is cleaver than functools.partial. summer = functools.partial(fold, op.add) listsummer = functools.partial(summer, []) However, it is an additional feature. Nobody forces you to use it, but if you need it... Yeah, you could rewrite it each time, but why? It is perfect in functools (: -- ___ Python tracker <http://bugs.python.org/issue13430> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue13430] Add a curry function to the functools module
Marko Nervo added the comment: > You will have to try a bit > harder and showcase examples of *useful* code that are made > significantly easier through the use of curry(). Curry is a more advanced functools.partial. So, it could be used *at least* as partial, but it is more powerfull, usable and readable. I think it's a valid reason; if it isn't, I haven't anything else to say. -- ___ Python tracker <http://bugs.python.org/issue13430> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue9750] sqlite3 iterdump fails on column with reserved name
New submission from Marko Kohtala : Sqlite3 fails to dump a database with column names that are keywords. -- components: Extension Modules files: sqlite3bug.py messages: 115420 nosy: Marko.Kohtala priority: normal severity: normal status: open title: sqlite3 iterdump fails on column with reserved name type: behavior versions: Python 2.7, Python 3.1 Added file: http://bugs.python.org/file18720/sqlite3bug.py ___ Python tracker <http://bugs.python.org/issue9750> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue9750] sqlite3 iterdump fails on column with reserved name
Marko Kohtala added the comment: Here is a patch that may resolve the bug. -- keywords: +patch Added file: http://bugs.python.org/file18721/sqlite3bug.patch ___ Python tracker <http://bugs.python.org/issue9750> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue9750] sqlite3 iterdump fails on column with reserved name
Marko Kohtala added the comment: The second patch contains also fixes for some places where the rules described in http://www.sqlite.org/lang_keywords.html are not followed. -- Added file: http://bugs.python.org/file18722/sqlite3ident.patch ___ Python tracker <http://bugs.python.org/issue9750> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue9750] sqlite3 iterdump fails on column with reserved name
Marko Kohtala added the comment: It also fails if table or column names contain double quote. -- Added file: http://bugs.python.org/file18725/sqlite3bug2.py ___ Python tracker <http://bugs.python.org/issue9750> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue9750] sqlite3 iterdump fails on column with reserved name
Changes by Marko Kohtala : Added file: http://bugs.python.org/file18726/sqlite3dump.patch ___ Python tracker <http://bugs.python.org/issue9750> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue9750] sqlite3 iterdump fails on column with reserved name
Changes by Marko Kohtala : Removed file: http://bugs.python.org/file18721/sqlite3bug.patch ___ Python tracker <http://bugs.python.org/issue9750> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue9750] sqlite3 iterdump fails on column with reserved name
Changes by Marko Kohtala : Removed file: http://bugs.python.org/file18722/sqlite3ident.patch ___ Python tracker <http://bugs.python.org/issue9750> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue9750] sqlite3 iterdump fails on column with reserved name
Marko Kohtala added the comment: Thank you for the review. I have very limited time to use on this. So even when I'd like to make everything easy for you, have the time you give to python be as productive as possible, I can not. But I'll respond to your comments on the patch. a) I added the quotes to every identifier based on a comment in sqlite documentation "SQLite adds new keywords from time to time when it takes on new features. So to prevent your code from being broken by future enhancements, you should normally quote any identifier that is an English language word, even if you do not have to." While fixing one place, I fixed it to follow this recommendation in other places as well. b) I added quotes using backslashes because it was consistent. The table name was already quoted like that. I agree it could be clearer. c) I know. I only tried to make minimal changes targeted only to the issues at hand. I did not want to hide the fixes in middlde of changes to style. -- ___ Python tracker <http://bugs.python.org/issue9750> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue12031] subprocess module does not accept file twice
New submission from marko kreen : I want to pass /dev/null as stdin and stderr. This works from python 2.4 .. 3.2a3 It fails in final 3.2 with 'Bad file descriptor': Traceback (most recent call last): File "test.py", line 11, in Popen(['cat', 'file.txt'], stdout = PIPE, stdin = _in, stderr = _err) File "/opt/apps/python320/lib/python3.2/subprocess.py", line 736, in __init__ restore_signals, start_new_session) File "/opt/apps/python320/lib/python3.2/subprocess.py", line 1330, in _execute_child raise child_exception_type(errno_num, err_msg) OSError: [Errno 9] Bad file descriptor -- components: Library (Lib) files: test.py messages: 135530 nosy: zmk priority: normal severity: normal status: open title: subprocess module does not accept file twice type: crash versions: Python 3.2 Added file: http://bugs.python.org/file21935/test.py ___ Python tracker <http://bugs.python.org/issue12031> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue38256] binascii.crc32 is not 64-bit clean
New submission from marko kreen : Following code: import binascii, zlib bigdata=memoryview(bytearray((1<<32) + 100)) print(binascii.crc32(bigdata)) crc = binascii.crc32(bigdata[:1000]) crc = binascii.crc32(bigdata[1000:], crc) print(crc) print(zlib.crc32(bigdata)) crc = zlib.crc32(bigdata[:1000]) crc = zlib.crc32(bigdata[1000:], crc) print(crc) --- Outputs with python3.7 $ python3.7 x.py 2575877834 2838121701 2838121701 2838121701 -- components: Library (Lib) messages: 352992 nosy: zmk priority: normal severity: normal status: open title: binascii.crc32 is not 64-bit clean type: behavior versions: Python 3.7 ___ Python tracker <https://bugs.python.org/issue38256> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue38256] binascii.crc32 is not 64-bit clean
marko kreen added the comment: Looking at the code, the bug is dependant on USE_ZLIB_CRC32 define and it's unfixed in master. Cause is zlib crc32 that takes length as integer, zlibmodule has workaround, binascii has not. -- ___ Python tracker <https://bugs.python.org/issue38256> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue38256] binascii.crc32 is not 64-bit clean
marko kreen added the comment: Found zlibmodule fix: https://bugs.python.org/issue10276 -- versions: +Python 3.8, Python 3.9 ___ Python tracker <https://bugs.python.org/issue38256> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue44899] tarfile: add support for creating an archive of potentially changing files
New submission from Marko Tuononen :
I have a use case where I need to create a tar archive from a collection of
potentially changing files. I need to use system resources sparingly and
because of that it is not possible to first make a copy of the files.
Current state of the tarfile library: Creating a tar archive is interrupted
with an OSError "unexpected end of data" (example below), if any of the files
changes when it is collected. Using the tarfile library in streaming mode does
not work either. You might find this bug report relevant:
https://bugs.python.org/issue26877
File "/usr/lib64/python3.7/tarfile.py", line 1946, in add
self.addfile(tarinfo, f)
File "/usr/lib64/python3.7/tarfile.py", line 1974, in addfile
copyfileobj(fileobj, self.fileobj, tarinfo.size, bufsize=bufsize)
File "/usr/lib64/python3.7/tarfile.py", line 249, in copyfileobj
raise exception("unexpected end of data")
OSError: unexpected end of data
Target state of the tarfile library: Creating a tar archive is not interrupted
even if a file changes while collected. The tarfile library's add() method
would just return an exit value indicating that some files were changed while
being archived. See e.g. how GNU tar handles similar situation:
https://man7.org/linux/man-pages/man1/tar.1.html#RETURN_VALUE
--
components: Library (Lib)
messages: 399443
nosy: marko-tuononen
priority: normal
severity: normal
status: open
title: tarfile: add support for creating an archive of potentially changing
files
type: enhancement
versions: Python 3.7
___
Python tracker
<https://bugs.python.org/issue44899>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue44899] tarfile: add support for creating an archive of potentially changing files
Marko Tuononen added the comment:
Please find attached an example how to reproduce the problem in question.
$ python3 -m unittest tarfile_ut.py
E
==
ERROR: test_stat (tarfile_ut.TestClass)
--
Traceback (most recent call last):
File "/usr/lib64/python3.6/unittest/mock.py", line 1183, in patched
return func(*args, **keywargs)
File "/var/work/mtuonone/tarfile_ut.py", line 39, in test_stat
tar.add(TEMP_FILENAME)
File "/usr/lib64/python3.6/tarfile.py", line 1952, in add
self.addfile(tarinfo, f)
File "/usr/lib64/python3.6/tarfile.py", line 1980, in addfile
copyfileobj(fileobj, self.fileobj, tarinfo.size, bufsize=bufsize)
File "/usr/lib64/python3.6/tarfile.py", line 257, in copyfileobj
raise exception("unexpected end of data")
OSError: unexpected end of data
--
Ran 1 test in 0.006s
FAILED (errors=1)
$
--
Added file: https://bugs.python.org/file50411/tarfile_ut.py
___
Python tracker
<https://bugs.python.org/issue44899>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue44899] tarfile: add support for creating an archive of potentially changing files
Change by Marko Tuononen : Added file: https://bugs.python.org/file50412/tarfile_ut.py ___ Python tracker <https://bugs.python.org/issue44899> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue44899] tarfile: add support for creating an archive of potentially changing files
Change by Marko Tuononen : Removed file: https://bugs.python.org/file50411/tarfile_ut.py ___ Python tracker <https://bugs.python.org/issue44899> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue9750] sqlite3 iterdump fails on column with reserved name
Marko Kohtala added the comment: Here is finally an update to my patch modified according to comments received. It should apply on 2.7 and 3.3 branches. -- Added file: http://bugs.python.org/file24429/sqlite3dump.patch ___ Python tracker <http://bugs.python.org/issue9750> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue9750] sqlite3 iterdump fails on column with reserved name
Changes by Marko Kohtala : Removed file: http://bugs.python.org/file18726/sqlite3dump.patch ___ Python tracker <http://bugs.python.org/issue9750> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue9750] sqlite3 iterdump fails on column with reserved name
Changes by Marko Kohtala : Removed file: http://bugs.python.org/file18725/sqlite3bug2.py ___ Python tracker <http://bugs.python.org/issue9750> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue9750] sqlite3 iterdump fails on column with reserved name
Changes by Marko Kohtala : Removed file: http://bugs.python.org/file18720/sqlite3bug.py ___ Python tracker <http://bugs.python.org/issue9750> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue14452] SysLogHandler sends invalid messages when using unicode
New submission from marko kreen : SysLogHandler converts message to utf8 and adds BOM, supposedly to conform with RFC5424, but the implementation is broken: the RFC specifies that the BOM should prefix only unstructured message part, but current Python implementation puts it in the middle of structured part, thus confusing RFC-compliant receivers. Simplest fix would be to just remove the BOM adding. -- components: Library (Lib) messages: 157135 nosy: zmk priority: normal severity: normal status: open title: SysLogHandler sends invalid messages when using unicode versions: Python 2.6, Python 2.7, Python 3.1, Python 3.2, Python 3.3, Python 3.4 ___ Python tracker <http://bugs.python.org/issue14452> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue14452] SysLogHandler sends invalid messages when using unicode
Changes by marko kreen : -- type: -> behavior ___ Python tracker <http://bugs.python.org/issue14452> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue14452] SysLogHandler sends invalid messages when using unicode
marko kreen added the comment: The 'msg' in SysLogHandler does not correspond to MSG in RFC. Nor to %(message)s in log record. RFC: SYSLOG-MSG = HEADER SP STRUCTURED-DATA [SP MSG] HEADER = PRI VERSION SP TIMESTAMP SP HOSTNAME SP APP-NAME SP PROCID SP MSGID PRI = "<" PRIVAL ">" [...] MSG = MSG-ANY / MSG-UTF8 MSG-ANY = *OCTET ; not starting with BOM MSG-UTF8= BOM UTF-8-STRING BOM = %xEF.BB.BF logging: The SysLogHandler does not provide any fields after PRI. Which is OK, those can be added via format string. We are using following formatter to get RFC-like structure to message: [formatter_syslog] format=%(hostname)s %(service_name)s %(job_name)s %(message)s That's not fully compliant, but that's beside the point. The problem is that SysLogHandler puts BOM before hostname not before %(message)s. In Python 2.6 it put it even before PRI, which was even more annoying. I see 2 solutions to this: 1) Stop putting BOM at all, its not necessary. 2) Put the BOM into %(message)s somehow, thus working with whatever format is given. -- status: pending -> open ___ Python tracker <http://bugs.python.org/issue14452> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue14452] SysLogHandler sends invalid messages when using unicode
marko kreen added the comment: Note additional brokenness in BOM addition: * It does add BOM even when msg is ascii, just because it was in unicode() string. * It does not add BOM to UTF8 string if it is already encoded to str(). So highly doubt anybody actually relies on it. And keeping such addition just increases chance that anybody starts to rely on it, so it would be good to just drop it. For 3.3, I think it would be best to move syslog packet formatting out from emit(), because emit() already contains noticeable amount of unrelated code, so full override it is annoying. I suggested changing BOM addition so that it only adds it to %(message)s, but such change could cause backwards incompatibility. So it does not look like good idea. But dropping BOM even in old Python *does* look like good idea, as anybody using SysLoghandler needs to deal with BOM-less UTF8 anyway, so dropping it will not create backwards incompatibility. -- ___ Python tracker <http://bugs.python.org/issue14452> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue29827] os.path.exists() returns False for certain file name
New submission from Marko Mavrinac: I have two files in two different folders, both on desktop. If I try using os.path.exists() on both of them, it returns True for one file and False for the other file. After renaming file that I got False from, I get returned True and when I rename it back, it returns False again. File name causing the problem is "testni.wav", I assume it's anything starting with "test", but I'm sure you guys will know better. Thank you Detailed description with screenshots can be seen here: http://stackoverflow.com/questions/42834408/os-path-exists-returning-false-for-one-and-true-for-another-file-both-files-e -- components: Windows messages: 289717 nosy: Marko Mavrinac, paul.moore, steve.dower, tim.golden, zach.ware priority: normal severity: normal status: open title: os.path.exists() returns False for certain file name type: behavior versions: Python 2.7 ___ Python tracker <http://bugs.python.org/issue29827> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue29827] os.path.exists() returns False for certain file name
Marko Mavrinac added the comment: I am sorry, I didn't realize \t affected how the path was recognized. -- ___ Python tracker <http://bugs.python.org/issue29827> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue34605] Avoid master/slave terminology
Gabriel Marko added the comment: @vstinner: > For diversity reasons, it would be nice to try to avoid "master" and "slave" > terminology which can be associated to slavery. This is too vague. Define what "diversity reasons" are and elaborate your point. Referring to some online discussions is not a way for making a serious argument. Make a point at least (i.e. define the term, add pro/contra arguments and explain why you've taken your decision). Your political standpoint is your political standpoint and it's not my business. However, making these changes without giving reasons and arguments for them is a problem. Let me ask: Are these "diversity reasons" really reasons? What I've heard seen so far regarding "diversity reasons, ..." had little to nothing to do with rational thinking or argumentation. Is it really necessary to pollute Python code base with SJW ideology/terminology? What comes next? Ad absurdum: If I find anything associated with something unpleasant to me in Python code or something which can be considered as e.g. "cultural appropriation", I'm free to change it for diversity reason? -- nosy: +suic ___ Python tracker <https://bugs.python.org/issue34605> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue34605] Avoid master/slave terminology
Gabriel Marko added the comment: @mcepl: I completely agree with you that we shouldn't waste time with this. I would be better not to dig into the discussion about "master-slave" terminology. IMO we don't even need to go into that as the problem here is more substantial: This case can create a very problematic precedent i.e. _on can modify (even drastically) a well established terminology based on "pseudo-reasons", political opinion or ideology_. IMO this should be stopped and prevented as soon as possible for all sake. On the other hand, I believe the @vstinner is a rational person and he is able to give _rational reasons_ for his decision which other could challenge as "potential offensiveness" of language is not an argument. -- ___ Python tracker <https://bugs.python.org/issue34605> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue34605] Avoid master/slave terminology
Gabriel Marko added the comment: @cheryl.sabella let me challenge some points in your arguments: > Based on that, I don't think it's fair to blame Victor for bringing it up for > discussion. Ok, but where was the discussion? @vstinner didn't even make a point and some of the PRs were merged. Maybe I'm too spoiled by the field where I come from but this can be hardly considered to be "bringing up something for a discussion" when someone doesn't even make a point (like e.g. "I think it should be changed because..."). Ad absurdum I could say: Because of because-it-can-hurt-someones-feelings reasons it would be nice to... > I don't recall that there were arguments a few months ago on the PR to make > the docs gender neutral. Maybe people were against that too as being 'too > politically correct', but they didn't feel the need to talk about. To me, > this issue is similar to that one. _Personally_, I consider that to be the same kind of PC/SJW nonsense and there should've been a similar discussion. However, there's a big difference. Making documentation gender neutral is unnecessary but it doesn't affect any established CS terminology and doesn't introduce artificial terminological inconsistency between related technologies. > But, I think it's mostly because it's what we're used to. Yes, and that's what is established terminology about. > Here's an idea -- find a friend and explain to them that there is a concept > in computer science... When you enter a new field a part of your responsibility is to learn its terminology and not voluntarily change it because it somehow affects you (hurts your feelings, not compatible with your political view point etc.). Imagine doing the same thing in physics, chemistry or mathematics. Would you redefine number 1 for diversity reasons (there are ways for making up diversity reasons even for this*)? The terminology used inside a field is primarily for the people who are inside the field and understand it. My arguments can sound a bit sarcastic as they try to illustrate the absurdity of this whole issue. They are by no means personal. Seeing all the PC/SJW nonsense around me, I'm afraid that this can be the starting of Python becoming PCython (by which I don't mean a combination of Python with Cython :)). * To see how far could this go, look at this video: https://www.youtube.com/watch?v=iKcWu0tsiZM -- nosy: +skrah ___ Python tracker <https://bugs.python.org/issue34605> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue34605] Avoid master/slave terminology
Gabriel Marko added the comment: The discussion under GH PRs is now censored. What will be the next level? -- ___ Python tracker <https://bugs.python.org/issue34605> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue34660] Replace ableist terms and pejoratives in source code.
Gabriel Marko added the comment: Come on guys. Stop this madness. :( -- nosy: +suic ___ Python tracker <https://bugs.python.org/issue34660> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue34694] Dismiss To Avoid Slave/Master wording cause it easier for non English spoken programmers
Gabriel Marko added the comment: @Mariatta: > There will be no further discussion about this. Repeating this over and over again won't solve the (any) issue. This madness reached another level here: https://bugs.python.org/issue34660. That was exactly my point here: https://bugs.python.org/issue34605#msg324825. But let me guess: There will no (further) discussion about that either. I find this behavior from the Python core developers and representatives simply irresponsible. -- nosy: +suic ___ Python tracker <https://bugs.python.org/issue34694> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue34605] Avoid master/slave terminology
Gabriel Marko added the comment:
@serhiy.storchaka: IMO, the problem isn't the master/slave terminology itself
but the way how the changes were introduced (no discussion) and the
justification ("diversity reasons"???).
IMO this is the next level: https://bugs.python.org/issue34660 and I can't
imagine what comes next. I find this nonsensical and I'm very disappointed that
this ideological nonsense is infecting Python.
IMO the core developers should make a clear statement about this (either pro or
contra). Once it's made, I'll have no other choice than respect that stance and
act accordingly. Saying that "there will be not more discussions" or sending
people to twitter like Guido did is not a solution and it's rather damaging the
Python community and its reputation. :(
--
___
Python tracker
<https://bugs.python.org/issue34605>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue34660] Replace ableist terms and pejoratives in source code.
Gabriel Marko added the comment: @terry.reedy: By madness I meant: 1. blank replacement of words without relevant justification. Collecting 5 links and labelling some words as pejorative or ist or do it for “diversity reasons” etc. is no justification. I have no problem with changing wording in documentation but it has to be justified. 2. that IMO this is _de facto_ PC/SJW language mutilation/censorship. I've made my main claim about that here: https://bugs.python.org/issue34605#msg324825 and IMO this is a continuation of the BPO34605 which is not any better or even worse than this one. I also expect more BPOs and PRs like this and IMHO _no more BPOs or PRs like this should be accepted or merged_. If I can advise: There should be a clear statement about how PSF and core developers will handle BPOs and PRs like this or BPO34605 i.e. if you accept/reject them in the future eventually what is going to be the rule of thumb for acceptance. It can bring some clarity into this whole issue/discussion. What I’ve experienced so far is very disappointing. Repeating “there will be no more discussion about this” is not a solution and I consider this to be damaging for Python community’s reputation. -- ___ Python tracker <https://bugs.python.org/issue34660> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue34660] Replace ableist terms and pejoratives in source code.
Gabriel Marko added the comment: @terry.reedy > Gabriel, I believe I addressed most your concerns in my previous post. I don't think so (see below) but we don't have to agree in everything. :) > Are you are suggesting that we judge proposals _by the proposer_, rather than > the substance of the proposal? Definitely not. It really doesn't matter who has made a proposal if _it makes sense_. However, that doesn't matter either when a proposal doesn't make sense or it's ill-advised or not justified. > who made the proposal and _why_ I don't care about who but the _why_ is the matter here as I put it in point 1. of my previous post. IMO one has to be clear and explicit about his/her intentions/justifications i.e. if one does something for clarity than he or she should declare it :) > There seems to be a misperception that we have collectively changed how we > judge doc proposals. Should we 'announce' that we proceed as we have been? When I use your word: PSF and core developers should address the misperception. To be honest with you, IMO the "Python officials" handled these issues very badly and unprofessionally. Let me clarify. I'm not the only one who has perceived this BPO and V. Stinner's master/slave change (or some "gender neutralizations" of the documentation in the past) as PC/SJW ergo politically/ideologically motivated. So, what is perceived to be the main issue is the motivation. The way these were handled brings quite an _ambiguous_ impression and it's not clear if PSF or core developers are willing to proceed in this PC/SJW (ergo political/ideological) direction or what exactly is their stance. I read the BPOs and GH PRs and also some other articles and discussions where this ambiguity created a lot of confusion. There were statements in those articles and discussions like "GvR were asked to decide this question and he agrees with PC/SJW direction..." Therefore, I don't know how to interpret "that w e proceed as we have been" as IMO no clear statement has been made so far. tl;dr To conclude: I think we still aren't at the same page. However, I'm not sure if it makes sense to continue in this debate _at the moment_ at least for me. The amount of absurdity, nonsense* and misconduct, _I've perceived_ while discussing these two BPOs, made me disappointed and discouraged me for any further participation on trying to make Python better at least for now. I want to give it some time and come back to this with a "cool(er) head". To be specific: merging unjustified politically or ideologically motivated changes without discussion, not addressing factual arguments, silencing and censoring discussions**, sending people to Twitter (even if they don't have an account), using Code of Conduct as a tool***, making "feeling-based" arguments aren't characteristics of rational discourse or open community. I can't imagine what comes next but after all these things, I'm (rather) pessimistic. * e.g. "cleaning/censoring" language based on its "potential offensiveness" is a nonsense as any language _is_ potentially offensive. ** "no further discussion is needed" (or even welcomed) without further context or clarification _can be perceived_ as arrogant as saying "Shut up! I know it better!" *** To be clear, I don't mean your warning under issue34694. I can completely agree that I shouldn't/mustn't make sarcastic comments. IMO CoC is a written-down common sense. If it needs to be used as an argument (i.e. a tool) in a discussion, it's a sign of deeper issues or that something went to far. -- ___ Python tracker <https://bugs.python.org/issue34660> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue34694] Dismiss To Avoid Slave/Master wording cause it easier for non English spoken programmers
Change by Gabriel Marko : -- nosy: -suic ___ Python tracker <https://bugs.python.org/issue34694> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue34660] Replace ableist terms and pejoratives in source code.
Change by Gabriel Marko : -- nosy: -suic ___ Python tracker <https://bugs.python.org/issue34660> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue34605] Avoid master/slave terminology
Change by Gabriel Marko : -- nosy: -suic ___ Python tracker <https://bugs.python.org/issue34605> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue34660] Replace ableist terms and pejoratives in source code.
Gabriel Marko added the comment: @terry.reed: I politely ask you: Please use my proper first name if you refer to me and please don't call me an extremist (like here https://bugs.python.org/msg325802). Feel free to criticize my opinion but don't put labels on me. We don't know each other. Labeling people (not actions or ideas) is ad hominem argumentation or can be considered to be a personal attack which as far as I understand isn't complaint with CoC either. And please don't misrepresent what I wrote: > Marko called our actions 'madness' and here called us 'irresponsible'. > (https://bugs.python.org/issue34694#msg325802) I called _the behavior_ irresponsible not the people. Even responsible people can sometimes have irresponsible choices or behavior. > I find this behavior from the Python core developers and representatives > simply rresponsible. (https://bugs.python.org/msg325499) I hope we can end this debate here. -- nosy: +suic ___ Python tracker <https://bugs.python.org/issue34660> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue34694] Dismiss To Avoid Slave/Master wording cause it easier for non English spoken programmers
Gabriel Marko added the comment: @Larry and Terry: I want to stay out of this discussion or participation on Python development for the future as I've expressed it elsewhere (https://bugs.python.org/issue34660#msg325515). However, I want to address the unfair treatment of my person and what I consider to be a violation of CoC in the previous comment (https://bugs.python.org/issue34694#msg325802). > We have been and will be attacked from extremists on both sides if consider > changes on a case-by-case basis, accepting some and rejecting others. > We have been and will be attacked from extremists on both sides for > considering changes on a case-by-case basis, accepting some and rejecting > others. Labeling people as "extremists" without justification is _ad hominem_ and can be considered to be a personal attack. The term "extremism" has strong negative connotations and it's often related to "calling for violent action". The comment doesn't make clear neither 1) who or what group of people is meant to be extremist here nor 2) what was considered to be extremist. In general _ad hominem_ arguments and attacking someones reputation are not part of civil and rational argumentation/discourse. As far as I understand this isn't compliant with the Code of Conduct either. > Marko called our actions 'madness' and here called us 'irresponsible'. I find this unfair and a misrepresentation of what I wrote: > I find this behavior from the Python core developers and representatives > simply irresponsible. (https://bugs.python.org/msg325499) I explicitly referred to _behavior being irresponsible_ not the people. Even responsible people can have sometimes irresponsible decisions or behaviour. I also explained what I'd meant by "madness" (see here: https://bugs.python.org/issue34660#msg325503). And as I've already said elswhere: > I can completely agree that I shouldn't/mustn't make sarcastic comments. > (https://bugs.python.org/issue34660#msg325515) I also agree with Larry that sarcasm isn't the best strategy :) However, silencing discussions like this "There will be no further discussion about this." (https://bugs.python.org/issue34694#msg325437) isn't a good strategy either. I don't want to be involved into further discussion and I politely ask you to not refer to me or labeling me. Thank you, Gabriel -- nosy: +suic ___ Python tracker <https://bugs.python.org/issue34694> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue18271] get_payload method returns bytes which cannot be decoded using the message's charset
New submission from Marko Lalic:
When the message's Content-Transfer-Encoding is set to 8bit, the
get_payload(decode=True) method returns the payload encoded using
raw-unicode-escape. This means that it is impossible to decode the returned
bytes using the content charset obtained by the get_content_charset method.
It seems this should be fixed so that get_payload returns the bytes as found in
the payload when Content-Transfer-Encoding is 8bit, exactly like Python2.7
handles it.
>>> from email import message_from_string
>>> message = message_from_string("""MIME-Version: 1.0
... Content-Type: text/plain; charset=utf-8
... Content-Disposition: inline
... Content-Transfer-Encoding: 8bit
...
... ünicöde data..""")
>>> message.get_content_charset()
'utf-8'
>>> message.get_payload(decode=True)
b'\xfcnic\xf6de data..'
>>> message.get_payload(decode=True).decode(message.get_content_charset())
Traceback (most recent call last):
File "", line 1, in
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xfc in position 0: invalid
start byte
>>> message.get_payload(decode=True).decode('raw-unicode-escape')
'ünicöde data..'
--
components: email
messages: 191526
nosy: barry, mlalic, r.david.murray
priority: normal
severity: normal
status: open
title: get_payload method returns bytes which cannot be decoded using the
message's charset
type: behavior
versions: Python 3.3
___
Python tracker
<http://bugs.python.org/issue18271>
___
___
Python-bugs-list mailing list
Unsubscribe:
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue18271] get_payload method returns bytes which cannot be decoded using the message's charset
Marko Lalic added the comment:
That will work fine as long as the characters are actually latin. We cannot
forget the rest of the unicode character planes. Consider::
>>> message = message_from_string("""MIME-Version: 1.0
... Content-Type: text/plain; charset=utf-8
... Content-Disposition: inline
... Content-Transfer-Encoding: 8bit
...
... 한글ᥡ╥ສए""")
>>> message.get_payload(decode=True).decode('latin1')
'\\ud55c\\uae00\\u1961\\u2565\\u0eaa\\u090f'
>>> message.get_payload(decode=True).decode('raw-unicode-escape')
'한글ᥡ╥ສए'
However, even if latin1 did work, the main point is that a different encoding
than the one the message specifies must be used in order to decode the bytes to
a unicode string.
--
___
Python tracker
<http://bugs.python.org/issue18271>
___
___
Python-bugs-list mailing list
Unsubscribe:
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue18271] get_payload method returns bytes which cannot be decoded using the message's charset
Marko Lalic added the comment: Thank you for your reply. Unfortunately, I have a use case where message_from_bytes has a pretty great disadvantage. I have to parse the received message and then forward it completely unchanged, apart from possibly adding a few new headers. The problem with message_from_bytes is that it changes the Content-Transfer-Encoding header to base64 (and consequently base64 encodes the content). Do you possibly have a suggestion how to currently go about solving this problem? A possible solution I can spot from your answer is to check the Content-Transfer-Encoding before getting the payload and use the version without decode=True when it is 8bit. Maybe there is something more elegant? Thank you in advance. -- ___ Python tracker <http://bugs.python.org/issue18271> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue16446] pdb raises BdbQuit on 'quit' when started with set_trace
Richard Marko added the comment: Would be nice to have this commited as without this change -if self.quitting: -return # None +if not self.botframe: +self.botframe = frame it's not possible to quit Bdb (and the code it's executing) as it just returns and doesn't raise BdbQuit. -- nosy: +rmarko ___ Python tracker <http://bugs.python.org/issue16446> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1074015] current directory in sys.path handles symlinks badly
Marko Schuetz added the comment: I think this behavior is causing a related problem: Assume I have directories currentWorkspace and branchRepository. On branchRepository I have files main.py and module.py. main.py imports module.py. In currentWorkspace main.py links to the repository version, but module.py has a new version in currentWorkspace. Running main.py will not import the module.py from currentWorkspace. I agree that branchRepository belongs on the search path, but currentWorkspace needs to be on the search path also and needs to take priority over branchRepository. The attached file is an edit of the original recreatebug.sh -- nosy: +marko Added file: http://bugs.python.org/file14317/recreatebug.sh ___ Python tracker <http://bugs.python.org/issue1074015> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue8842] sqlite3 library outdated in Windows builds
New submission from Marko Kohtala : The Windows builds seem to come with SQLite library version 3.5.9, as seen from sqlite3.sqlite_version. This is from 2008-May-12. I've been using the sqlite3 module, but keep running into bugs on Windows. Replacing the DLLs\sqlite3.dll with a newer library (sqlite is going at version 3.6.23), seems to fix those problems. One problem was locking failures when performing a lot of changes and committing after each change. This happens within a single script accessing the file, apparently locking himself out. I did not want users needing to patch installed Python, so I got around that by removing the smaller commits and making one huge commit at end. Now I had a problem that ANALYZE does not result in good queries. Performing ANALYZE with newer library speeded queries significantly. I do not know how to get around that. On Linux I see Python 2.6 using sqlite 3.6.x versions, so I'd expect the reason for old library on Windows can not be incompatibility. -- components: Library (Lib), Windows messages: 106647 nosy: Marko.Kohtala priority: normal severity: normal status: open title: sqlite3 library outdated in Windows builds type: behavior versions: Python 2.6, Python 3.1 ___ Python tracker <http://bugs.python.org/issue8842> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
