Charles-François Natali added the comment:
Shall we close this, since realpath(3) is fundamentally broken, and pathlib now
does The Right Thing?
--
___
Python tracker
<http://bugs.python.org/issue12
Charles-François Natali added the comment:
> Why not using os.fstat() instead of os.dup() to check if a file descriptor is
> open or not?
I asked myself the same question, but IIRC, fstat() doesn't always
work on Windows (of course).
--
___
Charles-François Natali added the comment:
> STINNER Victor added the comment:
>
>> I asked myself the same question, but IIRC, fstat() doesn't always
> work on Windows (of course).
>
> Can you please elaborate?
Not really, since I don't know much about Windows,
Charles-François Natali added the comment:
> Ok, let's go with the option (B): use set_wakeup_fd() on all platforms, but
> only accept socket handles on Windows.
Sorry, why restrict it to sockets on Windows?
If someone wants to pass e.g. a pipe, why
Charles-François Natali added the comment:
> Pipes cannot be configured in non-blocking mode on Windows. It sounds
> dangerous to call a blocking syscall in a signal handler.
>
> In fact, it works to write the signal number into a pipe on Windows, but I'm
> worried about th
Charles-François Natali added the comment:
> In the issue #22042, I would like to make automatically the file desscriptor
> or socket handler in non-blocking mode. The problem is that you cannot make a
> file descriptor in non-blocking mode on Windows.
I don't think we sho
Charles-François Natali added the comment:
> It doesn't answer to my complain: I don't want to support file descriptors on
> Windows anymore because file descriptors cannot be configured in non-blocking
> mode.
I think it does : if an exception is raised if an FD/ha
Charles-François Natali added the comment:
Should be fixed now, thanks!
--
resolution: -> fixed
stage: -> resolved
status: open -> closed
___
Python tracker
<http://bugs.python.or
Charles-François Natali added the comment:
Backported to 2.7 (don't know how Iforgot it).
3.3 is only open for security issues, so not backporting.
--
___
Python tracker
<http://bugs.python.org/is
Charles-François Natali added the comment:
I agree with Akira, although it's probably too late now to rename.
--
___
Python tracker
<http://bugs.python.org/is
Charles-François Natali added the comment:
> Richard Oudkerk added the comment:
>
> I can't remember why I did not use fstat() -- probably it did not occur to me.
I probably have Alzeihmer, I was sure I heard a reasonable case for
dup() vs fstat().
The only thing I can think of
New submission from Charles-François Natali:
The patch attached enables -Wsign-compare and -Wunreachable-code if supported
by the compiler.
AFAICT, mixed sign comparison warning is automatically enabled by Microsoft's
compiler, and is usually a good thing.
It does add some warnings though
Charles-François Natali added the comment:
> Antoine Pitrou added the comment:
>
> Enabling the warnings may be a good incitation for other people to fix them ;)
That was my intention...
Can I push it, and let warnings be fixed on a case-by-c
Charles-François Natali added the comment:
Closing, since it's likely a kernel bug.
--
resolution: -> third party
stage: -> resolved
status: open -> closed
___
Python tracker
<http://bugs.pyth
Charles-François Natali added the comment:
Closing, I haven't seen this in a while.
--
resolution: -> out of date
stage: needs patch -> resolved
status: open -> closed
___
Python tracker
<http://bugs.pyth
Charles-François Natali added the comment:
Thanks for the reminder Mark.
Yes, it is probably still an issue with the latest 2.7 release.
There were actually two issues:
- send send()/sendall() call didn't block because the test doesn't write enough
data: we have since added a SOC
Charles-François Natali added the comment:
Committed.
Sorry for the extra ~70 warnings :-)
--
resolution: -> fixed
stage: -> resolved
status: open -> closed
___
Python tracker
<http://bugs.python.or
Charles-François Natali added the comment:
This patch should probably be moved to its own issue.
--
___
Python tracker
<http://bugs.python.org/issue22120>
___
___
Changes by Charles-François Natali :
--
Removed message: http://bugs.python.org/msg224550
___
Python tracker
<http://bugs.python.org/issue22120>
___
___
Python-bug
New submission from Charles-François Natali:
I noticed that socket.sendto() got noticably slower since 3.4 (at least),
compared to 2.7:
2.7:
$ ./python -m timeit -s "import socket; s = socket.socket(socket.AF_INET,
socket.SOCK_DGRAM); DATA = b'hello'; TARGET=('127.0.0.1
Changes by Charles-François Natali :
--
title: performance regression in socket.getsockaddr() -> performance regression
in socket getsockaddrarg()
___
Python tracker
<http://bugs.python.org/issu
Charles-François Natali added the comment:
> For Python, the encoder is only used when you pass a Unicode string.
Hm...
I'm passing ('127.0.0.1', 4242)as destination, and you can see in the
above profile that the idna encode function is called.
This does
Charles-François Natali added the comment:
OK, I think I see what you mean:
$ ./python -m timeit -s "import socket; s =
socket.socket(socket.AF_INET, socket.SOCK_DGRAM)" "s.sendto(b'hello',
('127.0.0.1', 4242))"1 loops, best of 3: 44.7 usec per loop
$
Charles-François Natali added the comment:
> Note that even the bytes version is still quite slow. UDP is used for
> light-weight protocols where you may send thousands or more messages per
> second. I'd be curious what the sendto() performance is in raw C.
Ah, I wouldn't r
Charles-François Natali added the comment:
Parsing a bytes object i.e. b'127.0.0.1' is done by inet_pton(), so
it's probably cheap (compared to a syscall).
If we had getaddrinfo() and gethostbyname() return bytes instead of
strings, it would
Charles-François Natali added the comment:
> Charles-François: you get the idna overhead in 2.7, too, by specifying
> u'127.0.0.1' as the address.
I don't see it in a profile output, and the timing doesn't change
whether I pass '12
Charles-François Natali added the comment:
> Please understand that Victor and I were asking you to pass a *unicode*
> object, with a *u* prefix. For me, the time more-than-doubles, on OSX, with
> the system python.
Sorry, I misread 'b'.
Charles-François Natali added the comment:
Thanks, I committed a simpler version of the patch.
--
resolution: -> fixed
stage: test needed -> resolved
status: open -> closed
___
Python tracker
<http://bugs.python.or
Charles-François Natali added the comment:
Patch attached.
The test wouldn't result in FD exhaustion on CPython because of the reference
counting, but should still trigger RessourceWarning.
--
keywords: +patch
nosy: +haypo, pitrou
stage: -> patch review
Added fi
Charles-François Natali added the comment:
> In this case, the issues are being caused by the following kernel parameters
> that we have for our default build -
>
> #
> ## TIBCO network tuning #
> #
> net.core.rm
Charles-François Natali added the comment:
Let's try with this instead:
>>> from socket import socket, SO_SNDBUF, SOL_SOCKET
>>> s = socket()
>>> s.getsockopt(SOL_SOCKET, SO_SNDBUF)
--
___
Python tracker
&l
Charles-François Natali added the comment:
Agreed with Antoine and Benjamin.
--
___
Python tracker
<http://bugs.python.org/issue21963>
___
___
Python-bugs-list m
Charles-François Natali added the comment:
Note that I'm not fussed about it: far from simplifying the code, it
will make it more complex, thus more error-prone.
--
___
Python tracker
<http://bugs.python.org/is
Charles-François Natali added the comment:
> The patch is not compatible with 3.4. Does this bug exist in 3.4?
No, selectors all expect a timeout in seconds, so this should be fixed in 3.4.
--
___
Python tracker
<http://bugs.python.org/issu
Charles-François Natali added the comment:
Since the patch doesn't use O_EXCL for the temporary file, it suffers
from the exact same race condition as the current code. It should be
using the "x" open mode.
--
nosy: +neologix
___
Charles-François Natali added the comment:
>
>
> > Since the patch doesn't use O_EXCL for the temporary file, it suffers
> > from the exact same race condition as the current code.
>
> This does not make race condition. Only one thread writes .tmp files.
>
> Th
Charles-François Natali added the comment:
Actually no, I don't understand this patch: if the makefile was correct, C
files depending on Include/Python-ast.h and Python/Python-ast.c shouldn't
be built before those files have been generated by asdl_c.py.
The problem doesn't
Charles-François Natali added the comment:
Hum...
Correct me if I'm wrong, but destroying the thread state of daemon threads
while they're running is really a bad idea in fact: for example, if
warnings are now emitted for unclosed file objects, this means that the
file object, and all
Charles-François Natali added the comment:
Here's a patch.
--
Added file: http://bugs.python.org/file33113/makefile_ast_h.diff
___
Python tracker
<http://bugs.python.org/is
Changes by Charles-François Natali :
--
assignee: -> neologix
___
Python tracker
<http://bugs.python.org/issue20065>
___
___
Python-bugs-list mailing list
Un
Charles-François Natali added the comment:
AFAICT, this also affects poll().
Although it's supposed to be passed an integer, passing a float will result
in a truncation towards 0:
"""
$ strace -e poll python -c "import select; p = select.poll(); p.poll(0
Charles-François Natali added the comment:
Just so it's clear, those bugs are theoretical: whether you pass 1e-7/1e-10 or
0 to select/kqueue is exactly the same (entering/leaving the syscall takes some
time)...
--
___
Python tracker
Charles-François Natali added the comment:
==
> FAIL: test_timeout_rounding (test.test_epoll.TestEPoll)
> --
> Traceback (most recent call last):
>
Charles-François Natali added the comment:
Sorry for the delay, I had completely forgotten this issue.
Igor, could you try the patch attached?
--
keywords: +patch
Added file: http://bugs.python.org/file33664/af_can_define.diff
___
Python tracker
Charles-François Natali added the comment:
> Ah? The manual page of epoll_wait() says:
>
> "The timeout argument specifies the minimum number of milliseconds that
epoll_wait() will block. (This interval will be rounded up to the system
clock granularity, and kernel scheduling de
Charles-François Natali added the comment:
> Did you read my Tulip? Maybe I didn't explain correctly.
No, it was quite clear, and I think I understand the original issue:
calling epoll_wait(0) when passed a timeout of 0.9ms was bad, because it
led to busy-loop during 0.9ms.
But here
Charles-François Natali added the comment:
> If the patch is accepted, my changes on Python 3.3 should also be
reverted.
I'm sorry, but I'm not convinced.
The selector's granularity is an implementation detail, and I don't think
it should be exposed.
Furthermore, it'
Charles-François Natali added the comment:
LGTM.
--
nosy: +neologix
___
Python tracker
<http://bugs.python.org/issue20331>
___
___
Python-bugs-list mailin
Charles-François Natali added the comment:
> > Once again, what's wrong with your initial approach of ceiling the
timeout?
>
> It looks like changing the rounding method doesn't solve anything.
> selector.select(timeout) may still take less than timeout, so it
> does
Charles-François Natali added the comment:
> > But what problem does it cause if, once in a while, the call takes less
> > than the passed timeout?
> > If that's the case, you'll simply perform another loop, an wake up 1ms
> > later, that's all.
>
Charles-François Natali added the comment:
The culprint isn't Python, but the libc:
"""
$ ./python -c "import socket; print(socket.getaddrinfo('836937931829', 80,
socket.AF_INET, 0, socket.SOL_TCP))"
Traceback (most recent call last):
File ""
Changes by Charles-François Natali :
--
resolution: -> invalid
stage: -> committed/rejected
status: open -> closed
___
Python tracker
<http://bugs.python.or
Charles-François Natali added the comment:
> It looks like select() and poll() in Linux < 2.6.28 has a resolution
> of 1/HZ, where HZ can be retrieved from os.sysconf('SC_CLK_TCK'). Since
> Linux 2.6.28, hrtimers are now used for timeouts.
> Attached patch improve
Charles-François Natali added the comment:
Well, now that timeouts are properly rounded, the granularity is useless.
--
___
Python tracker
<http://bugs.python.org/issue20
Charles-François Natali added the comment:
> I don't think so. Please read again the issue #20452, for example this
message:
>
> http://bugs.python.org/issue20452#msg209772
"""
Ok, it looks better: waiting 99.9 ms took 99.6 ms and 99.9 ms, and waiting
9.9 ms t
Charles-François Natali added the comment:
> > Once again: a slight early wakeup isn't an issue
>
> That's your opinion, but I disagree.
>
> Please open a new issue with a patch, or reopen at least this issue because
> it is now closed.
>
> I already spent
Charles-François Natali added the comment:
> Shouldn't this be fixed in the C implementation of the select module or
in selectors.py? It seems likely that the exact range might be different
for each syscall and possibly per OS or even OS version.
Agreed: if we want to fix this, it s
Charles-François Natali added the comment:
> BreamoreBoy: why did you remove Arfrever from this issue?
Noisy lists members are sorted by alphabetical order: since Arfrever comes just
before BreamoreBoy, I assume his fingers tripped ;-)
--
nosy: +Arfrever, neolo
Charles-François Natali added the comment:
> To solve a performance issue in asyncio, I added a new resolution
attribute to selectors.BaseSelector and a new _granularity attribute to
asyncio.BaseEventLoop. If I understood correctly, Charles-François (author
and so maintainer of the new select
Charles-François Natali added the comment:
> How sure are you? Suppose I use poll() with a 0.5 msec timeout. This
> presumably gets rounded up to 1 msec. But if the system clock has e.g. a
10
> msec resolution, won't this still wait 0 msec? Or will it wait until the
> next &qu
Charles-François Natali added the comment:
Thanks for the detailed report.
> connections.py from multiprocessing has been rewrittend between 3.2
> and 3.3 but I can't see what's wrong in the way it has been done,
> basic socket options seem to be exactl
Charles-François Natali added the comment:
> select() and kqueue() are able to sleep less than 1 ms. Using a slack of
1 ms would reduce the accuracy. I don't see why we should limit the
accuracy. Why 1 ms? Because of poll/epoll? What about Windows and its
resolution of 15.6 ms?
>
>
Charles-François Natali added the comment:
> Updated patch with the 3.2 heuristic for deciding when to concatenate.
LGTM.
For posterity, a quick benchmark performing 100 trivial remote calls:
Before:
$ ./python ~/test_manager.py
8.202659845352173
After:
$ ./python ~/test_manager
Charles-François Natali added the comment:
Just a quick comment on the patch: in as_completed(), if the future is
cancelled or finished, control is yielded back to the caller with the
condition's lock held.
As a general rule, libraries should not yield control to the caller with a lock
Charles-François Natali added the comment:
> Note that the context manager will be called in this case to release the
lock before f is yielded to the caller.
>
> class MiniContext():
> def __init__(self):
> pass
>
> def __enter__(self):
> pr
Charles-François Natali added the comment:
> I'm asking why it's 3 seconds by default in the tests?
> Could we modify the value to 6 instead of 3, is it acceptable?
No, see this comment:
"""
self.assertRaises(KeyboardInterrupt, lock.acquire, timeout=5)
dt
Charles-François Natali added the comment:
Should be fixed now.
--
resolution: -> fixed
stage: -> committed/rejected
status: open -> closed
___
Python tracker
<http://bugs.python.or
Charles-François Natali added the comment:
Wow, 10 messages in one night...
Could you try to gather all your finding at once, because reading so many
messages in difficult to follow?
> GetQueuedCompletionStatus(1 ms)->None took 0.307 ms (monotonic: 0.000 ms)
So basically, on Windows,
Charles-François Natali added the comment:
> It's not easy because I collect informations from various buildbots and
different virtual machines. But I planned to try to summarize the overall
work done on time in asyncio, select and selectors.
Thanks for the summary.
> The first prob
Charles-François Natali added the comment:
> STINNER Victor added the comment:
>
>> Just so it's clear, those bugs are theoretical: whether you pass
>> 1e-7/1e-10 or 0 to select/kqueue is exactly the same (entering/leaving the
>> syscall takes some time)...
>
>
Charles-François Natali added the comment:
> So this looks like a bug.
I mean an OpenBSD bug.
--
___
Python tracker
<http://bugs.python.org/issue20564>
___
_
Charles-François Natali added the comment:
So this looks like a bug.
When running the test, is faulthandler enabled (--timeout option)?
--
___
Python tracker
<http://bugs.python.org/issue20
Charles-François Natali added the comment:
> I didn't see test_threading failing anymore recently, so I'm closing the
issue. Thanks.
Hm...
Could someone review my message http://bugs.python.org/msg206028?
Because AFAICT, this will lead to random crashes with d
Charles-François Natali added the comment:
Random thoughts:
- executor-created threads are daemon (unfortunately)
- see issue #19466: I think that the change to clear the frame of daemon
threads was a mistake
--
___
Python tracker
<h
Charles-François Natali added the comment:
Well, then it's definitely an OpenBSD bug, and we can't do anything
more than skip the test.
--
___
Python tracker
<http://bugs.python.o
Charles-François Natali added the comment:
> It seems that the EINTR should be caught by the standard library in all
cases:
> http://bugs.python.org/issue1628205
Yes, it should.
> But it's not the case for the "socket.create_connection" method (verified
in 3.3 and 2.7
Charles-François Natali added the comment:
Please revert.
To debug, since I guess it's due to a memory corruption because objects are
deallocated while they're still in use, you could try to use valgrind.
Unfortunately, since it's due to a race condition, the overhead will
p
Charles-François Natali added the comment:
FWIW I agree with Antoine and Martin: ignoring EBADF is a bad idea,
quite dangerous.
The man page probably says this to highlight that users shouldn't
*retry* close():
"""
Retrying the close() after a failure return is t
Charles-François Natali added the comment:
Shouldn't the documentation be updated?
https://docs.python.org/3.6/library/weakref.html#weakref.WeakKeyDictionary
Note Caution: Because a WeakKeyDictionary is built on top of a Python
dictionary, it must not change size when iterating over it.
Charles-François Natali added the comment:
One reason for not calling sys.exit() is because on Linux, the default
implementation uses fork(), hence the address space in the chilren is
a clone of the parent: so all atexit handlers, for example, would be
called multiple times.
There's als
Charles-François Natali added the comment:
Could you try with recent checkout of python 2.7?
I wonder if this could be an occurrence of issue #13992 fixed by Antoine a
couple months ago.
--
nosy: +neologix
___
Python tracker
<http://bugs.python.
Charles-François Natali added the comment:
Could you please give an example of why this would be a useful addition?
I can see the usage of removing a handler, but the user should know which
handlers are registered.
--
nosy: +neologix
___
Python
Charles-François Natali added the comment:
> Barry advised me to open this issue as it's a functional regression from
> Python 2.
But it was relying on a private and non documented feature in Python
2, so it's not really a regression.
> My use case is unit testing code th
Charles-François Natali added the comment:
> Shouldn't you try to make the permission removal atomic?
> Otherwise there's a window of opportunity to exploit the suid bit.
Actually there's already a race even without setuid bit:
http://bugs.python.org/issue15100
All
Charles-François Natali added the comment:
The heapq C implementation is apparently not thread-safe:
PyObject_RichCompareBool() and Py_DECREF() can lead to arbitrary python code
being called, which can result in a context switch.
--
nosy: +neologix
Charles-François Natali added the comment:
Alright, here's what's going on.
When the main thread exits, it triggers the interpreter shutdown, which clears
all the tstates in PyInterpreterState_Clear():
"""
void
PyInterpreterState_Clear(PyInterpreterState *inter
Charles-François Natali added the comment:
> This shouldn't be a problem in Python 3 where Python threads cannot switch
> during shutdown.
What happens if the GIL is relased during shutdown?
Also, I'm a bit worried about this code:
"""
void
PyThreadState_Clear(
Charles-François Natali added the comment:
I'm closing, since issue #17025 proposes to do this as part of performance
optimization.
--
nosy: +neologix
status: open -> closed
superseder: -> reduce multiprocessing.Queue contention
___
Pyt
Charles-François Natali added the comment:
> Note that in my original application where I encountered this (with sqlite),
> the backtrace looks slightly different. It is at shutdown, but not at
> interpreter shutdown - the main thread is still running.
Could you post a traceback of t
Charles-François Natali added the comment:
> Here is one. Others are in the issue report on GitHub.
Yes, I've seen it, but I'd need a backtrace with line numbers (like
the one you posted above).
thread 5 is crashing, but I don
Charles-François Natali added the comment:
> - how do you know the crash really happens because of thread 5?
All other threads are blocked on locks or condition variables, it's
the only runnable thread.
> Another question: are threads being started or stopped while the thread loca
Changes by Charles-François Natali :
--
nosy: +pitrou
___
Python tracker
<http://bugs.python.org/issue16945>
___
___
Python-bugs-list mailing list
Unsubscribe:
Changes by Charles-François Natali :
Added file: http://bugs.python.org/file29219/cgi_subprocess-1.diff
___
Python tracker
<http://bugs.python.org/issue16945>
___
___
Charles-François Natali added the comment:
Thanks, it's surprising this was never noticed before.
Your patch has two issues:
- it doesn't clear the thread state before deleting it, which would leak the
frame, thread-specific dict, etc
- it only clears the thread states after the curr
Charles-François Natali added the comment:
I don't know how OS X crash report works, but it seems to have at
least some debug info available, since some ymbols are resolved in the
backtrace.
You might be able to get more info with gdb, with something like:
"""
gdb /path/to/py
Charles-François Natali added the comment:
> Note that in threadmodule.c, in local_clear, we are iterating through all
> threads:
>
> In PyDict_DelItem, if the GIL is released and meanwhile, the list of
> threadstates is altered, is that a problem for this loop? So maybe ts
Charles-François Natali added the comment:
And here's a patch.
--
keywords: +patch
Added file: http://bugs.python.org/file29244/thread_local_concurrent.diff
___
Python tracker
<http://bugs.python.org/is
Charles-François Natali added the comment:
> Wouldn't it be better to expose and re-use the HEAD_LOCK and HEAD_UNLOCK
> macros from pystate.c?
I don't like holding locks before calling "alien" code, it's a recipe
for deadlocks: for example, if another thread-local
Charles-François Natali added the comment:
> Did you forget to attach the patch?
Oops...
--
Added file: http://bugs.python.org/file29295/tstate_after_fork.diff
___
Python tracker
<http://bugs.python.org/issu
Charles-François Natali added the comment:
The test assumes that the input flags (EV_ADD, EV_ENABLE...) will be returned
in the output events. It's apparently not the case on OpenBSD and NetBSD (and
probably on OS-X neither, because this check is disabled on this platform), and
I can&
1501 - 1600 of 1826 matches
Mail list logo