Changes by Charles-François Natali :
--
nosy: +orsenthil
___
Python tracker
<http://bugs.python.org/issue16945>
___
___
Python-bugs-list mailing list
Unsubscribe:
Charles-François Natali added the comment:
> It's interesting. The example behaves unstable on 3.3+ with C implementation
> of picle, sometimes works, sometimes fails. With Python implementation and on
> 3.2 it works always. On 2.7 it fails always.
That's hash randomizatio
Changes by Charles-François Natali :
--
resolution: -> duplicate
stage: patch review -> committed/rejected
status: open -> closed
superseder: -> Broken test_kqueue.py on OpenBSD
___
Python tracker
<http://bugs.python
Charles-François Natali added the comment:
There are actually two distinct issues.
For the first one, the problem is really a distribution issue: the libc is more
recent than the kernel, and exports *utimes() whereas the kernel doesn't
implement those syscalls, which results in ENOS
Charles-François Natali added the comment:
Could you provide the patch?
It's trivial, but at least we'll make sure the patch fixes the problem on your
platform.
--
nosy: +neologix
___
Python tracker
<http://bugs.python.o
Changes by Charles-François Natali :
--
nosy: +v+python
___
Python tracker
<http://bugs.python.org/issue16945>
___
___
Python-bugs-list mailing list
Unsubscribe:
Charles-François Natali added the comment:
Committed, thanks for the patch!
--
stage: -> committed/rejected
status: open -> closed
versions: +Python 2.7, Python 3.2, Python 3.4
___
Python tracker
<http://bugs.python.org/i
Charles-François Natali added the comment:
Time for a stupid question from someone who doesn't know anything about
Windows: if WSAPoll() is really terminally broken, is it really worth the
hassle exposing it and warping the API?
AFAICT, FD_SETSIZE is already bumped to 512 on Windows
Charles-François Natali added the comment:
> I have created an iocp branch.
You could probably report the fixes for spurious notifications in the
default branch.
--
___
Python tracker
<http://bugs.python.org/issu
Charles-François Natali added the comment:
> It appears that Linux's "spurious readiness notifications" are a deliberate
> deviation from the POSIX standard. (They are mentioned in the BUGS section
> of the man page for select.)
I don't think it's a del
Charles-François Natali added the comment:
> If only one byte is available, recv(4096) should simply return a partial
> result.
Of course, but how do you know if there's data left to read without
calling select() again? It's much better to call read() until you get
EAGAIN tha
Charles-François Natali added the comment:
> For SOCK_STREAM, yes, not for SOCK_DGRAM (or for a pipe when trying to
> write more than PIPE_BUF, although I guess any sensible implementation
> doesn't report the pipe write ready if there's less than PIPE_BUF
> space lef
Charles-François Natali added the comment:
> I thought SOCK_DGRAM messages just got truncated at the receiving end.
You were referring to partial writes: for a datagram-oriented
protocol, if the datagram can't be sent atomically (in one
send()/write() call), the kernel will return EA
New submission from Charles-François Natali:
Here's an implementation of the idea posted on python-ideas
(http://mail.python.org/pipermail/python-ideas/2013-January/018846.html).
The principle is really simple, we just serialize/unserialize the objects
before/after holding the locks.
Changes by Charles-François Natali :
Added file: http://bugs.python.org/file28813/multi_queue.py
___
Python tracker
<http://bugs.python.org/issue17025>
___
___
Python-bug
Charles-François Natali added the comment:
Hello,
> So I'm sorry if it's not a bug...
Don't be afraid, we don't byte :-)
Concerning your problem, my guess would be that one of the modules you import
sets the process CPU affinity (maybe as a workaround to mitigate t
Charles-François Natali added the comment:
> Indeed some value change when I print "cpu" line from /proc/self/status but I
> don't really understand what that mean...
It means that the CPU affinity is changed, so the scheduler binds your
process to a subset of the availa
Changes by Charles-François Natali :
--
resolution: -> invalid
stage: -> committed/rejected
status: open -> closed
___
Python tracker
<http://bugs.python.or
Charles-François Natali added the comment:
> I've added (some) docs and added checking of the BCM constants to the
> test_socket module.
This version looks good to me.
I'll commit it next week (I currently don't have access to my
development machine).
> I would
Charles-François Natali added the comment:
> Charles-François, are you going to finish this before 2.7.4 RC released? The
> patch should be updated to address Antoine's comments.
No.
I don't have access to a 64-bit box, which makes it difficult to write
(I don't get compil
Charles-François Natali added the comment:
> I would like to suggest again my idea of doing it in Connection instead,
> with new methods (e.g. locked_send and locked_recv). Especially given
> it can be useful in user code to have a thread-safe Connection (I'm in
> this situati
Charles-François Natali added the comment:
I'm unable to reproduce it.
Are you using the attached script?
Did you apply the patch by hand, or are you using a recent Python version?
--
___
Python tracker
<http://bugs.python.org/is
Charles-François Natali added the comment:
> For the record, I tried the Connection approach and here is what I ended up
> with.
I don't really like the API.
Having to pass an external lock is IMO a bad idea, it should be a
private instance field.
Also, for consistency we'
Charles-François Natali added the comment:
> Using handle_accept() in my code and remembering to call listen() in my
> asyncore.dispatcher server's constructor did the trick.
>
> I am not sure if we still have a bug here though, since if the subclass
> doesn't define a
Charles-François Natali added the comment:
You happen to have a script named time.py, so when the subprocess module is
imported, it imports this script instead of the correct time module.
Nothing is wrong, closing.
--
nosy: +neologix
resolution: -> invalid
stage: -> com
Charles-François Natali added the comment:
Committed.
Brian, thanks for the patch!
--
resolution: -> fixed
stage: -> committed/rejected
status: open -> closed
___
Python tracker
<http://bugs.python.or
Charles-François Natali added the comment:
> Have you read and checked what the RFCs about the
> FTP protocol say about maximum line length?
vsftpd seems to use a 4096 limit (and the guy knows his stuff :-):
ftp://vsftpd.beasts.org/users/cevans/untar/vsftpd-3.0.2/defs.h
-
Charles-François Natali added the comment:
> Well, that is not from RFC (or I hadn't find it):) however I'd lie if I'd
> call myself an expert, should I change limit to 4096 then?
It's probably not in the RFC: this just shows that the lim
Changes by Charles-François Natali :
--
status: open -> pending
___
Python tracker
<http://bugs.python.org/issue18907>
___
___
Python-bugs-list mailing list
Un
Charles-François Natali added the comment:
Alright, I committed it to default.
I don't think it should go into 2.7 or 3.3, since it's more an
enhancement than a bug fix.
--
___
Python tracker
<http://bugs.python.o
New submission from Charles-François Natali:
Currently, in case of invalid file descriptor (which can happen easily e.g. if
the user closes the FD before unregistering it), selector.select() will react
differently depending on the backend:
SelectSelector fails with EBADF
PollSelector returns
Charles-François Natali added the comment:
> Or we could remove the fd *and* still raise an error the first time, or at
> least a warning. It sounds like a programming error, so I'm not sure it's a
> good idea to silence it.
Yeah, sure. By "silently" I meant "
Charles-François Natali added the comment:
Closing, thanks for the report!
--
resolution: -> fixed
stage: patch review -> committed/rejected
status: open -> closed
versions: -Python 2.7, Python 3.3
___
Python tracker
<http://bug
Changes by Charles-François Natali :
--
nosy: -neologix
___
Python tracker
<http://bugs.python.org/issue18709>
___
___
Python-bugs-list mailing list
Unsubscribe:
Charles-François Natali added the comment:
I noticed this when converting the code to use PyModule_AddIntMacro().
Vajrasky, did you see Ezio's review on
http://bugs.python.org/review/18994/#ps9258 ?
--
___
Python tracker
<http://bugs.py
Charles-François Natali added the comment:
> Interesting issue. ISTM that closing the FD before unregistering it is a
> programmer's mistake that shouldn't pass silently. And closing it in a
> separate thread while the selector is active sounds like an even bigger bug.
Charles-François Natali added the comment:
> So this may have been a DNS related issue, perhaps a load balancer referring
> the connection to a dead machine. Yet, for some reason, the Windows command
> line FTP client can recover from this problem after 20 seconds? What are they
Charles-François Natali added the comment:
Well, as Ned explained, that's a libc bug.
If getaddrinfo() fails with EAI_SYSTEM, errno should be set.
There's nothing more Python can do.
If the error is reproducible, you can run the failing code with strace, to see
what's going o
Charles-François Natali added the comment:
(replying to Guido's post in another thread)
> Charles-Francois, sorry to add you back to the bug, but (a) I thought you had
> agreed to a compromise patch that restarts signals in most cases but not for
> select(), poll() etc.; (b) I
New submission from Charles-François Natali:
The patch attached uses selector in telnetlib.
This removes a lot of duplicated code.
--
components: Library (Lib)
files: telnetlib_selectors.diff
keywords: needs review, patch
messages: 198986
nosy: neologix
priority: normal
severity: normal
New submission from Charles-François Natali:
This adds a keys() method to selectors, to return all the registered keys.
It's useful, because one often needs to loop until all registered file objects
have been unregistered e.g. (inspired from subprocess, see #18923):
while selector
Charles-François Natali added the comment:
Here's a patch updating subprocess to use selectors.
(It depends on the new keys() method - issue #19172.)
--
dependencies: +selectors: add keys() method
keywords: +needs review, patch
nosy: +neologix
stage: -> patch review
Added fi
Charles-François Natali added the comment:
Here's an updated patch (the previous one mistakenly removed a testcase).
--
Added file: http://bugs.python.org/file31970/telnetlib_selectors-1.diff
___
Python tracker
<http://bugs.python.org/is
Charles-François Natali added the comment:
> If you want to unregister while iterating on .keys(), just copy .keys(), as
> I do when removing items from a list or a dict while iterating on it.
Of course, but as noted by Antoine, it makes the return type dependent of an
implementation det
Charles-François Natali added the comment:
> BaseSelector.register(fd) raises a KeyError if fd is already registered,
> which means that any selector must know the list of all registered FDs. For
> EpollSelector, the list may be inconsistent *if* the epoll is object is
> modifie
Charles-François Natali added the comment:
> R. David Murray added the comment:
OSError: [Errno 23] Too many open files in system:
'/home/rdmurray/p34/Lib/tarfile.py'
Well, the buildbot is hitting ENFILE.
Try to run "lsof" as root, to see which process (more likely
proce
Charles-François Natali added the comment:
> Here is a more useful traceback:
If the failures aren't linked to ENFILE, then you could use strace to
find the process on which the test is doing a waitpid(), and then
perform an strace and gdb on that process to see where it's stuck.
Charles-François Natali added the comment:
Honestly, I think the extra complexity and non-portability isn't worth it.
--
___
Python tracker
<http://bugs.python.org/is
Charles-François Natali added the comment:
I think the problem is that those buildbots are really slow (just look at the
second buildbot's backlog), and the signal is delivered before the large buffer
is allocated, hence the write() syscall doesn't fail with EINTR.
--
nosy:
Charles-François Natali added the comment:
Actually, it should probably be using a generator-based version of os.listdir().
See #11406.
--
dependencies: +There is no os.listdir() equivalent returning generator instead
of list
nosy: +neologix
Changes by Charles-François Natali :
--
nosy: +haypo, pitrou
___
Python tracker
<http://bugs.python.org/issue19170>
___
___
Python-bugs-list mailing list
Unsub
Charles-François Natali added the comment:
> It looks as though fork() is indirectly calling something in
> _ssl.cpython-34dm.so which is not
> completing.
The stack is strange: what is /lib/libncursesw.so.5 doing there?
Also, we can see exit() is being called. And indeed, _fini(
Charles-François Natali added the comment:
Well, I don't mean to be a pain, but I still think this patch
shouldn't have been applied: it makes fork() not async-safe, which
can trigger hard to debug, random bugs (#19227 might or might not be
a direct consequence).
at the very least,
Charles-François Natali added the comment:
Does AIX have an equivalent to Linux's strace?
If yes, having a sample output would be helpful.
Otherwise, maybe just adding manual debug traces to PollSelector.select() would
help to see which events are reported.
--
nosy: +neo
Charles-François Natali added the comment:
Wait, I think there's a misunderstanding here.
POSIX does indeed leave undefined the behavior of syscalls when a
signal is received: whether or not it fails with EINTR depends on the
operating system, the SA_RESTART flag, etc.
But the tests
Charles-François Natali added the comment:
> Guido van Rossum added the comment:
>
> Actually, I think we're talking about the actual select() syscal not
> returning when SIGCHLD is received and handled. Running the C-level handler
> isn't enough, the syscall must return
Charles-François Natali added the comment:
Here's a patch changing asyncio to not rely on a Python-level signal handler at
all: it reads the signal number from the self-pipe.
It should therefore work even if the select()/poll()/etc syscall doesn't fail
with EINTR, and incidentally re
Charles-François Natali added the comment:
> The patch cleans up a lot of the other failures for asyncio on AIX
Great.
> but does not fix the hang.
I was expecting this :-)
Note that the hang is really strange, especially since the first
run_until_complete() succeeds.
Withou
Charles-François Natali added the comment:
> Guido van Rossum added the comment:
>
> Actually, I think we're talking about the actual select() syscal not
> returning when SIGCHLD is received and handled. Running the C-level handler
> isn't enough, the syscall must return
Charles-François Natali added the comment:
> David Edelsohn added the comment:
>
> AIX has an equivalent to strace (called truss). I have recorded all AIX
> system calls and signals for test_process_interactive, which hangs, following
> all children created by fock. The uncom
Charles-François Natali added the comment:
> Guido van Rossum added the comment:
>
>> Apparently, the stdout pipe was closed by the parent process
>
> Could it be that selecting for *read* on the *write* end of a pipe is
> always ready?
That's exactly what I was thi
New submission from Charles-François Natali:
See https://groups.google.com/forum/#!topic/python-tulip/9T2_tGWe0Sc
The attached patch just changes waitpid(0) to waitpid(-1), and comes with a
trivial test.
--
components: Library (Lib)
files: asyncio_process_group.diff
keywords: easy
New submission from Charles-François Natali:
The current SIGCHILD handler has two bugs:
- it reschedules itself if waitpid() returns 0: so if this ever happens, it
will enter a busy-loop until all children have exited
- it doesn't reschedule itself if waitpid() succeeds in reaping a
Charles-François Natali added the comment:
> I guess we'll have to write platform-dependent code and make this an
> optional feature. (Essentially, on platforms like AIX, for a
> write-pipe, connection_lost() won't be called unless you try to write
> some more bytes to it.)
Charles-François Natali added the comment:
Committed! (The retry loop is addressed in another issue).
--
resolution: -> fixed
stage: patch review -> committed/rejected
status: open -> closed
___
Python tracker
<http://bugs.python.or
Charles-François Natali added the comment:
I don't mean to be rude, but AIX is IMO a niche platform: proprietary Unices
are doomed to disappear sooner or later.
As such, I'm not convinced that adding an AIX-specific polling interface is
really worth it.
--
nosy:
Charles-François Natali added the comment:
I'm completely lost: how many issues is this thread conflating?
For example, David, I think you said that my patch regarding signal
handling did fix some tests.
Is that still the case?
If yes, could you open a specific issue for the remaining fa
Charles-François Natali added the comment:
> David Edelsohn added the comment:
>
> This enhancement issue is not a demand that you or anyone else in the Python
> community volunteer to implement it. AIX is a very lucrative business and
> deployed in many businesses that the P
New submission from Charles-François Natali:
(This is a spinoff from http://bugs.python.org/issue19293#msg200598)
When SIGCHLD is received, _sig_chld() is executed:
def _sig_chld(self):
[...]
transp = self._subprocesses.get(pid)
if transp is not None
Charles-François Natali added the comment:
Just to let you know that I'm not forgetting this issue.
I'd just like to iron-out issue #19172 (keys()-like method for selectors),
since it could impact the API, to avoid repeated r
Charles-François Natali added the comment:
Gregory, did you make any progress on this?
I think it would be a nice addition.
--
___
Python tracker
<http://bugs.python.org/issue11
Charles-François Natali added the comment:
> Christian Heimes added the comment:
>
> I think it's more likely that my patch is triggering an existing bug. The
> locking code for the SSL module and OpenSSL doesn't release locks on fork. I
> have attached an experiment
Charles-François Natali added the comment:
I took the freedom to push a fix for the test_pydoc failures (all the buildbots
were red). It should fix the failures, but since I'm not familiar with the
code, it'd be good to have someone double-check it.
Ethan, you just broke all the
Charles-François Natali added the comment:
I have a couple random remarks:
- now that FDs are non-inheritable by default, fork locks around
subprocess and multiprocessing shouldn't be necessary anymore? What
other use cases does the fork-lock have?
- the current implementation keeps
Charles-François Natali added the comment:
> Other parts of your reasoning are somewhat circular because most of the
> problems on AIX are Linux-isms (or BSD-isms where BSD has not yielded to
> Linux behavior). So you argue against adding lots of AIX-specific code, yet
> fixing
Charles-François Natali added the comment:
> Richard Oudkerk added the comment:
>
>> - now that FDs are non-inheritable by default, fork locks around
>> subprocess and multiprocessing shouldn't be necessary anymore? What
>> other use cases does the fork-lock have?
Charles-François Natali added the comment:
> This is by design. Let me try to defend the design.
OK, if that's a know limitation, then that's fine.
It would be nice to add maybe a note somewhere in the documentation, so that
people don't get bitten by this (and also proba
Charles-François Natali added the comment:
> 3- From another process, like bash, continuously write to the file.
That's the problem: reducing (actually truncating in your case) a file
currently mmaped *can only result in a core dump*: when you try to read from a
location in memo
Charles-François Natali added the comment:
> Well it is customary for callback-based APIs to hold strong references to
> their callbacks. If a library wants to avoid leaks, it should register a
> single callback which will then walk the current "live" resources and protec
Changes by Charles-François Natali :
--
resolution: -> fixed
stage: patch review -> committed/rejected
status: open -> closed
___
Python tracker
<http://bugs.python.or
Charles-François Natali added the comment:
You can go ahead!
--
___
Python tracker
<http://bugs.python.org/issue19339>
___
___
Python-bugs-list mailing list
Unsub
Charles-François Natali added the comment:
I'm also skeptical: I doubt the list allocation weights much compared
to the syscall.
And like Christian, I'm curious to know which implementation you'd chose.
--
___
Python tracker
<http
Charles-François Natali added the comment:
The test is failing with ENOSYS on one of the buildbots:
"""
==
ERROR: test_prlimit (test.test_resour
Charles-François Natali added the comment:
> I didn't expect to see a combination of glibc with prlimit() and Kernel
> without prlimit(). According to man prlimit it's not suppose to fail with
> ENOSYS, too.
Yeah, we've seen this several times on some buildbots.
Charles-François Natali added the comment:
This failure doesn't make sense: whether SIGALRM is delivered or not, epoll()
should time out after 2 seconds, not 105s.
What's more, here's another test_sub
Charles-François Natali added the comment:
> Flaky VM clock perhaps?
Yeah, thought about that, but here we see *exactly* the same value for
two time intervals (using time.monotonic()), computed over different
test suite runs. I'd expected something more
Charles-François Natali added the comment:
Could you try to run the test in a loop under strace ("strace -ttTf") ?
--
___
Python tracker
<http://bugs.python.o
Charles-François Natali added the comment:
Victor, did you see Serhiy's comment?
--
___
Python tracker
<http://bugs.python.org/issue19339>
___
___
Pytho
Charles-François Natali added the comment:
> Antoine Pitrou added the comment:
>
> FWIW, I think the "ideal" solution would be for keys() (*) to return a
> read-only Mapping implementation, allowing for file object lookup (using
> __getitem__) as well as iteration on Sel
Charles-François Natali added the comment:
Note that no matter where the atfork-hook is executed (prepare,
parent, child), it will still be deadlock-prone in case of
double-fork, since the hook will effectively be called after the first
fork (and before the second one). And double-fork is common
Charles-François Natali added the comment:
> Seriously, nobody gives a ... about SPARC and MIPS. :) It's nice that
> Python still works on these CPU architectures. But I neither want to
> deviate from the SipHash24 implementation nor make the code slower on
> all relevant plat
Charles-François Natali added the comment:
>> Well, unaligned memory access is usually slower on all architectures :-)
>> Also, I think some ARM architectures don't support unaligned access, so
>> it's not really a thing of the past...
>
> On modern computer
Charles-François Natali added the comment:
Here's another failure:
"""
==
FAIL: test_interpreter_shutdown
(test.test_concurrent_futures.Pro
Charles-François Natali added the comment:
> Also, unless people haven't been reporting other errors, these buildbots have
> been stable up until recently
Well, the fact that several tests - which were working just fine -
started failing randomly recently (like test_concurr
Charles-François Natali added the comment:
I agree this should be fixed.
Robert, want to submit a patch?
--
___
Python tracker
<http://bugs.python.org/issue19
Charles-François Natali added the comment:
Here's an updated patch using the new selector.get_map() method.
It removes ~100 lines to subprocess, which is always nice.
--
Added file: http://bugs.python.org/file32429/subprocess_selectors-1
Charles-François Natali added the comment:
Committed.
Antoine, thanks for the idea and patch!
--
resolution: -> fixed
stage: patch review -> committed/rejected
status: open -> closed
___
Python tracker
<http://bugs.python.or
Charles-François Natali added the comment:
> Berker Peksag added the comment:
>
> + .. method:: get_map()
> +
> + Return a mapping of file objects to selector keys.
>
> I kind of feel like a walking linter though I think this needs a versionadded
> tag :)
Well, s
Changes by Charles-François Natali :
Added file: http://bugs.python.org/file32448/subprocess_selectors-2.diff
___
Python tracker
<http://bugs.python.org/issue18
Charles-François Natali added the comment:
There are actually two reasons to choosing poll over epoll/kqueue
(i.e. no extra FD):
- it's a bit faster (1 syscall vs 3)
- but more importantly - and that's the main reason I did it in
telnetlib/multiprocessing/subprocess - sometimes, you re
1301 - 1400 of 1826 matches
Mail list logo
