Charles-François Natali added the comment:
> Given PEP 446 (fds are now CLOEXEC by default) I prepared an updated patch
> where the fork lock is undocumented and subprocess no longer uses the fork
> lock. (I did not want to encourage the mixing of threads with fork() without
&g
Charles-François Natali added the comment:
Of course, when I have 300 connections to remote nodes, I use poll()
to multiplex between them.
But there are times when you can have a large number of threads
running concurrently, and if many of them call e.g.
subprocess.check_output() at the same
Charles-François Natali added the comment:
> What is the status of this issue?
AFAICT, we haven't reached a consensus yet on the best way to handle EBADF.
--
___
Python tracker
<http://bugs.python.org
Charles-François Natali added the comment:
> By which you mean that you still don't agree with my proposal? Which is to
> fix it for most syscalls but not for select(), poll() and similar (anything
> that the new selectors module interfaces to).
I agree with your proposal, but
Charles-François Natali added the comment:
Here's an updated patch with a better logic: in the previous version - based on
current poll-based implementation, the FD was inferred from the event (i.e.
read ready -> stdout/stderr, write ready -> stderr). The new version directly
check
Changes by Charles-François Natali :
--
resolution: -> fixed
stage: patch review -> committed/rejected
status: open -> closed
___
Python tracker
<http://bugs.python.or
Charles-François Natali added the comment:
> When I try this with a UDP socket, the thread calling shutdown
> raises an OS Error (transport end point not connected).
Which is normal, since UDP sockets aren't connected.
> In contrast, sock.close does not cause the blocked thr
Changes by Charles-François Natali :
Added file: http://bugs.python.org/file32549/3bf73dcd0b42.diff
___
Python tracker
<http://bugs.python.org/issue18874>
___
___
Pytho
Changes by Charles-François Natali :
--
resolution: -> wont fix
stage: -> committed/rejected
status: open -> closed
___
Python tracker
<http://bugs.python.or
Charles-François Natali added the comment:
> After some research...
>
>> Which is normal, since UDP sockets aren't connected.
>
> But UDP sockets can be connected!
>
No, they can't.
"Connecting" a UDP socket doesn't established a duplex connec
Charles-François Natali added the comment:
I think it's simply due to file descriptor inheritance (files being
inherited by other subprocess instance): since Windows can't remove open
files, kaboom. It doesn't have anything to do with threads.
--
Charles-François Natali added the comment:
> Bernt Røskar Brenna added the comment:
>
> @neologix: How can it not have anything to do with threads?
>
> - It always works with max_workers == 1
> - When max_workers == 2, shutil.rmtree sometimes fails
It has nothing to do with
Charles-François Natali added the comment:
> mpb added the comment:
>
> Someone wrote a kernel patch based on my bug report.
>
> http://www.spinics.net/lists/netdev/msg257653.html
It's just a patch to avoid returning garbage in the address.
But AFAICT, recvfrom() returning
Charles-François Natali added the comment:
> R. David Murray added the comment:
>
> neologix noted that *when redirection is used* the way that *all* windows
> file handles are inherited changes.
That's true (but that was from
Charles-François Natali added the comment:
Would it be possible to generate a clean patch?
The latest one contains many unrelated commits.
--
nosy: +neologix
___
Python tracker
<http://bugs.python.org/issue18
Charles-François Natali added the comment:
> Serhiy Storchaka added the comment:
>
> I don't know. But a function call have a cost.
I'm with Serhiy here.
Writing a performance optimization without benchmark is generally a
bad idea (
Charles-François Natali added the comment:
I made a review at http://bugs.python.org/review/18874/#ps9860 (not sure you
got notified).
--
___
Python tracker
<http://bugs.python.org/issue18
Charles-François Natali added the comment:
If you really want to use packing, keep it.
But please remove this:
"""
+/* ensure that the frame_t structure is packed */
+assert(sizeof(frame_t) == (sizeof(PyObject
Charles-François Natali added the comment:
> STINNER Victor added the comment:
>
> I'm not convinced myself that hashtable.c/h can be reused immediatly, so I
> prefer to move these two files to Modules. The files may be moved later if
> the containers are reused.
Please
Charles-François Natali added the comment:
Victor, is the attached patch up-to-date?
--
___
Python tracker
<http://bugs.python.org/issue18874>
___
___
Python-bug
Charles-François Natali added the comment:
> Antoine Pitrou added the comment:
>
> I think this is more of a documentation issue. People who don't want a new fd
> can hardcode PollSelector (poll has been POSIX for a long time).
That's also what I now think.
I don't
Charles-François Natali added the comment:
IMO, this functionality is subsumed by http://bugs.python.org/issue19776
(Path.expanduser).
--
nosy: +neologix
___
Python tracker
<http://bugs.python.org/issue19
Charles-François Natali added the comment:
> STINNER Victor added the comment:
>
> @neologix: I would like your opinion first, because you accepted the PEP.
Well, I'm not a native speaker, but "peak" does sound better than
"max" (I'd say the later
Charles-François Natali added the comment:
> Antoine Pitrou added the comment:
>
> Calling it _PyThread_set_key_value is prone to confusion.
> Ideally we would fix PyThread_set_key_value's behaviour. Are there users of
> the function outside from CPython?
The question i
Charles-François Natali added the comment:
I'm with Antoine, this is *really* going too far.
SOCK_CLOEXEC and friends are useful to avoid race conditions: there's
no such concern with the non-blocking flag.
Shaving one or two syscalls is IMO completely useless, and doesn't
justify
Charles-François Natali added the comment:
AFAICT, there's no link (FWIW I wrote the patch for #10517, then the fix
for threads created outside of Python calling into a subinterpreter -
issue #13156).
Actually, this "fix" would have avoided the regression in issue #13156.
But si
New submission from Charles-François Natali:
Here's a patch improving the "examples" section for the selectors module.
--
assignee: docs@python
components: Documentation
files: selectors_examples.diff
keywords: needs review, patch
messages: 204750
nosy: docs@python, ne
Changes by Charles-François Natali :
Removed file: http://bugs.python.org/file32894/selectors_examples.diff
___
Python tracker
<http://bugs.python.org/issue19836>
___
___
Changes by Charles-François Natali :
Added file: http://bugs.python.org/file32895/selectors_examples.diff
___
Python tracker
<http://bugs.python.org/issue19836>
___
___
Charles-François Natali added the comment:
Here's an updated patch adding your echo server to the examples section.
--
Added file: http://bugs.python.org/file32899/selectors_example-1.diff
___
Python tracker
<http://bugs.python.org/is
Charles-François Natali added the comment:
Could someone with a dual-core machine try the attached simplistic
benchmark with and without Victor's patch?
I can see some user-time difference with 'time' on my single-core
machine, but I'm curious to see how this would affect
New submission from Charles-François Natali:
Initially, BaseSelector was simply designed as the base implementation used by
concrete ones like SelectSelector & Co.
Then BaseSelector evolved to be an ABC, but the problem is that it's really not
usable as such: the register() and u
Charles-François Natali added the comment:
Alright, here's a first step: select/poll/epoll/etc now return empty
lists/tuples upon EINTR. This comes with tests (note that all those tests
could probably be factored, but that's another story).
--
keywords: +needs review, patch
st
Charles-François Natali added the comment:
> I'm wondering, is there a reason we made BaseSelector a public API?
The idead was to have an ABC so that users can implement their own
selector, and pass it to e.g. asyncio or anything alse expecting a
selector.
Other than that, the only use
Charles-François Natali added the comment:
> The problem for documentation use is that we're christening it as an
> official API, and thus it becomes more difficult to refactor the
> inheritance hierarchy.
So what would you suggest?
--
Charles-François Natali added the comment:
Here's a patch adding socketpair to test.support.
This version has been used in test_selectors for quite some time now,
and would probably be useful for other tests as well.
--
Added file: http://bugs.python.org/file32909/socketpair-1
Charles-François Natali added the comment:
> Guido van Rossum added the comment:
>
> LGTM, although I wish that you had time to implement the comment "TODO:
> Subclasses can probably optimize this even further" rather than just
> shuffling the existing code around. :-)
Charles-François Natali added the comment:
Just for the record, I was initially in favor of recomputing the
timeout and retrying upon EINTR, but Guido prefers to return empty
lists, and since that's a better compromise than the current situation
(I've seen *many* people complainin
Charles-François Natali added the comment:
> STINNER Victor added the comment:
>
>> I think that's a cleaner design.
>
> For common containers like Sequence or Mapping, ABC are useful. But I don't
> expect new implementations of BaseSelector. Is it just for
Charles-François Natali added the comment:
Impressive speedup :-)
--
___
Python tracker
<http://bugs.python.org/issue19506>
___
___
Python-bugs-list mailin
Charles-François Natali added the comment:
> Well this is wishing thinking, since by returning an empty list you
> force the user to handle EINTR - just in a different way.
I know that returning an empty list changes the semantics: I just
think that's better - or not as bad - than
New submission from Charles-François Natali:
asyncio makes heavy use of SIGCHLD for subprocesses.
A consequence of this if that a lot of syscalls can fail with EINTR (see e.g.
issue #18885).
The attached patch uses SA_RESTART (through signal.siginterrupt()) to limit
EINTR occurrences, e.g
Charles-François Natali added the comment:
How about this?
--
keywords: +needs review, patch
stage: -> patch review
Added file: http://bugs.python.org/file32924/selectors_select_signal.diff
___
Python tracker
<http://bugs.python.org/issu
Changes by Charles-François Natali :
--
resolution: -> fixed
stage: patch review -> committed/rejected
status: open -> closed
versions: +Python 3.2 -Python 3.4
___
Python tracker
<http://bugs.python.or
Changes by Charles-François Natali :
--
status: open -> pending
___
Python tracker
<http://bugs.python.org/issue18843>
___
___
Python-bugs-list mailing list
Un
Changes by Charles-François Natali :
--
resolution: -> rejected
stage: patch review -> committed/rejected
status: pending -> closed
___
Python tracker
<http://bugs.python.org/
Charles-François Natali added the comment:
Vajrasky, thanks for the patch!
--
resolution: -> fixed
stage: patch review -> committed/rejected
status: open -> closed
___
Python tracker
<http://bugs.python.or
Charles-François Natali added the comment:
Thanks for the feedback!
--
resolution: -> invalid
stage: -> committed/rejected
status: open -> closed
___
Python tracker
<http://bugs.python.or
Charles-François Natali added the comment:
> It sounds like doing this is fine (as Glyph suggests in the email thread) but
> it isn't magically going to solve all EINTR problems, just reduce the number
> of calls that could encounter them.
Indeed, hence "*limit* EINTR occur
New submission from Charles-François Natali:
http://buildbot.python.org/all/builders/AMD64%20FreeBSD%209.0%203.x/builds/5901/steps/test/logs/stdio
"""
[134/387/1] test_signal
Timeout (1:00:00)!
Thread 0x000801e24800 (most recent call first):
File
"/usr/home/buildbo
Charles-François Natali added the comment:
> Guido van Rossum added the comment:
>
> That's just rhetoric -- I am beginning to believe that nobody has any data.
> Here's some opposite rhetoric. If it ain't broke, don't fix it. Plus, if
> it's so mu
Charles-François Natali added the comment:
This test is inherently subject to a race condition:
"""
port = support.find_unused_port()
[...]
try:
self.assertRaises(OverflowError, sock.bind, (host, big_port))
self.assertRaise
Charles-François Natali added the comment:
Well, unregister() documentation currently contains this:
"""
.. method:: unregister(fileobj)
Unregister a file object from selection, removing it from monitoring. A
file object shall be unregistered prior to being close
Charles-François Natali added the comment:
> The patch is fine, but it is hard to rely on it to prevent bugs from
> happening because that requires cooperation from all modules registering
> signal handlers.
Once again, that's why the bug report says "*limit* EINTR occurr
Charles-François Natali added the comment:
> Guido van Rossum added the comment:
>
> I think you're looking for the discussion in issue 19017.
>
> IIRC the conclusion is that not only do you not get the same error
> everywhere, but you get it at different points -- so
Charles-François Natali added the comment:
> STINNER Victor added the comment:
>
> Kristján> Only that issue #10517 mentions reasons to keep the old behavior,
> specifically http://bugs.python.org/issue10517#msg134573 (...)
>
> @Kristján: The behaviour of PyThread_set_ke
Charles-François Natali added the comment:
I agree with Antoine.
Silencing warning is fine, as long as it's not to the detriment of
clarity (see Debian Openssl's vulnerability extreme example).
--
___
Python tracker
<http://bu
Charles-François Natali added the comment:
Thanks!
--
resolution: -> fixed
stage: patch review -> committed/rejected
status: open -> closed
___
Python tracker
<http://bugs.python.or
Charles-François Natali added the comment:
Sorry for the delay, I didn't have any free time this week.
I'll review the patch shortly, but the idea sounds fine (I just need
to check if we can't be a little more specific for errnos
Charles-François Natali added the comment:
> Guido van Rossum added the comment:
>
> Here's an attempt at fixing the ValueError.
>
> I don't like the exhaustive search much, but the alternative is to maintain
> an inverse dict. What do you think?
I was going t
Charles-François Natali added the comment:
LGTM!
--
___
Python tracker
<http://bugs.python.org/issue19876>
___
___
Python-bugs-list mailing list
Unsubscribe:
Changes by Charles-François Natali :
--
resolution: -> fixed
stage: needs patch -> committed/rejected
status: open -> closed
___
Python tracker
<http://bugs.python.or
Charles-François Natali added the comment:
The test is failing on Windows buildbot:
http://buildbot.python.org/all/builders/x86%20Windows%20Server%202003%20%5BSB%5D%203.x/builds/1851/steps/test/logs/stdio
"""
=
New submission from Charles-François Natali:
This is a spinoff of issue #19506: currently, subprocess.communicate() uses a
4K buffer when reading data from pipes.
This was probably optimal a couple years ago, but nowadays most operating
systems have larger pipes (e.g. Linux has 64K), so we
Charles-François Natali added the comment:
> STINNER Victor added the comment:
>
> I don't like generic "except OSError: pass". Here is a first patch for
> epoll() to use "except FileNotFoundError: pass" instead. Kqueue selector
> should also be patch
Charles-François Natali added the comment:
> STINNER Victor added the comment:
>
> Since Popen.communicate() returns the whole content of the buffer, would it
> be safe to increase the buffer size? For example, use 4 GB as the buffer size?
Sure, if you want to pay the CPU and mem
Charles-François Natali added the comment:
Looks like a kernel bug.
errno 512 is ERESTARTSYS, which shouldn't leak to user-mode.
--
___
Python tracker
<http://bugs.python.org/is
Charles-François Natali added the comment:
> Roundup Robot added the comment:
>
> New changeset 03a056c3b88e by Gregory P. Smith in branch '3.3':
> Fixes issue #19929: Call os.read with 32768 within subprocess.Popen
> http://hg.python.org/cpython/rev/03a056c3b88e
No
Charles-François Natali added the comment:
> I will look into the Windows problem, but I suspect the best we can do there
> is skip the test.
I already took care of that:
http://hg.python.org/cpython/rev/01676a4c16ff
--
___
Python tracker
Charles-François Natali added the comment:
Berker, I've committed your patch, thanks!
--
nosy: +neologix
___
Python tracker
<http://bugs.python.org/is
Charles-François Natali added the comment:
Committed, thanks!
--
resolution: -> fixed
stage: patch review -> resolved
status: open -> closed
type: resource usage -> behavior
versions: +Python 3.3, Python 3.4
___
Python tr
Charles-François Natali added the comment:
> If I'm receiving data from a socket (several bytes) and making the
> first call to socket.recv(1) all is fine but the second call won't get
> any further data. But doing this again with socket.recv(2) instead will
> successfully
Charles-François Natali added the comment:
> I'm wondering how would it be possible then to fetch packets of an unknown
> size without using an extremely big buffer.
IP packets are limited to 64K, so just pass a 64K buffer, that's not
"extremely big".
If you really
Charles-François Natali added the comment:
There's probably a special mechanism due to vserver which makes the
kernel kill the process instead of failing with EPERM, but it's really
surprising.
What happens if you try the following:
$ python -c "from resource import *; _, h
Charles-François Natali added the comment:
LGTM.
--
___
Python tracker
<http://bugs.python.org/issue21090>
___
___
Python-bugs-list mailing list
Unsubscribe:
Changes by Charles-François Natali :
--
resolution: -> out of date
stage: -> resolved
status: open -> closed
___
Python tracker
<http://bugs.python.or
Charles-François Natali added the comment:
> So do we agree that the resolution for this is "wont fix"?
Yes.
We don't want to backport this (we've had enough regressions already, and
people have been living withut loc interruptibility for a long time before
Python 3).
Charles-François Natali added the comment:
> Tin Tvrtković added the comment:
>
> For your examples, my first instinct would be to use a thread pool executor.
> It's a nice high level API and can already do the aggregate join.
Indeed, the examples posted don't make much
Charles-François Natali added the comment:
>> rdmurray@pydev:~/python/p34>python -c 'import resource;
>> print(resource.getrlimit(resource.RLIMIT_NOFILE))'
>> (1024L, 1048576L)
>
> Oh, 1 million files is much bigger than 4 thousand files (4096).
>
>
Charles-François Natali added the comment:
Alright, I'll cap the value then (no need to expose FD_SETSIZE).
--
___
Python tracker
<http://bugs.python.org/is
Charles-François Natali added the comment:
On Unix at least, this doesn't apply: rm -f doesn't mean "ignore permissions",
but but rather don't ask confirmation which the rm commands asks in some cases
(empty file, directory, etc).
Ans the code posted wouldn't
Changes by Charles-François Natali :
--
resolution: -> duplicate
stage: -> resolved
status: open -> closed
superseder: -> handle EINTR in the stdlib
___
Python tracker
<http://bugs.python
Changes by Charles-François Natali :
--
resolution: -> duplicate
stage: needs patch -> resolved
status: open -> closed
superseder: -> handle EINTR in the stdlib
___
Python tracker
<http://bugs.python
Charles-François Natali added the comment:
> Actually it does apply on unix:
No, it does not apply: here's what I've written:
"""
rm -f doesn't mean "ignore permissions", but but rather don't ask
confirmation which the rm commands asks in some c
Charles-François Natali added the comment:
Thanks for the patch, but I'm not even sure AIX is an officially supported
platform, so I'm not sure what to do with this patch.
--
nosy: +neologix
___
Python tracker
<http://bugs.python.o
Charles-François Natali added the comment:
Sorry for the delay, should be fixed now.
--
resolution: -> fixed
stage: -> resolved
status: open -> closed
___
Python tracker
<http://bugs.python.or
Charles-François Natali added the comment:
Jerrfey, thanks for your patch, but apparently in the - long - meantime, the
declaration has already been moved up.
--
nosy: +neologix
resolution: -> out of date
stage: -> resolved
status: open -&g
Charles-François Natali added the comment:
> Jeffrey Armstrong added the comment:
>
> Next time I see a bug I'll be sure to wait rather than actually submit a
> patch...
I understand your frustration, but please don't do that: you have to
understand that we don't hav
Charles-François Natali added the comment:
> In other words, on unix shutil.rmtree is *already* 'rm -rf'.
Exactly :-)
--
___
Python tracker
<http://bugs.pytho
Charles-François Natali added the comment:
> Huh? What does officially supported platform mean? CPython builds and runs
> on AIX.
It means a platform for which we have regular contributors committed
to support the port, and ideallly with a stable buildbot.
We apparently have an un
Charles-François Natali added the comment:
Committed, thanks for the reviews (I only updated one call to socket.listen(),
inside multiprocessing, the rest is in the test suite).
--
resolution: -> fixed
stage: patch review -> resolved
status: open -&g
Charles-François Natali added the comment:
> There is a question. What should pathlib's expanduser() do in case
> when user directory can't be determined (or user does not exist)?
> Perhaps unlike to os.path.expanduser() it should raise an exception
> (as in many other path
Charles-François Natali added the comment:
Barring any objection, I'll commit the patch attached within a couple days.
--
title: implement socketpair() on Windows -> add a fallback socketpair()
implementation in test.support
Added file: http://bugs.python.org/file36050/sock
Charles-François Natali added the comment:
> You should copy the code from asyncio.windows_utils.socketpair(). It checks
> also type and proto parameter: raise a ValueError for unsupported values
> (only SOCK_STREAM and proto=0 are supported). It also supports IPv6. It
Charles-François Natali added the comment:
> By the way, we should reuse socket.socketpair() in
> asyncio.windows_utils. The Tulip is written for Python 3.3 and shares
> exactly the same code base, so you should write
>
> Something like:
>
> if hasattr(socket, 'sock
Charles-François Natali added the comment:
Here's a patch.
--
keywords: +patch
Added file: http://bugs.python.org/file36070/find_unused_race.diff
___
Python tracker
<http://bugs.python.org/issue19875>
___
Charles-François Natali added the comment:
As I said offline to Victor, I think it would be better to have a single
function, i.e. keep set_wakeup_fd(). It makes the API simpler, less confusing
and error prone: some people will wonder which one they should use, might end
up using the wrong
Charles-François Natali added the comment:
The patch looks good.
I just think it would be nice to expose _fdcount() in test.support.
--
nosy: +neologix
___
Python tracker
<http://bugs.python.org/issue18
Charles-François Natali added the comment:
I'm closing, since it's amlost certainly an OS-X bug (similar to time-wit
socket exhaustion & Co).
--
resolution: -> third party
stage: -> resolved
status: open -> closed
__
Charles-François Natali added the comment:
Closing as wont't fix, since we now have asyncio which handles this much better.
--
resolution: -> wont fix
stage: needs patch -> resolved
status: open -> closed
___
Python tracker
<http:
1401 - 1500 of 1826 matches
Mail list logo
