Charles-François Natali added the comment:
Christian, could you post it as a mercurial diff for review?
Also, it would maybe be better to rename this issue to "rewrite stat module in
C".
Shouldn't we also remove the Python version?
--
New submission from Charles-François Natali:
In many cases, PyModule_AddIntMacro() could be used instead of
PyModule_AddIntConstant(), e.g. in socketmodule.c and posixmodule.c:
PyModule_AddIntMacro(m, AF_INET6);
vs (currently)
PyModule_AddIntConstant(m, "AF_INET6", AF_INET6);
It r
Charles-François Natali added the comment:
> We've been *adding* python implementations for other modules, so I don't see
> why we would remove this one.
Because it's buggy, and cannot be implemented correctly in python.
--
_
Charles-François Natali added the comment:
test_dis is failing on some buildbots:
http://buildbot.python.org/all/builders/AMD64 Ubuntu LTS
3.x/builds/1674/steps/test/logs/stdio
Re-running test 'test_dis' in verbose mode
test test_dis crashed -- Traceback (most recent call last):
Charles-François Natali added the comment:
Sorry for the delay, it should be fixed now.
Federico, thanks for the patch!
--
resolution: -> fixed
stage: patch review -> committed/rejected
status: open -> closed
___
Python track
Charles-François Natali added the comment:
Here's a (gigantic) patch.
I used an ad-hoc script for the conversion (next time I might try with
coccinelle).
I tested it on Linux, FreeBSD, Openindiana, OS-X and Windows.
--
keywords: +needs review, patch
nosy: +pitrou
stage: needs
Changes by Charles-François Natali :
Added file: http://bugs.python.org/file30155/ins_macro.diff
___
Python tracker
<http://bugs.python.org/issue17917>
___
___
Python-bug
Charles-François Natali added the comment:
Are there other places in the code base using linked-list?
If yes, I think it could be interesting to add a generic linked-list
implementation, like the one used in the Linux kernel:
http://kernelnewbies.org/FAQ/LinkedLists
http://isis.poly.edu/kulesh
Charles-François Natali added the comment:
The test is still failling:
http://buildbot.python.org/all/builders/AMD64 OpenIndiana
2.7/builds/1670/steps/test/logs/stdio
"""
==
ERROR: test_extract_un
Changes by Charles-François Natali :
Removed file: http://bugs.python.org/file30155/ins_macro.diff
___
Python tracker
<http://bugs.python.org/issue17917>
___
___
Pytho
Changes by Charles-François Natali :
Added file: http://bugs.python.org/file30157/ins_macro-1.diff
___
Python tracker
<http://bugs.python.org/issue17917>
___
___
Pytho
Charles-François Natali added the comment:
> I also vote +1 for returning None when the information is unknown.
I still don't like it.
If a function returns a number of CPU, it should either return an
integer >= 1, or raise an exception.
None is *not* an integer.
And returning an e
Charles-François Natali added the comment:
> PC/_msi.c: Oh, here you should remove cast to int. Example:
>
> PyModule_AddIntMacro(m, (int)MSIDBOPEN_CREATEDIRECT);
>
> I'm surprised that the does compile. You may have a
> "(int)MSIDBOPEN_CREATEDIRECT" variable :
Charles-François Natali added the comment:
Fair enough, I guess we should use it then.
We just have to agree on the value to return when the number of CPUs
can't be determined ;-)
--
___
Python tracker
<http://bugs.python.org/is
Charles-François Natali added the comment:
> There are a couple other places, IIRC. That said, I'm not sure what the
> point is, since a linked list is quite a simple structure anyway?
Well, it was just to avoid code duplication, and gain a nice iteration macro ;-)
Anyway, I'l
Changes by Charles-François Natali :
Added file: http://bugs.python.org/file30167/ins_macro-2.diff
___
Python tracker
<http://bugs.python.org/issue17917>
___
___
Pytho
Changes by Charles-François Natali :
--
keywords: +needs review, patch
stage: needs patch -> patch review
Added file: http://bugs.python.org/file30168/pystate.diff
___
Python tracker
<http://bugs.python.org/issu
Charles-François Natali added the comment:
IMO, this shouldn't be implemented atop thread, but ought to be a regular
thread pool: this way, you won't get behind if some task takes too long to
execute, the thread pool can start new threads as needed, and we get the
general work sub
Changes by Charles-François Natali :
--
resolution: -> fixed
stage: patch review -> committed/rejected
status: open -> closed
___
Python tracker
<http://bugs.python.or
Charles-François Natali added the comment:
I'm attaching a proof of concept code for a ScheduledExecutor
interface, and a ScheduledThreadPoolExecutor implementation
(unfortunately I can't upload it as a mercurial diff for now).
Here's what the API looks like:
"""
Charles-François Natali added the comment:
> Based on the conversation and the particular inputs to the thread form
> neologix and ezio, I would like to submit this patch.
>
> It probably needs modification(s) as I am not sure what to do with the
> implementation that is alr
Charles-François Natali added the comment:
> I think the idiom `os.cpu_count() or 1` should be mentioned in the
> documentation an officially recommended. Otherwise people will produce a
> non-portable code which works on their developer's computers but not on
> exotic
New submission from Charles-François Natali:
Here's an implementation of a new ScheduledExecutor abstract class, with a
concrete ScheduledThreadPoolExecutor implementation (see #995907).
The aim is to provide a flexible, efficient and consistent framework for
delayed and periodic
Charles-François Natali added the comment:
OK, I just created #17956 for ScheduledExecutor, closing this one.
--
resolution: -> duplicate
stage: test needed -> committed/rejected
status: open -> closed
superseder: -> add Sched
Changes by Charles-François Natali :
Added file: http://bugs.python.org/file30222/scheduled-2.diff
___
Python tracker
<http://bugs.python.org/issue17956>
___
___
Pytho
Charles-François Natali added the comment:
> Returning None from C code sounds reasonable to me. Anyone else wants to
> pitch in with suggestions for/against this?
Go for it ;-)
--
___
Python tracker
<http://bugs.python.org/i
Charles-François Natali added the comment:
> It will be good to be compatible with sched.scheduler.
What do you mean? Actually, this would kind of supersede the sched
module (except that the sched module supports custom time and delay
functions).
By the way, for those that didn't re
Charles-François Natali added the comment:
> 1. Extends an abstract interface to support of a priority
I'm not sure I see the use case for priority support, do you have a
sample use case?
Furthermore, the executor is backed by a thread pool, so tasks can be
run concurrently.
Fina
Charles-François Natali added the comment:
I'd like to have your opinion regarding some implementation choices I made.
1) There's a new ScheduledFuture deriving from Future, which adds
methods to check if the task is periodic, to get the delay until the
next scheduled execution,
Charles-François Natali added the comment:
> Here is a new patch (cpu_count.patch) with a different approach:
>
> * add a os.cpu_count() which returns the raw value (may be zero or negative
> if the OS returns a dummy value) and raise an OSError on error
> * os.cpu_count() is n
Charles-François Natali added the comment:
> To be honest I can't find much to say about this proposal,
Hum, OK, I thought it would be a useful addition :-)
> but I think it would be good if the time function were configurable (both for
> test purposes, and to all
Charles-François Natali added the comment:
> Also, even though an IO call _can_ block, that doesn't mean that
> we _must_ release the gil for the duration.
Yes, otherwise some people will complain when the whole interpreter is stuck
while a socket/NFS file handle/whatever is shut
Charles-François Natali added the comment:
Just for giggles, here's the glibc default implementation on non Linux
platforms:
http://sourceware.org/git/?p=glibc.git;a=blob;f=misc/getsysstats.c;hb=HEAD
"""
int
__get_nprocs ()
{
/* We don't know how to determine the numb
Charles-François Natali added the comment:
> Python's goal is not to emulate the suboptimal parts of other languages.
Well, I'm sure they could have returned -1 or 0, which are valid C
long distinct from any valid integer representing a number of CPUs. If
the libc guys (and many o
Charles-François Natali added the comment:
> Well, they can be wrong sometimes, too :-)
Indeed, as can I ;-)
> The patch doesn't seem to rely on the glibc, so we are fine here.
> Or do the other libs work likewise?
sysconf(_SC_NPROCESSORS_CONF) is implemented with the above f
Charles-François Natali added the comment:
> Catching regressions is what we have the regression tests for. If it is not
> in caught by the regression tests, then it is not a regression, by definition.
You do realize this sentence doesn't make se
Charles-François Natali added the comment:
> I assume my glibc and fwrite aren't broken though
Actually, it's a glibc bug when the last character is a '\n':
$ python -c "f = open('/dev/full', 'w', 1); f.write('hello'); f.close()
Charles-François Natali added the comment:
> Indeed, fwrite() can return expected number of items and set errno. Here is a
> simple example on C. An output is:
Yeah, who's volunteering to report it to the glibc?
That's not a python bug, but I feel bad ignoring it.
Note tha
Charles-François Natali added the comment:
> Why is ferror() not reliable?
Because the glibc doesn't check the errno return code after the
write() syscall, and thus doesn't set the file's stream error flag
(ferror() just checks this flag).
That's what I saw from th
Charles-François Natali added the comment:
> Strange. I too modified Serchiy's code and my version of glibc (2.15) set the
> error flag at the same fwrite call as errno was set:
>
> setvbuf 0 0 0
> fwrite 5 0 0
> fwrite 1 28 1
> fwrite 1 28 1
>
> (the last
Charles-François Natali added the comment:
> (Note that the Beaker version would need to be enhanced with the extra API
> parameters from Victor's version, as well as updated to use the exclusive
> open and close-on-exec flags)
I think the API would be nicer if it was just a
Charles-François Natali added the comment:
Could you provide the output of:
strace -ttT -f ./python Lib/test/test_asynchat.py
--
nosy: +neologix
___
Python tracker
<http://bugs.python.org/issue17
Charles-François Natali added the comment:
That's what I thought:
08:01:24.824406 bind(3, {sa_family=AF_INET, sin_port=htons(0),
sin_addr=inet_addr("127.0.0.1")}, 16) = 0 <0.24>
[pid 1698] 08:01:24.825502 listen(3, 1) = 0 <0.35>
[pid 1698] 08:01:24.825786
Charles-François Natali added the comment:
> What ports are needed?
Many tests use random ephemeral ports on the loopback interface (e.g. 43785
above).
You should update your rules to apply to external NIC, not on the loopback.
--
resolution: -> invalid
stage: -> committed
Charles-François Natali added the comment:
It's a dupe of #15500.
--
nosy: +neologix
resolution: -> duplicate
stage: -> committed/rejected
status: open -> closed
superseder: -> Python should support naming threads
___
Pyth
Charles-François Natali added the comment:
Well, this just needs a patch :-)
--
___
Python tracker
<http://bugs.python.org/issue17996>
___
___
Python-bugs-list m
Charles-François Natali added the comment:
Alright, committed.
Yogesh, thanks for the patch!
I'm attaching a patch to replace several occurrences of
multiprocessing.cpu_count() by os.cpu_count() in the stdlib/test
suite.
--
Added file: http://bugs.python.org/file
Charles-François Natali added the comment:
> * rewrite Mac OS X implementation: code in 5e0c56557390 looks wrong. It
> gets a MIB but then don't use it when calling _bsd_cpu_count(). But I didn't
> check my patch nor the commit version on Mac OS X.
Indeed.
I just removed th
Charles-François Natali added the comment:
Committed, thanks!
--
resolution: -> fixed
stage: -> committed/rejected
___
Python tracker
<http://bugs.python.org/i
Changes by Charles-François Natali :
--
status: open -> closed
___
Python tracker
<http://bugs.python.org/issue17684>
___
___
Python-bugs-list mailing list
Un
Charles-François Natali added the comment:
Here's a patch.
Note that I had to adapt test_socket (which was passing bytes).
--
components: +Library (Lib)
keywords: +needs review
stage: -> patch review
type: -> behavior
Added file: http://bugs.python.org/file30321/af_un
Charles-François Natali added the comment:
This has been fixed some time ago (I don't remember the commit though).
--
resolution: -> fixed
stage: -> committed/rejected
status: open -> closed
___
Python tracker
<http://bugs.pyth
Changes by Charles-François Natali :
--
resolution: -> invalid
stage: -> committed/rejected
status: pending -> closed
___
Python tracker
<http://bugs.python.or
Changes by Charles-François Natali :
--
resolution: -> invalid
stage: -> committed/rejected
status: pending -> closed
___
Python tracker
<http://bugs.python.or
Charles-François Natali added the comment:
It's fixed now that subprocess defaults to close_fds=True.
--
resolution: -> out of date
stage: -> committed/rejected
status: open -> closed
___
Python tracker
<http://bugs.pyt
Charles-François Natali added the comment:
IIRC, os.rename() will fail on Windows if the target file already exists.
That's why os.replace() was added.
--
___
Python tracker
<http://bugs.python.org/is
Charles-François Natali added the comment:
The workaround would be to unlink the file first, and then try to
create it with O_EXCL. You have a short window where there's no file,
but that shouldn't be a problem in this specific case, and it would
work on Windows.
As for issue #17222,
Charles-François Natali added the comment:
Antoine, I need your help :-)
http://buildbot.python.org/all/builders/x86 Gentoo Non-Debug
3.x/builds/4311/steps/test/logs/stdio
"""
==
ERROR: testLinuxAb
Charles-François Natali added the comment:
Hum, IIUC, basically what happens is that the user could - and still
can - pass arbitrary bytes as address (which is legtit), but depending
on the encoding, getsockaddr() and friends might blow up when decoding
it.
If that's correct, that'
Changes by Charles-François Natali :
--
resolution: -> fixed
stage: patch review -> committed/rejected
status: open -> closed
___
Python tracker
<http://bugs.python.or
Changes by Charles-François Natali :
--
stage: needs patch -> patch review
___
Python tracker
<http://bugs.python.org/issue17914>
___
___
Python-bugs-list mai
Charles-François Natali added the comment:
> I don't understand "This is a lot of code churn, but it touches code that is
> unlikely to be modified otherwise, so I guess it's ok.". What does it mean
> it's okay when it touches on code that's unl
Charles-François Natali added the comment:
> Nevermind, it seems changing regen is not necessary. Patch is in attachment.
Go ahead!
--
___
Python tracker
<http://bugs.python.org/issu
Charles-François Natali added the comment:
Martin, do you think your latest patch can be committed?
--
___
Python tracker
<http://bugs.python.org/issue13
Charles-François Natali added the comment:
I can't speak for Antoine, but I guess that the result of pybench
would be enough to make sure it doesn't introduce any regression
(which would be *really* suprising).
As for the memory savings, the benchmark you posted earlier is
conclusive
Charles-François Natali added the comment:
I'm not familiar with windows, but if I understand the patch
correctly, you can only select from a single subprocess at a time,
which is IMO an important limitation.
Also, the fact that close() can fail with BlockingIOError is really a
pain, and
Charles-François Natali added the comment:
> Shouldn't the surrogateescape error handler (PEP 383) prevent this?
Yes, it shoud (I just read PEP 383, I told you I didn't know anything
about encoding :-).
So basically, for the test failure, the issue is simply that the
platf
Charles-François Natali added the comment:
My only comment would be to use subprocess instead of os.popen().
--
___
Python tracker
<http://bugs.python.org/issue17
Charles-François Natali added the comment:
Why is that a different issue?
The code you *add in this patch* uses os.popen, why not use subprocess instead?
Furthermore, the code catches OSError when calling popen(), but
popen() doesn't raise an exce
Charles-François Natali added the comment:
Annoying.
I thought CAN_RAW_FD_FRAME would be a macro, which would have made conditional
compilation easy, but it's apparently a enum value, which means we have to add
a configure-time check...
--
components: +Library (Lib
Changes by Charles-François Natali :
--
resolution: -> fixed
stage: patch review -> resolved
status: open -> closed
title: add a fallback socketpair() implementation in test.support -> add a
fallback socketpair() implementation to the s
Charles-François Natali added the comment:
Adding ioctl constants is fine.
However, I think that if we do this, it'd be great if we could also
expose this information in a module (since psutil inclusion was
discussed recently), but that's probably ano
Charles-François Natali added the comment:
> Maries Ionel Cristian added the comment:
>
> Serhiy, I don't think this is a duplicate. Odd that you closed this without
> any explanation.
>
> This happens in a internal lock in cpython's runtime, while the other bug i
Charles-François Natali added the comment:
> Serhiy, I believe this still happens in Python 3.4, but it is harder to
> reproduce. I couldn't get Armin's script to produce the problem either, but
> I'm pretty sure that this is what causes e.g.
> https://bugs.debia
Charles-François Natali added the comment:
Thanks for taking care of this.
--
___
Python tracker
<http://bugs.python.org/issue23009>
___
___
Python-bugs-list m
Charles-François Natali added the comment:
RuntimeError sounds better to me (raising ValueError when no value is
provided, e.g. in select() sounds definitely strange).
--
___
Python tracker
<http://bugs.python.org/issue23
Changes by Charles-François Natali :
--
components: Library (Lib)
hgrepos: 293
nosy: haypo, neologix, pitrou
priority: normal
severity: normal
status: open
title: PEP 475 - EINTR hanndling
type: enhancement
___
Python tracker
<http://bugs.python.
New submission from Charles-François Natali:
The test runs fine on Linux, but hangs in test_send() on OS-X and *BSD.
I don't know what's wrong, so if someone with access to one of these OS could
have a look, it would be great.
--
___
Pyth
Changes by Charles-François Natali :
--
keywords: +patch
Added file: http://bugs.python.org/file37797/ff1274594739.diff
___
Python tracker
<http://bugs.python.org/issue23
Changes by Charles-François Natali :
--
title: PEP 475 - EINTR hanndling -> PEP 475 - EINTR handling
___
Python tracker
<http://bugs.python.org/issu
Charles-François Natali added the comment:
> The review diff is weird: it seems it contains changes that aren't
> EINTR-related (see e.g. argparse.rst).
Here's a manually generated diff.
--
Added file: http://bugs.python.org/file
Charles-François Natali added the comment:
Or we should acknowledge that this is overkill, and take the same approach as
all major web browser: disable the Nagle algorithm.
For a protocol like http which is transaction oriented it's probably the best
thing to do.
--
nosy: +neo
Charles-François Natali added the comment:
Interestingly, there is no close() method on SimpleQueue...
--
nosy: +neologix
___
Python tracker
<http://bugs.python.org/issue23
Charles-François Natali added the comment:
> I added a few prints to the send and receive loops of _test_send. When
> running on a reasonably current Debian testing Linux:
Thanks, that's what I was suspecting, but I really don't understand
why 200ms isn't enough for a soc
Charles-François Natali added the comment:
> It turns out the times are not important; the hangup is the default size of
> the socket buffers on OS X and possibly BSD in general. In my case, the send
> and receive buffers are 8192, which explains why the chunks written are so
>
Charles-François Natali added the comment:
> Charles-François Natali added the comment:
>
> Hmmm...
> Basically, with a much smaller socket buffer, we get much more context
> switches, which increases drastically the test runtime.
> But I must admit I'm still really s
Charles-François Natali added the comment:
OK, it turns out the culprit was repeated calls to BytesIO.getvalue(),
which forced large allocations upon reception of every message.
The patch attached fixes this (without changing the socket buffer size).
--
Added file: http
Charles-François Natali added the comment:
> eintr-1.diff doesn't seem to make any significant difference from eintr.diff
> on my system. It's still pegging a CPU at 100% and takes 7 minutes wall time
> to complete.
Alright, enough played: the patch attached us
Charles-François Natali added the comment:
> The way socket timeouts are implemented is by using select() to determine
> whether the socket is ready for read/write. In this case, select() probably
> marks the socket ready even though the queue is full, which later raises
> EAG
Charles-François Natali added the comment:
> With eintr-2.diff, fast!:
Victory \°/.
> Instrumented test_send, 3 socket.send calls, many socket.recv_into calls:
Yep, that's expected.
I think we should keep the default socket buffer size: it increases
the test coverage, and it
Charles-François Natali added the comment:
I just realized I didn't retry upon EINTR for open(): eintr-4.diff
adds this, along with tests (using a fifo).
Also, I added comments explaining why we don't retry upon close() (see
e.g. http://lwn.net/Articles/576478/ and
http://linux.der
Changes by Charles-François Natali :
--
status: open -> pending
___
Python tracker
<http://bugs.python.org/issue23351>
___
___
Python-bugs-list mailing list
Un
Charles-François Natali added the comment:
Well, I'd like to see at least one benchmark.
--
___
Python tracker
<http://bugs.python.org/issue18932>
___
___
Charles-François Natali added the comment:
It's a kernel bug closing (working fine on my Debian wheezy with a more recent
kernel BTW).
--
resolution: -> third party
status: open -> closed
___
Python tracker
<http://bugs.python.
Charles-François Natali added the comment:
> Antoine Pitrou added the comment:
>
>> Would it be possible to push the latest patch right now
>
> It's ok for me. Please watch the buildbots :)
Cool, I'll push o
Charles-François Natali added the comment:
Does anyone have a realistic use case where modify() is actually a
non-negligible part?
--
___
Python tracker
<http://bugs.python.org/issue18
Charles-François Natali added the comment:
> @neologix: Would you be ok to add a *private* _at_fork() method to selectors
> classes in Python 3.4 to fix this issue?
Not really: after fork(), you're hosed anyway:
"""
Q6 Will closing a file descriptor cause
New submission from Charles-François Natali:
The following segfaults in _PyObject_GenericGetAttrWithDict:
"""
from socket import socketpair
from _multiprocessing import Connection
class Crash(Connection):
pass
_, w = socketpair()
Crash(w.fileno
Charles-François Natali added the comment:
Here's a patch against 2.7 using _PyOS_URandom(): it should apply as-is to 3.3.
--
keywords: +patch
nosy: +neologix
versions: +Python 3.3
Added file: http://bugs.python.org/file39679/mp_sem_race
Changes by Charles-François Natali :
--
keywords: +needs review
nosy: +haypo
___
Python tracker
<http://bugs.python.org/issue24303>
___
___
Python-bugs-list mailin
1701 - 1800 of 1826 matches
Mail list logo
