Re: [Python-Dev] Parallel test execution on buildbot
On 09/05/2010 03:32, Nick Coghlan wrote: Martin v. Löwis wrote: Has anyone considered using regrtest's -j option in the buildbot configuration to speed up the test runs? Yes, I did. I turned it off again when the tests started failing because of it. Yeah, a lot of our tests weren't written with parallel execution in mind (e.g. the existence of test_support.TESTFN, using specific ports for test servers). While they've been getting better (e.g. increased use of randomly generated temporary directories over specific filenames, letting the OS assign a free port to servers), I believe there is still a fair bit of work to be done to make them all "parallel execution friendly" (of course, some tests, such as those that try to trigger MemoryError, are inherently parallel execution unfriendly). So yes, we've thought about it, but there's still work to be done before that option can be used without having to worry about false alarms. FWIW I *usually* run the test suite with parallelization (it is just so much quicker) and these days *rarely* see spurious failures as a result. This is on Mac OS X, YMMV. Michael Foord Cheers, Nick. -- http://www.ironpythoninaction.com/ ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Parallel test execution on buildbot
Le dimanche 09 mai 2010 11:08:29, Michael Foord a écrit : > FWIW I *usually* run the test suite with parallelization (it is just so > much quicker) and these days *rarely* see spurious failures as a result. > This is on Mac OS X, YMMV. I use regrtest.py -j 4 on a Intel Quad Core on Linux: 4 tests are really running at the same time, and I only get only one spurious failure: test_ioctl. It should be easy to fix this test. Or we can maybe write a blacklist of tests not compatible with multiprocessing mode (or the opposite, a whitelist of compatible tests). -- Victor Stinner http://www.haypocalc.com/ ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Parallel test execution on buildbot
> FWIW I *usually* run the test suite with parallelization (it is just so > much quicker) and these days *rarely* see spurious failures as a result. > This is on Mac OS X, YMMV. I may misremember the details, but IIRC, the multiprocessing tests would fail to terminate on Solaris. This made it unsuitable for buildbot usage. Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] urlparse.urlunsplit should be smarter about +
John Arbash Meinel writes:
> Stephen J. Turnbull wrote:
> > David Abrahams writes:
> > >
> > > This is a bug report. bugs.python.org seems to be down.
> > >
> > > >>> from urlparse import *
> > > >>> urlunsplit(urlsplit('git+file:///foo/bar/baz'))
> > > git+file:/foo/bar/baz
> > >
> > > Note the dropped slashes after the colon.
> >
> > That's clearly wrong, but what does "+" have to to do with it? AFAIK,
> > the only thing special about + in scheme names is that it's not
> > allowed as the first character.
>
> Don't you need to register the "git+file:///" url for urlparse to
> properly split it?
>
> if protocol not in urlparse.uses_netloc:
> urlparse.uses_netloc.append(protocol)
I don't know about the urlparse implementation, but from the point of
view of the RFC I think not. Either BCP 35 or RFC 3986 (or maybe both)
makes it plain that if the scheme name is followed by "://", the
scheme is a hierarchical one. So that URL should parse with an empty
authority, and be recomposed the same. I would do this by parsing
'git+file:///foo/bar/baz' to ('git+file', '', '/foo/bar/baz') or
something like than, and 'git+file:/foo/bar/baz' to ('git+file', None,
'/foo/bar/baz').
I don't see any reason why implementations should abbreviate the empty
authority by removing the double slashes, unless specified in the
scheme definition. Although my reading of RFC 3986 is that a missing
authority (no "//") *should* be dereferenced in the same way as an
empty one:
If the URI scheme defines a default for host, then that default
applies when the host subcomponent is undefined or when the
registered name is empty (zero length). (Sec. 3.2.2)
I don't see why urlparse should try to enforce that by converting from
one to the other.
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Did I miss the decision to untabify all of the C code?
On Thu, May 6, 2010 at 4:52 AM, Joao S. O. Bueno wrote: > On Wed, May 5, 2010 at 9:59 PM, Eric Smith wrote: >> That's my point. Since it's basically unreviewable, is it smart to do it >> during a beta? > > Hello folks - > I don't think these modifications are that "unreviewable": the > generated binaries have to be exactly the same with the untabified > files don't they? So is a matter of stashing the binaries, applying > the patches, rebuild and check to see if the binaries match. Any > possible script defects undetected by this would be only (C code) > indentation, which could be fixed later. That's not foolproof, though: there are lots of sections of code that will only get compiled on certain platforms, or with certain configure options, etc. Mark ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] PEP 7 updated
Hello, The untabification of C files didn't produce any noticeable problem on the buildbots. I've updated PEP 7 with the mention that all C files should be 4-space indented, and removed the obsolete wording about some files being indented with tabs. Regards Antoine. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] [RELEASED] Python 2.7 beta 2
* Benjamin Peterson: > http://doc.python.org/dev/whatsnew/2.7.html or Misc/NEWS in the Python > distribution. Something is missing here: "* Multiple context managers in" ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] urlparse.urlunsplit should be smarter about +
At Sat, 08 May 2010 11:04:47 -0500,
John Arbash Meinel wrote:
>
> Stephen J. Turnbull wrote:
> > David Abrahams writes:
> > >
> > > This is a bug report. bugs.python.org seems to be down.
> > >
> > > >>> from urlparse import *
> > > >>> urlunsplit(urlsplit('git+file:///foo/bar/baz'))
> > > git+file:/foo/bar/baz
> > >
> > > Note the dropped slashes after the colon.
> >
> > That's clearly wrong, but what does "+" have to to do with it? AFAIK,
> > the only thing special about + in scheme names is that it's not
> > allowed as the first character.
>
> Don't you need to register the "git+file:///" url for urlparse to
> properly split it?
Yes. But the question is whether urlparse should really be so fragile
that every hierarchical scheme needs to be explicitly registered.
Surely ending with “+file” should be sufficient to have it recognized
as a file-based scheme
--
Dave Abrahams Meet me at BoostCon: http://www.boostcon.com
BoostPro Computing
http://www.boostpro.com
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] urlparse.urlunsplit should be smarter about +
On Sun, May 09, 2010 at 03:19:40PM -0600, David Abrahams wrote: > Yes. But the question is whether urlparse should really be so fragile > that every hierarchical scheme needs to be explicitly registered. > Surely ending with “+file” should be sufficient to have it recognized > as a file-based scheme How do you figure? ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Reindenting patches
On Wed, May 5, 2010 at 8:10 AM, Antoine Pitrou wrote:
> For the record, I've added to the untabify script a patch rewriting option
> ("-p") which reindents all patch hunks for C files containing tabs. It
> should
> minimize manual reformatting work with existing patches.
>
I just tried '-p' with a local patch that I had created
pre-whitespace-fixes. It worked like a charm. Thanks Antoine.
-- Meador
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] urlparse.urlunsplit should be smarter about +
On Sun, May 09, 2010 at 03:19:40PM -0600, David Abrahams wrote: > John Arbash Meinel wrote: > > Don't you need to register the "git+file:///" url for urlparse to > > properly split it? > > Yes. But the question is whether urlparse should really be so fragile > that every hierarchical scheme needs to be explicitly registered. > Surely ending with “+file” should be sufficient to have it recognized > as a file-based scheme Not all urls have the 'authority' component after the scheme. (sip based urls for e.g) urlparse differentiates those by maintaining a list of scheme names which will follow the pattern of parsing, and joining for the urls which have a netloc (or authority component). This is in general according to RFC 3986 itself. Yes,'+' is a valid char in url schemes and svn, svn+ssh will be as per your expectations. But git and git+ssh was missing in there and I attached a patch in issue8657 to include the same. It is rightly a bug in the module. But for any general scheme and assuming '+file' would follow valid authority component, is not something I am sure that should be in urlparse's expected behavior. -- Senthil Do not seek death; death will find you. But seek the road which makes death a fulfillment. -- Dag Hammarskjold ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] urlparse.urlunsplit should be smarter about +
David Abrahams writes:
> At Sat, 08 May 2010 11:04:47 -0500,
> John Arbash Meinel wrote:
> > Don't you need to register the "git+file:///" url for urlparse to
> > properly split it?
>
> Yes. But the question is whether urlparse should really be so fragile
> that every hierarchical scheme needs to be explicitly registered.
Exactly. And the answer is "no". The RFCs are quite clear that
hierarchical schemes are expected to be extremely common, and provide
several requirements for how they should be parsed, even by
nonvalidating parsers.
It's pretty clear to me that
urlunsplit(urlsplit('git+file:///foo/bar/baz'))
should be the identity. The remaining question is, "Should
urlunsplit(urlsplit('git+file:/foo/bar/baz'))
be the identity?" I would argue that if git+file is *not* registered,
it should be the identity, while there should be an optional
registry of schemes which may (or perhaps should) be canonicalized
(ie, a *missing* authority would be unsplit as an *empty* authority).
> Surely ending with "+file" should be sufficient to have it recognized
> as a file-based scheme
What's a "file-based scheme"? If you mean an RFC 3986 hierarchical
scheme, that is recognized by the presence of the authority portion,
which is syntactically defined by the presence of "//" immediately
after the scheme ":" terms. No need for any implicit magic.
In general, EIBTI applies here. If a registry as described above is
implemented, I would argue that canonicalization should not happen
implicitly. Nor should validation (eg, error or warning on a URI with
a scheme registered as hierarchical but lacking authority, or vice
versa). The API should require an explicit statement from the user to
invoke those functionalities. It might be useful for the OOWTDI API
to canonicalize/validate by default (especially given XSS attacks and
the like), but it should be simple for consenting adults to turn that
feature off.
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
