Re: [Python-Dev] gcc 4.2 exposes signed integer overflows
Jack Howarth wrote: > Guido, > You'll never win that argument with the gcc developers. If you > rely on undefined behavior in the c language standard, they have > in the past, and will continue to, feel free to ignore those cases. > If you plan on ignoring this issue, just be prepared to see a > testcase failure in the python testsuite when python is built with > gcc 4.2. In addition, i'm surprised that you don't see these failures on other compilers. I know XLC, for example, would treat signed integer overflow as undefined at all opt levels as well. I would not be surprised to find Intel's compiler doing the same thing. Speaking as a gcc optimization person, if we were to treat signed integer overflow as wrapping, you would inhibit a very large number of loop optimizations on a very large class of loops (due to a number of reasons, from no longer being able to prove termination of loops or estimation of number of iterations, to other things). Heck, XLC at -O3+ will ignore *unsigned* integer wraparound of loop indices unless you use -qstrict_induction. As the docs for this option say, using that option can cause severe performance degradation (I happen to know the numbers from when i worked at IBM, and they are surprisingly high). GCC is willing to take the performance hit relative to other compilers on things like that to be a standard conforming compiler, just like we are willing to optimize code based on the assumption that you do not invoke undefined behavior. --Dan ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] [Python-3000] Invitation to try out open source code review tool
There is a google code support project where google code issues are tracked. http://code.google.com/p/support/ On Mon, May 5, 2008 at 2:56 PM, Guido van Rossum <[EMAIL PROTECTED]> wrote: > That's a good question. I'll find out. > > > > On Mon, May 5, 2008 at 10:56 AM, Georg Brandl <[EMAIL PROTECTED]> wrote: > > Guido van Rossum schrieb: > > > > > > > On Mon, May 5, 2008 at 9:46 AM, "Martin v. Löwis" <[EMAIL PROTECTED]> > > wrote: > > > > > > > > This code is now open source! Browse it here: > > > > > > > > > > http://code.google.com/p/rietveld/source/browse > > > > > > > > Are you also going to call it Rietveld then? Sounds better > > > > to me than "the open source code review tool". > > > > > > > > > > I've been reluctant to use the Rietveld name too much since Americans > > > can't spell it. :-) But the open source project *is* called Rietveld, > > > so I suppose I should start using that name... > > > > > > > BTW, Google code's Python highlighting is broken, look at > > http://code.google.com/p/rietveld/source/browse/trunk/views.py line 205ff. > > > > Where can this be reported? > > > > Georg > > > > > > > > ___ > > Python-Dev mailing list > > Python-Dev@python.org > > http://mail.python.org/mailman/listinfo/python-dev > > Unsubscribe: > > http://mail.python.org/mailman/options/python-dev/guido%40python.org > > > > > > > -- > --Guido van Rossum (home page: http://www.python.org/~guido/) > ___ > > > Python-Dev mailing list > Python-Dev@python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > http://mail.python.org/mailman/options/python-dev/dberlin%40dberlin.org > ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Fwd: Distributed RCS
On Sun, 2005-08-14 at 11:12 -0600, Neil Schemenauer wrote:
> On Sun, Aug 14, 2005 at 06:16:11PM +0200, "Martin v. Löwis" wrote:
> > It depends on what "a bit" is. Waiting a month would be fine; waiting
> > two years might be pointless.
>
> It looks like the process of converting a CVS repository to
> Bazaar-NG does not yet work well (to be kind). The path
> CVS->SVN->bzr would probably work better. I suspect cvs2svn has
> been used on quite a few CVS repositories already. I don't think
> going to SVN first would lose any information.
It doesn't.
As a data point, CVS2SVN can handle gcc's massive cvs repository, which
has merged rcs file information in it dating back to 1987, >1000 tags,
and > 300 branches.
Besides monotone's cvs_import, it's actually the only properly designed
cvs converter I've seen in a while (Properly designed in that it
actually uses the necessary and correct algorithms to get all the
weirdities of cvs branches and tags right).
I'm not sure how big python's repo is, but you probably want to use the
attached patch to speed up cvs2svn. It changes it to reconstruct the
revisions on it's own instead of calling cvs or rcs. For GCC, and KDE,
this makes a significant difference (17 hours for our 4 gig cvs repo
convresion instead of 52 hours), because it was spawning cvs/rcs 50
billion times, and the milliseconds add up :)
> My vote is to continue with the migration to SVN. We can
> re-evaluate Bazaar-NG at a later time.
GCC is moving to SVN (very soon now, within 2 months), and this has been
my viewpoint as well.
It's much easier to go from something that has changesets and global
revisions, to a distributed system, if you want to, than it is to try to
reconstruct that info from CVS on your own :).
Subversion also has excellent language bindings, including the python
bindings. That's how i've hooked it up to gcc's bugzilla. You could
easily write something to transform *from* subversion to another system
using the bindings.
Things like viewcvs use the python bindings to deal with the svn
repository entirely.
--Dan
Index: cvs2svn
===
--- cvs2svn (revision 1423)
+++ cvs2svn (working copy)
@@ -166,6 +166,10 @@
# grouping. See design-notes.txt for details.
DATAFILE = 'cvs2svn-data'
+REVISIONS_DB = 'cvs2svn-cvsrepo.db'
+
+CHECKOUT_DB = 'cvs2svn-cvsco.db'
+
# This file contains a marshalled copy of all the statistics that we
# gather throughout the various runs of cvs2svn. The data stored as a
# marshalled dictionary.
@@ -355,40 +359,7 @@
" cvsroot\n" % (error_prefix, cvsroot, fname))
sys.exit(1)
-def get_co_pipe(c_rev, extra_arguments=None):
- """Return a command string, and the pipe created using that string.
- C_REV is a CVSRevision, and EXTRA_ARGUMENTS is used to add extra
- arguments. The pipe returns the text of that CVS Revision."""
- ctx = Ctx()
- if extra_arguments is None:
-extra_arguments = []
- if ctx.use_cvs:
-pipe_cmd = [ 'cvs' ] + ctx.cvs_global_arguments + \
- [ 'co', '-r' + c_rev.rev, '-p' ] + extra_arguments + \
- [ ctx.cvs_module + c_rev.cvs_path ];
- else:
-pipe_cmd = [ 'co', '-q', '-x,v', '-p' + c_rev.rev ] + extra_arguments + \
- [ c_rev.rcs_path() ]
- pipe = SimplePopen(pipe_cmd, True)
- pipe.stdin.close()
- return pipe_cmd, pipe
-
-def generate_ignores(c_rev):
- # Read in props
- pipe_cmd, pipe = get_co_pipe(c_rev)
- buf = pipe.stdout.read(PIPE_READ_SIZE)
- raw_ignore_val = ""
- while buf:
-raw_ignore_val = raw_ignore_val + buf
-buf = pipe.stdout.read(PIPE_READ_SIZE)
- pipe.stdout.close()
- error_output = pipe.stderr.read()
- exit_status = pipe.wait()
- if exit_status:
-sys.exit("%s: The command '%s' failed with exit status: %s\n"
- "and the following output:\n"
- "%s" % (error_prefix, pipe_cmd, exit_status, error_output))
-
+def generate_ignores(raw_ignore_val):
# Tweak props: First, convert any spaces to newlines...
raw_ignore_val = '\n'.join(raw_ignore_val.split())
raw_ignores = raw_ignore_val.split('\n')
@@ -614,9 +585,7 @@
DB_OPEN_READ = 'r'
DB_OPEN_NEW = 'n'
-# A wrapper for anydbm that uses the marshal module to store items as
-# strings.
-class Database:
+class SDatabase:
def __init__(self, filename, mode):
# pybsddb3 has a bug which prevents it from working with
# Berkeley DB 4.2 if you open the db with 'n' ("new"). This
@@ -635,22 +604,24 @@
self.db = anydbm.open(filename, mode)
- def has_key(self, key):
-return self.db.has_key(key)
+ def __getattr__(self, name):
+return getattr(self.db, name)
+# A wrapper for anydbm that uses the marshal module to store items as
+# strings.
+class Database(SDatabase):
+
def __getitem__(self, key):
return marshal.loads(self.db[key])
def __setitem__(self, key, value):
self.db[key] = marshal.dumps(value)
- def __delitem__(self, key):
Re: [Python-Dev] Fwd: PEP: Migrating the Python CVS to Subversion
On Sun, 2005-08-14 at 11:13 -0700, Guido van Rossum wrote: > Here's another POV. (Why does evereybody keep emailing me personally?) > Because we love you, and I forgot to cc python-dev. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Fwd: PEP: Migrating the Python CVS to Subversion
On Sun, 2005-08-14 at 23:58 +0200, "Martin v. Löwis" wrote: > Guido van Rossum wrote: > > Here's another POV. > > I think I agree with Daniel's view, in particular wrt. to performance. > Whatever the replacement tool, it should perform as well or better > than CVS currently does; it also shouldn't perform much worse than > subversion. Then, in fairness, I should note that annotate is slower on subversion (and monotone, and anything using binary deltas) than CVS. This is because you can't generate line-diffs that annotate wants from binary copy + add diffs. You have to reconstruct the actual revisions and then line diff them.Thus, CVS is O(N) here, and SVN and other binary delta users are O(N^2). You wouldn't really notice the speed difference when you are annotating a file with 100 revisions. You would if you annotate the 800k changelog which has 30k trunk revisions. CVS takes 4 seconds, svn takes ~5 minutes, the whole time being spent in doing diffs of those revisions. I rewrote the blame algorithm recently so that it will only take about 2 minutes on changelog, but it cheats because it knows it can stop early because it's blamed all the revisions (since our changelog rotates). For those curious, you also can't directly generate "always-correct" byte-level differences from the diffs, since their goal is to find the most space efficient way to transform rev old into rev new, *not* record actual byte-level changes that occurred between old and new. It may turn out that doing an add of 2 bytes is cheaper than specifying the opcode for copy(start,len). Actual diffs are produced by reproducing the texts and line diffing them. Such is the cost of efficient storage :). > > I've been using git (or, rather, cogito) to keep up-to-date with the > Linux kernel. While performance of git is really good, storage > requirements are *quite* high, and initial "checkout" takes a long > time - even though the Linux kernel repository stores virtual no > history (there was a strict cut when converting the bitkeeper HEAD). > So these distributed tools would cause quite some disk consumption > on client machines. bazaar-ng apparently supports only-remote > repositories as well, so that might be no concern. The argument "network and disk is cheap" doesn't work for us when you are talking 5-10 gigabytes of initial transfer :). However, I doubt it's more than a hundred meg or so for python, if that. You may run into these problems in 10 years :) ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Fwd: Distributed RCS
On Mon, 2005-08-15 at 00:15 +0200, "Martin v. Löwis" wrote: > Daniel Berlin wrote: > > I'm not sure how big python's repo is, but you probably want to use the > > attached patch to speed up cvs2svn. It changes it to reconstruct the > > revisions on it's own instead of calling cvs or rcs. > > Thanks for the patch, but cvs2svn works fairly well for us as is (in > the version that was released with Debian sarge); see > > http://www.python.org/peps/pep-0347.html > > for the conversion procedure. On the machine where I originally did > the conversion, the script required 7h; on my current machine, it is > done in 1:40 or so, which is acceptable. > > Out of curiosity: do you use the --cvs-revnums parameter? Should we? No. In our case, it doesn't buy us anything. In the name of continuity, we have to make the old cvsweb urls work with new viewcvs urls anyway (they appear in bug reports, etc). We also don't want to destroy the ability for people to diff existing cvs working copies. I may have been able to hack something around with cvs-revnums, but not easily. Thus, we are just going to keep a readonly version of the repo around, and a readonly cvsweb, with a warning at the top of the page that the current source is stored in subversion. > > Regards, > Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP: Migrating the Python CVS to Subversion
On Mon, 2005-08-15 at 12:27 -0400, Nicholas Bastin wrote: > On 8/8/05, "Martin v. Löwis" <[EMAIL PROTECTED]> wrote: > > Nicholas Bastin wrote: > > > It's a mature product. I would hope that that would count for > > > something. > > > > Sure. But so is subversion. > > I will then assume that you and I have different ideas of what 'mature' means. Bigger projects than Python use it and consider it mature for real use (All the Apache projects, all of KDE, GNOME is planning on switching soon, etc). I've never seen a corrupted FSFS repo, only corrupted BDB repos, and I will happily grant that using BDB ended up being a big mistake for Subversion. Not one that could have easily been foreseen at the time, but such is life. But this is why FSFS is the default for 1.2+ I've never seen you post about a corrupted repository to svn-users or svn-dev or file a bug, so i can't say why you see corrupted repositories if they are FSFS ones. --Dan ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] svn checksum error
On Sun, 2005-10-30 at 19:08 -0600, [EMAIL PROTECTED] wrote: > Martin> The natural question then is: what operating system, what > Martin> subversion version are you using? > > Sorry, wasn't thinking in terms of svn bugs. I was anticipating some sort > of obvious pilot error. I am on Mac OSX 10.3.9, running svn 1.1.3 I built > from source back in the May timeframe. Should I upgrade to 1.2.3 as a > matter of course? > > Fredrik> "welcome to the wonderful world of subversion error messages" > ... > Fredrik> deleting the offending directory and doing "svn up" is the > Fredrik> easiest way to fix this. > > Thanks. I zapped Objects. The next svn up complained about Misc. The next > about Lib. After that, the next svn up ran to completion. > > Skip You didn't happen to try to update a checked out copy from a repo that had an older cvs2svn conversion to the one produced by the final conversion, did you? Cause that will cause these errors too. --Dan ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] should I really have to install Python before Ican build it ?
On Mon, 2005-12-12 at 20:43 +0100, Fredrik Lundh wrote: > Jeremy Hylton wrote: > > > The C files are checked into subversion. Perhaps there is some > > problem with the timestamps that causes the Makefile to try to rebuild > > them anyway? I have a modern Python and I've been doing a fair amount > > of development on these files; as a result, I haven't noticed a > > problem. > > ah, of course. subversion sets the timestamp to the checkout time for each > file, so things may or may not work after a fresh checkout. You can change this by setting use-commit-times=true in ~/.subversion/config ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
