On Wed, Sep 28, 2011 at 11:01, Trevor Schaffer <trevorschaf...@smarttech.com> wrote: > We definitely use svn copy for revisions, but I think the issue is because > our tags are too flat vs not flat enough. > > E.g. tags/builds/ is where we put all of our tags (all done with svn copy) > And over time, this folder has over 7000 tags in it. So now, ever new tag we > put into /tags/builds has a listing of every other tag from tags/builds in > the revs/db/<rev> file. Now when you have 4 lines in each rev file for each > entry in the tags folder, that's 28000 lines of text, which gives us the > roughly 600KB of data. And you can see how this grows over time. So > obviously, in hindsight... we need to lessen the tags significantly, but > current processes in our company will not allow us to do that quickly, or > easily.
Would it be possible to clean up your tags directory, maybe on a quarterly or annual basis? So instead of having a flat /tags/builds "dumping ground", you have; /tags/builds/ /tags/builds/2011Q1 /tags/builds/2011Q2 /tags/builds/2011Q3 /tags/builds/2011Q4 Your tags go into /tags/builds as they do today, but then at the end of each quarter, you move that quarter's builds into the corresponding directory. > -----Original Message----- > From: Stefan Sperling [mailto:s...@elego.de] > Sent: Wednesday, September 28, 2011 8:23 AM > To: Trevor Schaffer > Cc: users@subversion.apache.org > Subject: Re: revs files growing over time, relatively > > On Wed, Sep 28, 2011 at 07:57:35AM -0600, Trevor Schaffer wrote: >> Server: Fedora Core 13 64-bit >> SVN: 1.6.16 running on Apache 2.2.16 >> >> We have a repository that currently is in the 128K revision count, and is >> now standing at 40+GB in size. Over time it's been growing faster, and we >> noticed a trend on svn copies... the db/revs/* files are getting bigger over >> time. >> >> After cracking one open, and another random commit, I saw that the >> commit entry not only lists information about the current path and >> it's siblings, but it also enumerates all parents, and their siblings >> as well... all the way to the root. > > Yes, this is how "cheap copies" in Subversion work. > There is no way of changing this. > It's called the "bubble up effect" which is described neatly at > http://red-bean.com/kfogel/beautiful-code/bc-chapter-02.html > >> This is a big issue for us, as we >> do a lot of build tagging, which means that every build tag commit >> lists the other thousands of build tags as well, which is why they are >> growing over time. At the current size of 600KB/commit for a single >> svn copy into our /tags area, performing 20,000 commits covers about >> 12GB of size, which I think is quite significant. Truthfully, >> probably 30+GB of this repo is just svn copies worth of commits. > > I am surprised that the copy references take that much space. > How deeply nested are the tag folders in the tree? > You could try to flatten out the hierarchy of your repository. > >> In the meantime, I'm going to try to dump+load the repo in the hopes that >> there were some optimizations in svn 1.6 that will help. We've had this >> repo running for nearly 5 years, since svn 1.4. > > Yes, try that. 1.6 and upwards support "representation-sharing" which > might cut down the size of the repository significantly. > > You could also try packing your repositories with 'svnadmin pack'. > This will cause the repository to use less inodes and free up space > lost to disk blocks which aren't completely filled up with data. > > I am curious about how you create tags. Do you really run 'svn copy' > or do you maybe use 'svn import' to create them (which would bloat the > repository significantly if representation-sharing is disabled)? > > >
