Re: filter-branch IO optimization

2012-10-12 Thread Jeff King
On Fri, Oct 12, 2012 at 04:49:54PM +0200, Enrico Weigelt wrote: > > The usual advice is "use an index-filter instead". It's *much* > > faster > > than a tree filter. However: > > I've tried the last example from git-filter-branch manpage, but failed. > Seems like the GIT_INDEX_FILE env variable

Re: filter-branch IO optimization

2012-10-12 Thread Enrico Weigelt
Hi folks, now finally managed the index-filter part. The main problem, IIRC, was that git-update-index didn't automatically create an empty index, so I needed to explicitly copy in (manually created it with an empty repo). My current filter code is: if [ ! "$GIT_AUTHOR_EMAIL" ] && [ ! "$GIT_COMM

Re: filter-branch IO optimization

2012-10-12 Thread Enrico Weigelt
Did some more experiments, and it seems that missing index file isn't automatically created. When I instead copy the original index file to the temporary location, it runs well. But I still have to wait for the final result to check whether it really overwrites the whole index or just adds new f

Re: filter-branch IO optimization

2012-10-12 Thread Enrico Weigelt
Hi, > The usual advice is "use an index-filter instead". It's *much* > faster > than a tree filter. However: I've tried the last example from git-filter-branch manpage, but failed. Seems like the GIT_INDEX_FILE env variable doesnt get honoured by git-update-index, no index.new file created, and

Re: filter-branch IO optimization

2012-10-11 Thread Thomas Rast
Enrico Weigelt writes: > for certain projects, I need to regularily run filter-branch on quite > large repos (>10k commits), and that needs to be run multiple times, > which takes several hours, so I'm looking for optimizations. [...] > #2: run a tree-filter which: > * removes all files not b

Re: filter-branch IO optimization

2012-10-11 Thread Johannes Sixt
Am 11.10.2012 17:39, schrieb Enrico Weigelt: > The main goal of this filtering is splitting out many modules from a > large upstream repo into their own downstream repos. ... > The next step I have in mind is using --subdirectory-filter, but open > questsions are: > > * does it suffer from the sam

filter-branch IO optimization

2012-10-11 Thread Enrico Weigelt
Hi folks, for certain projects, I need to regularily run filter-branch on quite large repos (>10k commits), and that needs to be run multiple times, which takes several hours, so I'm looking for optimizations. The main goal of this filtering is splitting out many modules from a large upstream rep