On Thu, Mar 20, 2014 at 10:21 PM, Duy Nguyen wrote:
> On Fri, Mar 21, 2014 at 08:51:18AM +0700, Duy Nguyen wrote:
>> On Thu, Mar 20, 2014 at 11:08 PM, Stefan Zager wrote:
>> > Duy, would you like to re-post your patch without the new pread
>> > implementation?
>
On Thu, Mar 20, 2014 at 2:35 PM, Karsten Blees wrote:
> Am 20.03.2014 17:08, schrieb Stefan Zager:
>
>> Going forward, there is still a lot of performance that gets left on
>> the table when you rule out threaded file access. There are not so
>> many calls to read, mma
On Thu, Mar 20, 2014 at 6:54 AM, Karsten Blees wrote:
> Am 19.03.2014 01:46, schrieb sza...@chromium.org:
>> This adds a Windows implementation of pread. Note that it is NOT
>> safe to intersperse calls to read() and pread() on a file
>> descriptor.
>
> This is a bad idea. You're basically fixing
shown that this is, in fact, a lie.
To accomodate that fact, this change also incorporates:
http://article.gmane.org/gmane.comp.version-control.git/196042
... which gives each index-pack thread its own file descriptor.
Signed-off-by: Stefan Zager
---
builtin/index-p
On Wed, Mar 19, 2014 at 9:57 AM, Stefan Zager wrote:
>>
>> I still don't understand how compat/pread.c does not work with pack_fd
>> per thread. I don't have Windows to test, but I forced compat/pread.c
>> on on Linux with similar pack_fd changes and it worked
On Wed, Mar 19, 2014 at 3:28 AM, Duy Nguyen wrote:
> On Wed, Mar 19, 2014 at 2:50 PM, Stefan Zager wrote:
>>
>> I suppose it would be possible to fix the immediate problem just by
>> using one fd per thread, without a new pread implementation. But it
>> seems bet
On Wed, Mar 19, 2014 at 12:30 AM, Duy Nguyen wrote:
> On Wed, Mar 19, 2014 at 7:46 AM, wrote:
>> This adds a Windows implementation of pread. Note that it is NOT
>> safe to intersperse calls to read() and pread() on a file
>> descriptor. According to the ReadFile spec, using the 'overlapped'
>
On Fri, Feb 14, 2014 at 4:45 PM, Duy Nguyen wrote:
> On Sat, Feb 15, 2014 at 2:16 AM, Zachary Turner wrote:
>> (Gah, sorry if you're receiving multiple emails to your personal
>> addresses, I need to get used to manually setting Plain-text mode
>> every time I send a message).
>>
>> For the mixed
On Fri, Feb 14, 2014 at 11:04 AM, Karsten Blees wrote:
>
> Damn...you're right, multi-threaded git-index-pack works fine, but some tests
> fail badly. Mixed reads would have to be from git_mmap, which is the only
> other caller of pread().
msysgit used git_mmap() as defined in compat/win32mmap.
On Fri, Feb 14, 2014 at 11:15 AM, Zachary Turner wrote:
> For the mixed read, we wouldn't be looking for another caller of pread()
> (since it doesn't care what the file pointer is), but instead a caller of
> read() or lseek(). In index-pack.c, I see two possible culprits:
>
> 1) A call to xread(
I uploaded a new patch; a few comments inline below...
On Wed, Feb 12, 2014 at 1:19 PM, Junio C Hamano wrote:
> sza...@chromium.org writes:
>
> Also I'd suggest s/pack_data_fn/collect_pack_data/ or something.
> "_fn" may be a good suffix for typedef'ed typename used in a
> callback function, but
On Thu, Feb 13, 2014 at 2:51 PM, Karsten Blees wrote:
> Am 13.02.2014 19:38, schrieb Zachary Turner:
>
>> The only reason ReOpenFile is necessary at
>> all is because some code somewhere is mixing read-styles against the same
>> fd.
>>
>
> I don't understand...ReadFile with OVERLAPPED parameter do
On Thu, Feb 13, 2014 at 12:27 AM, Johannes Sixt wrote:
> Am 2/12/2014 20:30, schrieb Stefan Zager:
>> On Wed, Feb 12, 2014 at 11:22 AM, Karsten Blees
>> wrote:
>>> Am 12.02.2014 19:37, schrieb Erik Faye-Lund:
>>>> ReOpenFile, that's fantastic. Than
On Wed, Feb 12, 2014 at 12:06 PM, Junio C Hamano wrote:
> Stefan Zager writes:
>
>> ... I used the Very Sleepy profiler
>> to see where all the time was spent on Windows: 55% of the time was
>> spent in OpenFile, and 25% in CloseFile (both in win32).
>
> This is
On Wed, Feb 12, 2014 at 11:22 AM, Karsten Blees wrote:
> Am 12.02.2014 19:37, schrieb Erik Faye-Lund:
>> On Wed, Feb 12, 2014 at 7:34 PM, Stefan Zager wrote:
>>> On Wed, Feb 12, 2014 at 10:27 AM, Erik Faye-Lund
>>> wrote:
>>>> On Wed, Feb 12, 2014 at 7:20
On Wed, Feb 12, 2014 at 10:50 AM, David Kastrup wrote:
> Stefan Zager writes:
>
>> Anything on Windows that touches a lot of files is miserable due to
>> the usual file system slowness on Windows, and luafv.sys (the UAC file
>> virtualization driver) seems to make it muc
On Wed, Feb 12, 2014 at 10:33 AM, Matthieu Moy
wrote:
> Stefan Zager writes:
>
>> I'm optimistic that parallelizing the stat calls will yield a further
>> improvement.
>
> It has already been mentionned in the thread, but in case you overlooked
> it: did you loo
On Wed, Feb 12, 2014 at 10:27 AM, Erik Faye-Lund wrote:
> On Wed, Feb 12, 2014 at 7:20 PM, Stefan Zager wrote:
>>
>> I don't want to steal the thunder of my coworker, who wrote the
>> implementation. He plans to submit it upstream soon-ish. It relies
>> on us
On Tue, Feb 11, 2014 at 11:29 PM, Chris Packham wrote:
> Hi,
>
> On 12/02/14 14:57, Stefan Zager wrote:
>> From b4796d9d99c03b0b7cddd50808a41413e45f1129 Mon Sep 17 00:00:00 2001
>> From: Stefan Zager
>> Date: Mon, 10 Feb 2014 16:55:12 -0800
>> Subject: [PATCH] Ma
On Wed, Feb 12, 2014 at 3:59 AM, Erik Faye-Lund wrote:
> On Wed, Feb 12, 2014 at 2:54 AM, Stefan Zager wrote:
>>
>> We are particularly concerned with the performance of msysgit, and we
>> have already chalked up a significant performance gain by turning on
>> the thr
On Tue, Feb 11, 2014 at 7:43 PM, Duy Nguyen wrote:
>
> From v1.9 shallow clone should work for all push/pull/clone... so
> history depth does not matter (on the client side). As for
> gentoo-x86's large worktree, using index v4 and avoid full-tree
> operations (e.g. "status .", not "status"..) sho
On Tue, Feb 11, 2014 at 6:11 PM, Duy Nguyen wrote:
>
> I have no comments about thread safety improvements (well, not yet).
> If you have investigated about git performance on chromium
> repositories, could you please sum it up? Threading may be an option
> to improve performance, but it's probabl
>From b4796d9d99c03b0b7cddd50808a41413e45f1129 Mon Sep 17 00:00:00 2001
From: Stefan Zager
Date: Mon, 10 Feb 2014 16:55:12 -0800
Subject: [PATCH] Make the global packed_git variable static to sha1_file.c.
This is a first step in making the codebase thread-safe. By and
large, the operati
st of pack files (packed_git) internal to sha1_file.c.
I realize this may be a contentious topic, and I'd like to get
feedback on the general effort to add more threading to git. I'd
appreciate any feedback you'd like to give up front.
Thanks!
Stefan Zager
--
To unsubscribe from
Howdy,
Is there any policy on making static initializers thread-safe? I'm
working on an experimental patch to introduce threading, but I'm
running into a few non-thread-safe bits like this, in convert.c:
static const char *conv_attr_name[] = {
"crlf", "ident", "filter", "eol", "text",
};
#de
This is probably a naive question, but: there are quite a lot of static
variables in the git code where it's really unnecessary. Is that just a
historical artifact, or is there some reason to prefer them? I'm working
on a patch that will introduce threading, so naturally I'm on the lookout
for st
On Thu, Oct 10, 2013 at 5:51 PM, Karsten Blees wrote:
> >> I've noticed that when working with a very large repository using msys
> >> git, the initial checkout of a cloned repository is excruciatingly
> >> slow (80%+ of total clone time). The root cause, I think, is that git
> >> does all the fi
Hi folks,
I don't follow the mailing list carefully, so forgive me if this has
been discussed before, but:
I've noticed that when working with a very large repository using msys
git, the initial checkout of a cloned repository is excruciatingly
slow (80%+ of total clone time). The root cause, I
We have uncovered a regression in this commit:
b8a2486f1524947f232f657e9f2ebf44e3e7a243
The symptom is that 'git fetch' dies with:
error: index-pack died of signal 10
fatal: index-pack failed
I have only been able to reproduce it on a Mac thus far; will try ubuntu next.
We can make it go away
ping?
On Tue, Oct 30, 2012 at 11:11 AM, Stefan Zager wrote:
> This is a refresh of a conversation from a couple of months ago.
>
> I didn't try to implement all the desired features (e.g., smart logic
> for passing a -j parameter to recursive submodule invocations), but I
>
.
>
> Signed-off-by: Stefan Zager
> ---
> Documentation/git-submodule.txt |8 ++-
> git-submodule.sh| 40
> ++-
> 2 files changed, 46 insertions(+), 2 deletions(-)
>
> diff --git a/Documentation/git-subm
On Fri, Oct 19, 2012 at 1:27 PM, Jeff King wrote:
>
> On Fri, Oct 19, 2012 at 07:10:46AM -0700, Shawn O. Pearce wrote:
>
> > > IOW, it seems like we are _already_ following the advice referenced in
> > > curl's manpage. Is there some case I am missing? Confused...
> >
> > The issue with the curren
dule is suspect, and should be reset
to HEAD.
Signed-off-by: Stefan Zager
---
Documentation/git-submodule.txt | 9 -
git-submodule.sh| 2 +-
t/t7406-submodule-update.sh | 12
3 files changed, 21 insertions(+), 2 deletions(-)
diff --git a/Documentation/git-
The --jobs parameter may be used to set the degree of per-submodule
parallel execution.
Signed-off-by: Stefan Zager
---
Documentation/git-submodule.txt | 8 +++-
git-submodule.sh| 23 ++-
2 files changed, 29 insertions(+), 2 deletions(-)
diff --git a
dule is suspect, and should be reset
to HEAD.
Signed-off-by: Stefan Zager
---
git-submodule.sh | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/git-submodule.sh b/git-submodule.sh
index dba4d39..621eff7 100755
--- a/git-submodule.sh
+++ b/git-submodule.sh
@@ -575,7 +575,7 @@ Ma
35 matches
Mail list logo
