Re: strcache scaling issue

2011-03-20 Thread Kirill Smelkov
On Sat, Mar 19, 2011 at 06:13:28PM -0400, Paul Smith wrote: > On Sat, 2011-03-19 at 20:47 +0300, Kirill Smelkov wrote: > > > 100020004000500060007000 > > > > 3.82.90 (with patch,2.2s4.8s14.0s 23.5s 31.6s 44.4s > > -O3, no asserts)

Re: strcache scaling issue

2011-03-19 Thread Paul Smith
On Sat, 2011-03-19 at 20:47 +0300, Kirill Smelkov wrote: > 100020004000500060007000 > > 3.82.90 (with patch,2.2s4.8s14.0s 23.5s 31.6s 44.4s > -O3, no asserts) > 3.81 -O32.5s5.2s11.5s 14.1s 18.3s 23.8s

Re: strcache scaling issue

2011-03-19 Thread Kirill Smelkov
On Wed, Feb 23, 2011 at 08:44:04PM +0100, Ralf Wildenhues wrote: > * Paul Smith wrote on Mon, Feb 21, 2011 at 04:01:49PM CET: > > On Mon, 2011-02-21 at 02:30 -0500, Paul Smith wrote: > > > Hi Ralf. I promoted a rework of strcache.c. For your simple scaling > > > test on my system with debug compi

Re: strcache scaling issue

2011-02-23 Thread Ralf Wildenhues
* Paul Smith wrote on Mon, Feb 21, 2011 at 04:01:49PM CET: > On Mon, 2011-02-21 at 02:30 -0500, Paul Smith wrote: > > Hi Ralf. I promoted a rework of strcache.c. For your simple scaling > > test on my system with debug compiled I get these results: > Try #2: > > Build Info

Re: strcache scaling issue

2011-02-21 Thread Paul Smith
On Mon, 2011-02-21 at 02:30 -0500, Paul Smith wrote: > Hi Ralf. I promoted a rework of strcache.c. For your simple scaling > test on my system with debug compiled I get these results: Gak. Evolution wrecked my table. Try #2: Build Info 10002000 4000

Re: strcache scaling issue

2011-02-20 Thread Paul Smith
On Sun, 2011-01-23 at 09:59 +0100, Ralf Wildenhues wrote: > * Paul Smith wrote on Sat, Jan 22, 2011 at 10:27:01PM CET: > > Thanks for your investigations here; that's very helpful. I had also > > had some thoughts about improving the strcache in various ways. For > > example, I was thinking maybe

Re: strcache scaling issue

2011-01-23 Thread Ralf Wildenhues
Hi Paul, * Paul Smith wrote on Sat, Jan 22, 2011 at 10:27:01PM CET: > Hi Ralf, sorry for the delay in reply. No worries. > On Sun, 2011-01-09 at 08:50 +0100, Ralf Wildenhues wrote: > > Just breaking out of the for loop in the if-true case avoids the problem > > for me. (The bulk of the patch is

Re: strcache scaling issue

2011-01-22 Thread Paul Smith
Hi Ralf, sorry for the delay in reply. As always your emails are information-packed and it takes me a while to find the time to read them with the appropriate amount of attention (I can whip off a response to your typical request-for-help in just a few minutes, in contrast :-) On Sun, 2011-01-09

Re: strcache scaling issue

2011-01-22 Thread Ralf Wildenhues
Ping! :-) * Ralf Wildenhues wrote on Sun, Jan 09, 2011 at 08:50:34AM CET: > 2011-01-09 Ralf Wildenhues > > * strcache.c (add_string): Use first fitting cache, to > avoid quadratic behavior searching for an optimal-size cache. > (strcache_iscached): Use the hash instead of wa

Re: strcache scaling issue

2011-01-10 Thread Mark Marshall
On 09/01/2011 07:59, Ralf Wildenhues wrote: Index: strcache.c === RCS file: /cvsroot/make/make/strcache.c,v retrieving revision 2.9 diff -u -r2.9 strcache.c --- strcache.c 13 Jul 2010 01:20:43 - 2.9 +++ strcache.c 9 Jan 20

Re: prerequisite scaling (was: strcache scaling issue)

2011-01-09 Thread Dave Hart
Ralf Wildenhues gmx.de> writes: > A doubly-linked list would be overkill (and memory-intensive), but I > think storing an end pointer to the dep chain in 'struct file' might > be prudent. That requires some changes throughout the code though, > and warrants some data structure change to avoid the

prerequisite scaling (was: strcache scaling issue)

2011-01-09 Thread Ralf Wildenhues
Continuing with the previous example makefile, but this time with 'make -r', I get a bit further, but only to hit the next bumper: this is with max=4: % cumulative self self total time seconds secondscalls s/call s/call name 96.75 25.88

Re: strcache scaling issue

2011-01-09 Thread Ralf Wildenhues
Looking at strcache, I see two other glitches: the size of the cache is not really aligned with page size: typical malloc implementations require two size_t entries overhead per allocated area. Also, there is a one-off computing the overhead size of the strcache struct. The patch below should cor

strcache scaling issue

2011-01-09 Thread Ralf Wildenhues
Hello bug-make readers, this originated in an interesting thread on the automake list: http://thread.gmane.org/gmane.comp.sysutils.automake.general/12421/focus=12431 There are more things discussed there, and most actual problems are solvable in Automake I guess, but there are interesting tidbits