On Tue, Aug 28, 2007 at 02:03:57PM -0700, Quanah Gibson-Mount wrote: > Here's the relevant information. > > Looks like x86_64 requires a specific option. Maybe it'd be best to > compile it that way for all platforms.
Are there reproducible tests/benchmarks that we can use to figure out whether or not this actually makes sense on any given architecture? > ------------ Forwarded Message ------------ > Date: Tuesday, August 28, 2007 1:47 PM -0700 > From: Howard Ch > To: Quanah Gibson-Mount > Cc: [EMAIL PROTECTED] > Subject: Re: OpenLDAP 2.3.38 and Berkeley DB 4.6.19 > > Quanah Gibson-Mount wrote: >> --On Tuesday, August 28, 2007 2:44 PM -0400 matthew sporleder >> <[EMAIL PROTECTED]> wrote: >> >>> That's an unfortunate development from oracle considering the >>> presentation that was released showing bdb4.6 to be very fast. >> >> Yup. I believe Howard is working with Sleepycat to find out what was >> done to 4.6 to make it suddenly be 3-4 times slower instead of >> significantly faster as it was in the early releases. > > Yes, it's been a long conversation so far. While their new memory manager > (inspired by Jong-Hyuk Choi's research on BerkeleyDB performance in > OpenLDAP) is a great improvement, their new lock manager is not. > > This may not be an issue on all platforms. On x86 though, they have a > hybrid mutex mechanism which is enabled by default. It uses both assembly > language spinlocks and pthread mutexes, first spinning in the assembly > language lock some number of times before falling back onto the mutex. They > claimed that this improved performance in their tests, because pthread > mutexes can be very expensive on some platforms. In my tests on x86_64 > Linux however, it just forced CPU usage to 100% (200% actually, dualcore) > and slowed down overall throughput. > > You can give an explicit "--with-mutex=POSIX/pthread" argument when > configuring BerkeleyDB to avoid the hybrid mutex scheme, in which case > performance of BDB 4.6.19 seems to match what we obtained with BDB 4.6.3. > > If you were using BerkeleyDB 4.5.20 before, the early BDB 4.6 releases will > work well for you. The only difference between 4.5.20 and 4.6.1 was in the > memory manager. 4.6.2 just tweaked some portability issues to support BREW > (Qualcomm's cellphone programming environment). 4.6.3 added the ability to > specify different cache priorities per database operation. (I didn't track > what changed between 4.6.3 and 4.6.18.) > > And for anyone curious - you can read ITS#3851 > http://www.openldap.org/its/index.cgi/Development?id=3851 for the > background on the problems in BerkeleyDB's memory allocator. While there > was a fair amount of debate as to whether Jong's proposed solution was of > any benefit, it was pretty clear that the existing code in BerkeleyDB could > be improved. But the changes in the memory allocator may not visibly affect > you if you're running a small-to-medium sized database. It's only apparent > when the total volume of data is much larger than the BerkeleyDB cache, > because that's the condition that leads to memory fragmentation, and it's > the fragmentation that causes the allocator's performance to degrade. > > The same kind of problem affects the slapd entry cache, when the number of > active entries is much larger than the entry cache, and you're using the > glibc malloc. (At least, for glibc 2.3 and older. I haven't retested with > glibc 2.5 yet.) > > To sum up - if you're already using BDB 4.5.20 with OpenLDAP 2.3 and the > performance is acceptable, I wouldn't be in any hurry to upgrade to BDB > 4.6. Releases 4.6.18 and 4.6.19 are incompatible with OpenLDAP 2.3. BDB > 4.6.1 is a drop-in replacement for 4.5.20 though. > -- > -- Howard Chu > Chief Architect, Symas Corp. http://www.symas.com > Director, Highland Sun http://highlandsun.com/hyc/ > Chief Architect, OpenLDAP http://www.openldap.org/project/ > > ---------- End Forwarded Message ---------- -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]
