On 31/08/2011, at 19.29, Quanah Gibson-Mount wrote: > --On Wednesday, August 31, 2011 11:33 AM +0200 Thomas Rasmussen > <[email protected]> wrote: > > >> After a restart and performing a ldapsearch slapd has allocated over >> 500MB of memory, which is not exactly a good thing :-( > > How many total entries are in your database? What is your entrycache size? I > don't see that information in your stripped slapd.conf. Personally, I > suggest you never "strip" slapd.conf when sending it out, because you aren't > necessarily going to know what is relevant to what you are doing.
I have about 3 million DNs in the database, I was required by our customer to strip out some of the information, but it was only the 'access' bits and the rootdn+rootpw which is why I stripped some information. > > I think you are misunderstanding some of how OpenLDAP works. In addition to > the BDB cachesize (which you have configured as 50MB (which is likely grossly > small given the size of your database), there are also 3 additional caches > for back-bdb/back-hdb in slapd itself. Those are *not* filled until you > start doing searches. Those 3 caches are: > > entrycache > idlcache > dn cache OK, I was not aware that it would use memory for other caches that wasn't possible to setup (or are they?) > > My guess is it is growing to 500MB of memory because it is filling up the DN > cache. Given the 10GB size of your id2entry database, I'm guessing you have > a *lot* of DNs. > > I.e., the only way you will know the total size of what the slapd process > *should* be is to start it, and then do an ldapsearch of the entire database. > Once that is finished, you'll get an approximate size of your DB with most > of the caches filled (although the idlcache may take more time to fill). I tried to perform an ldapsearch on cn=* on my database (on ldap compiled with gcc, running BDB 5.2.x and tcmalloc enabled) and it allocated about 550MB of memory. > > Basically, I don't see 500MB as problematic, given the DB size you reported. > However, now that you have tcmalloc in place, the general size of slapd > should be significantly more stable. > > Finally, I will note that using "db_upgrade" to upgrade an OpenLDAP database > has not, and it is unlikely every will be, supported. The *only* supported > method of migrating to a newer version of BDB is to slapcat the original > database and to slapadd it back in. So it is entirely possible you have > created a very interesting issue by doing a completely unsupported operation. > Hmm, I found the db_upgrade process in the documentation of berkeley DB, so I'd imagine that it was supported, but apparently not by openldap... I will try to make a series of tests with my new-found knowledge about the way slapd will eat memory... hopefully it will continue to behave (it seems to have stabilazed around 552MB) Regards Thomas
