On Tue, Feb 28, 2023 at 11:16:45 +0100, Ondřej Kuzník wrote:
> Unless your entries are larger than pagesize *and* you have massive
> churn on those, you don't want to do this. Are you confident that's the
> case? What is your number of overflow pages? What kind of entries is
> it down to? If it's entries with large number of values in an attribute
> (e.g. groups), you might also want to look into sortvals (see man 5
> slapd.conf) and multival (man 5 slapd-mdb) to store them more
> efficiently.
Hi
We've had (and still have) this issue with large attributes and large
multi-valued attributes with Zimbra (see previous discussion with Quanah),
where we applied sortvals and multival. But in this scenario it's not the
case; all objects are of similar small size, with (mostly) single valued
attributes. Yet our freelist reaches 200K+ free pages during periods with
heavy updates (mostly deletes/adds), which has a measurable impact on write
performance.
For batch migrations we recently tried combining multiple updates into LDAP
transactions, which is significantly faster on a clean db, but makes the
freelist performance impact *worse* once the freelist is large enough.
Could it be because transactions require larger free pages which makes it
go through the entire freelist?
> It should be safe to include the accesslog *if* server was shut down
> cleanly and everything was flushed into both.
Should nightly backups include the accesslog as well then? (implying we
can no longer make simple mdb_copy backups while slapd is running...
Or is it good enough to dump the accesslog *after* the main db, so it
includes the relevant AND newer accesslog data?)
> Do you configure persistent or in-memory sessionlog?
in memory
> Are your accesslog entries so large that they don't fit a page? If not,
> just let the freelist be reused for the next time you have a large batch
> of updates again. That's what it's there for. And even then, accesslog
> in particular shouldn't really suffer from fragmentation as much as the
> main DB would.
Ok. We're seeing 1M+ free pages in the accesslog after large batch jobs
and subsequent logpurge. It could be completely innocent like you say,
we just clean it up as a precaution due to the proven main db perf issue
mentioned above. We'll hold this off for now and see.
> Yeah, in δ-multiprovider both main DB and accesslog (and their
> contextCSNs) are used together and should be monitored as such.
Thanks for your advice, will revise the monitoring.
Geert
