On 18.11.2013 10:41, Adrian Chadd wrote:
Your patch does three things:
* adds a couple new buckets;
These new buckets make bucket size self-tuning more soft and precise.
Without them there are buckets for 1, 5, 13, 29, ... items. While at
bigger sizes difference about 2x is fine, at smallest ones it is 5x and
2.6x respectively. New buckets make that line look like 1, 3, 5, 9, 13,
29, reducing jumps between steps, making algorithm work softer,
allocating and freeing memory in better fitting chunks. Otherwise there
is quite a big gap between allocating 128K and 5x128K of RAM at once.
* reduces some lock contention
More precisely patch adds check for congestion on free to grow bucket
sizes same as on allocation. As consequence that indeed should reduce
lock congestion, but I don't have specific numbers. All I see is that VM
and UMA mutexes no longer appear in profiling top after all these changes.
* does soft back pressure
In this list you have missed mentioning small but major point of the
patch -- we should prevent problems, not just solve them. As I have
written in original email, this specific change shown me 1.5x
performance improvement in low-memory condition. As I understand, that
happened because VM no longer have to repeatedly allocate and free
hugely oversized buckets of 10-15 * 128K.
* does the aggressive backpressure.
After all above that is mostly just a safety belt. With 40GB RAM that
code was triggered only couple times during full hour of testing with
debug logging inserted there. On machine with 2GB RAM it is triggered
quite regularly and probably that is unavoidable since even with lowest
bucket size of one item 24 CPUs mean 48 cache buckets, i.e. up to 6MB of
otherwise unreleasable memory for single 128K zone.
So, do you get any benefits from just the first one, or first two?
I don't see much reason to handle that in pieces. As I have described
above, each part has own goal, but they much better work together.
On 17 November 2013 15:09, Alexander Motin <m...@freebsd.org> wrote:
Hi.
I've created patch, based on earlier work of avg@, to add back pressure to
UMA allocation caches. The problem of physical memory or KVA exhaustion
existed there for many years and it is quite critical now for improving
systems performance while keeping stability. Changes done in memory
allocation last years improved situation. but haven't fixed completely. My
patch solves remaining problems from two sides: a) reducing bucket sizes
every time system detects low memory condition; and b) as last-resort
mechanism for very low memory condition, it cycling over all CPUs to purge
their per-CPU UMA caches. Benefit of this approach is in absence of any
additional hard-coded limits on cache sizes -- they are self-tuned, based on
load and memory pressure.
With this change I believe it should be safe enough to enable UMA allocation
caches in ZFS via vfs.zfs.zio.use_uma tunable (at least for amd64). I did
many tests on machine with 24 logical cores (and as result strong allocation
cache effects), and can say that with 40GB RAM using UMA caches, allowed by
this change, by two times increases results of SPEC NFS benchmark on ZFS
pool of several SSDs. To test system stability I've run the same test with
physical memory limited to just 2GB and system successfully survived that,
and even showed results 1.5 times better then with just last resort measures
of b). In both cases tools/umastat no longer shows unbound UMA cache growth,
that makes me believe in viability of this approach for longer runs.
I would like to hear some comments about that:
http://people.freebsd.org/~mav/uma_pressure.patch
--
Alexander Motin
_______________________________________________
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
