On 11.11.2012 22:40, Alan Cox wrote:
On Sat, Nov 10, 2012 at 7:20 AM, Konstantin Belousov <[email protected]>wrote:
Your analysis is right, there is nothing to add or correct.
This is the reason to strongly prefer M_WAITOK.
Agreed. Once upon time, before SMPng, M_NOWAIT was rarely used. It was
well understand that it should only be used by interrupt handlers.
The trouble is that M_NOWAIT conflates two orthogonal things. The obvious
being that the allocation shouldn't sleep. The other being how far we're
willing to deplete the cache/free page queues.
When fine-grained locking got sprinkled throughout the kernel, we all to
often found ourselves wanting to do allocations without the possibility of
blocking. So, M_NOWAIT became commonplace, where it wasn't before.
Yes, we have many places where we don't want to sleep for example in
the network code. There we simply want to be told that we've run out
of memory and handle the failure. It's expected to happen from time
to time. We don't need or want to dig deep or into reserves. Packets
are expected to get lost from time to time and upper layer protocols
will handle retransmits just fine. What we *don't* want normally is to
get blocked on a failing memory allocation. We'd rather drop this one
and go on with the next packet to avoid the head of line blocking
problem where everything cascades to a total halt.
As a side note we don't do many, if any, true interrupt time allocations
anymore. Usually the interrupt is just acknowledged in interrupt
context and a taskqueue or ithread is scheduled to do all the hard work.
Neither runs in interrupt context.
This had the unintended consequence of introducing a lot of memory
allocations in the top-half of the kernel, i.e., non-interrupt handling
code, that were digging deep into the cache/free page queues.
Also, ironically, in today's kernel an "M_NOWAIT | M_USE_RESERVE"
allocation is less likely to succeed than an "M_NOWAIT" allocation.
However, prior to FreeBSD 7.x, M_NOWAIT couldn't allocate a cached page; it
could only allocate a free page. M_USE_RESERVE said that it ok to allocate
a cached page even though M_NOWAIT was specified. Consequently, the system
wouldn't dig as far into the free page queue if M_USE_RESERVE was
specified, because it was allowed to reclaim a cached page.
In conclusion, I think it's time that we change M_NOWAIT so that it doesn't
dig any deeper into the cache/free page queues than M_WAITOK does and
reintroduce a M_USE_RESERVE-like flag that says dig deep into the
cache/free page queues. The trouble is that we then need to identify all
of those places that are implicitly depending on the current behavior of
M_NOWAIT also digging deep into the cache/free page queues so that we can
add an explicit M_USE_RESERVE.
I don't think many places depend on M_NOWAIT digging deep. I'm
perfectly happy with having M_NOWAIT give up on first try. Only
together with M_TRY_REALLY_HARD it would dig into reserves.
PS: We have a really nasty namespace collision with the mbuf flags
which use the M_* prefix as well.
--
Andre
_______________________________________________
[email protected] mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "[email protected]"
