On Tue, 2005-08-02 at 06:54 +1000, Daniel Phillips wrote:
> Hi guys,
>
> Well I have been reading net code seriously for two days, so I am still
> basically a complete network klutz. But we have a nasty network-realted vm
> deadlock that needs fixing and there seems to be little choice but to wade in
> and try to sort things out.
>
We are also working on a similar problem where a set of critical TCP
connections need to successfully send/receive messages even under very
low memory conditions. But the assumption is that the low memory
situation lasts only for a short time(in the order of few minutes)
which should not cause any TCP timeouts to expire so that normal
connections can recover once the low memory situation is resolved.
> Here is the plan:
>
> * All protocols used on an interface that supports block IO must be
> vm-aware.
>
> If we wish, we can leave it up to the administrator to ensure that only
> vm-aware protocols are used on an interface that supports block IO, or we can
> do some automatic checking.
>
> * Any socket to be used for block IO will be marked as a "vmhelper".
I am assuming your 'vmhelper' is similar to a critical socket which can
be marked using a new socket option(ex: SO_CRITICAL).
>
> The number of protocols that need to have this special knowledge is quite
> small, e.g.: tcp, udp, sctp, icmp, arp, maybe a few others. We are talking
> about a line or two of code in each to add the necessary awareness.
>
> * Inside the network driver, when memory is low we will allocate space
> for every incoming packet from a memory reserve, regardless of whether
> it is related to block IO or not.
>
> * Under low memory, we call the protocol layer synchronously instead of
> queuing the packet through softnet.
>
> We do not necessarily have to bypass softnet, since there is a mechanism for
> thottling packets at this point. However, there is a big problem with
> throttling here: we haven't classified the packet yet, so the throttling
> might discard some block IO packets, which is exactly what we don't want to
> do under memory pressure.
>
> * The protocol receive handler does the socket lookup, then if memory is
> low, discards any packet not belonging to a vmhelper socket.
>
> Roughly speaking, the driver allocates each skb via:
>
> skb = memory_pressure ? dev_alloc_skb_reserve() : dev_alloc_skb();
Instead of changing all the drivers to make them vm aware, we could add
a new priority flag(something like GFP_CRITICAL) which can be passed to
__dev_alloc_skb(). dev_alloc_skb becomes
return __dev_alloc_skb(length, GFP_ATOMIC|GFP_CRITICAL);
Based on the memory pressure conditon, the VM can decide if the skb
needs to allocated from an emergency reserve.
>
> Then the driver hands off the packet to netif_rx, which does:
>
> if (from_reserve(skb)) {
> netif_receive_skb(skb);
> return;
> }
>
> And in the protocol handler we have:
>
> if (memory_pressure && !is_vmhelper(sock) && from_reserve(skb))
> goto drop_the_packet;
I am not sure if we need the from_reserve() checks above.
We have to assume that all incoming packets are critical until we can
find the matching sk in the protocol handler code.
>
> That is pretty much it. Now, being a net newbie, it is not entirely clear to
> me that we can call netif_receive_skb directly when packets are also being
> queued through the softnet interface. May I have some guidance on this
> point, please?
>
> If that works, I am prepared to justify and prove the rest.
>
> Regards,
>
> Daniel
> -
> To unsubscribe from this list: send the line "unsubscribe netdev" in
> the body of a message to [EMAIL PROTECTED]
> More majordomo info at http://vger.kernel.org/majordomo-info.html
>
-
To unsubscribe from this list: send the line "unsubscribe netdev" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at http://vger.kernel.org/majordomo-info.html
