Re: [RFC][PATCH 0/3] TCP/IP Critical socket communication mechanism

2005-12-21 Thread David Stevens
> Why not do it the other way? "If you don't hear from me for 2 minutes, > do a switchover". Then all you have to do is _not_ to send a packet -- > easier to do. > > Anything else seems overkill. > Pavel Because in some of the scenarios, including ours, it isn't a

Re: [RFC][PATCH 0/3] TCP/IP Critical socket communication mechanism

2005-12-21 Thread Pavel Machek
Hi! > > If it is only one place, why not pre-allocate one "I'm sick now" > > skb and hold onto it. Any bigger solution seems to snowball into > > a huge mess. > > But the problem is even sending/receiving a single packet can cause > multiple dynamic allocations in the networking path all the way

Re: [RFC][PATCH 0/3] TCP/IP Critical socket communication mechanism

2005-12-16 Thread Sridhar Samudrala
On Fri, 2005-12-16 at 09:48 -0800, Stephen Hemminger wrote: > On Thu, 15 Dec 2005 18:09:22 -0800 > Sridhar Samudrala <[EMAIL PROTECTED]> wrote: > > > On Thu, 2005-12-15 at 00:21 -0800, David S. Miller wrote: > > > From: Sridhar Samudrala <[EMAIL PROTECTED]> > > > Date: Wed, 14 Dec 2005 23:37:37 -0

Re: [RFC][PATCH 0/3] TCP/IP Critical socket communication mechanism

2005-12-16 Thread Stephen Hemminger
On Thu, 15 Dec 2005 18:09:22 -0800 Sridhar Samudrala <[EMAIL PROTECTED]> wrote: > On Thu, 2005-12-15 at 00:21 -0800, David S. Miller wrote: > > From: Sridhar Samudrala <[EMAIL PROTECTED]> > > Date: Wed, 14 Dec 2005 23:37:37 -0800 (PST) > > > > > Instead, you seem to be suggesting in_emergency to

Re: [RFC][PATCH 0/3] TCP/IP Critical socket communication mechanism

2005-12-16 Thread Bodo Eggert
David S. Miller <[EMAIL PROTECTED]> wrote: > The idea to mark, for example, IPSEC key management daemon's sockets > as critical is flawed, because the key management daemon could hit a > swap page over the iSCSI device. Don't even start with the idea to > lock the IPSEC key management daemon into

Re: [RFC][PATCH 0/3] TCP/IP Critical socket communication mechanism

2005-12-15 Thread Sridhar Samudrala
On Thu, 2005-12-15 at 00:21 -0800, David S. Miller wrote: > From: Sridhar Samudrala <[EMAIL PROTECTED]> > Date: Wed, 14 Dec 2005 23:37:37 -0800 (PST) > > > Instead, you seem to be suggesting in_emergency to be set dynamically > > when we are about to run out of ATOMIC memory. Is this right? > > N

Re: [RFC][PATCH 0/3] TCP/IP Critical socket communication mechanism

2005-12-15 Thread jamal
On Thu, 2005-15-12 at 14:07 +0100, Arjan van de Ven wrote: > On Thu, 2005-12-15 at 08:00 -0500, jamal wrote: > > The big hole punched by DaveM is that of dependencies: a http tcp > > connection is tied to ICMP or the IPSEC example given; so you need a lot > > more intelligence than just what your

Re: [RFC][PATCH 0/3] TCP/IP Critical socket communication mechanism

2005-12-15 Thread Arjan van de Ven
On Thu, 2005-12-15 at 08:00 -0500, jamal wrote: > On Thu, 2005-15-12 at 12:47 +0100, Arjan van de Ven wrote: > > > > > > You are using the wrong hammer to crack your nut. > > > You should instead approach your problem of why the ARP entry gets lost. > > > For example, you could give as critical pr

Re: [RFC][PATCH 0/3] TCP/IP Critical socket communication mechanism

2005-12-15 Thread jamal
On Thu, 2005-15-12 at 12:47 +0100, Arjan van de Ven wrote: > > > > You are using the wrong hammer to crack your nut. > > You should instead approach your problem of why the ARP entry gets lost. > > For example, you could give as critical priority to your TCP session, > > but that still won't cure

Re: [RFC][PATCH 0/3] TCP/IP Critical socket communication mechanism

2005-12-15 Thread Arjan van de Ven
> > You are using the wrong hammer to crack your nut. > You should instead approach your problem of why the ARP entry gets lost. > For example, you could give as critical priority to your TCP session, > but that still won't cure your ARP problem. > I would suggest that the best way to cure your

Re: [RFC][PATCH 0/3] TCP/IP Critical socket communication mechanism

2005-12-15 Thread James Courtier-Dutton
Mitchell Blank Jr wrote: James Courtier-Dutton wrote: When I had the conversation with Matt at KS, the problem we were trying to solve was "Memory pressure with network attached swap space". s/swap space/writable filesystems/ You can hit these problems even if you have no swap. Too much of

Re: [RFC][PATCH 0/3] TCP/IP Critical socket communication mechanism

2005-12-15 Thread David Stevens
"David S. Miller" <[EMAIL PROTECTED]> wrote on 12/15/2005 12:58:05 AM: > From: David Stevens <[EMAIL PROTECTED]> > Date: Thu, 15 Dec 2005 00:44:52 -0800 > > > In our internal discussions > > I really wish this hadn't been discussed internally before being > implemented. Any such internal discus

Re: [RFC][PATCH 0/3] TCP/IP Critical socket communication mechanism

2005-12-15 Thread David S. Miller
From: David Stevens <[EMAIL PROTECTED]> Date: Thu, 15 Dec 2005 00:44:52 -0800 > In our internal discussions I really wish this hadn't been discussed internally before being implemented. Any such internal discussions are lost completely upon the community that ends up reviewing such a core and in

Re: [RFC][PATCH 0/3] TCP/IP Critical socket communication mechanism

2005-12-15 Thread David Stevens
> Also, all this stuff is just a band aid because linux OOM behavior is so > fucked up. In our internal discussions, characterizing this as "OOM" came up a lot, and I don't think of it as that at all. OOM is exactly what the scheme is trying to avoid! The actual situation we have in mind is a swa

Re: [RFC][PATCH 0/3] TCP/IP Critical socket communication mechanism

2005-12-15 Thread Arjan van de Ven
On Thu, 2005-12-15 at 00:21 -0800, David S. Miller wrote: > From: Sridhar Samudrala <[EMAIL PROTECTED]> > Date: Wed, 14 Dec 2005 23:37:37 -0800 (PST) > > > Instead, you seem to be suggesting in_emergency to be set dynamically > > when we are about to run out of ATOMIC memory. Is this right? > > N

Re: [RFC][PATCH 0/3] TCP/IP Critical socket communication mechanism

2005-12-15 Thread David S. Miller
From: Sridhar Samudrala <[EMAIL PROTECTED]> Date: Wed, 14 Dec 2005 23:37:37 -0800 (PST) > Instead, you seem to be suggesting in_emergency to be set dynamically > when we are about to run out of ATOMIC memory. Is this right? Not when we run out, but rather when we reach some low water mark, the "c

Re: [RFC][PATCH 0/3] TCP/IP Critical socket communication mechanism

2005-12-14 Thread Sridhar Samudrala
On Wed, 14 Dec 2005, David S. Miller wrote: > From: Matt Mackall <[EMAIL PROTECTED]> > Date: Wed, 14 Dec 2005 19:39:37 -0800 > > > I think we need a global receive pool and per-socket send pools. > > Mind telling everyone how you plan to make use of the global receive > pool when the allocation ha

Re: [RFC][PATCH 0/3] TCP/IP Critical socket communication mechanism

2005-12-14 Thread Stephen Hemminger
On Thu, 15 Dec 2005 06:42:45 +0100 Andi Kleen <[EMAIL PROTECTED]> wrote: > On Wed, Dec 14, 2005 at 08:30:23PM -0800, David S. Miller wrote: > > From: Matt Mackall <[EMAIL PROTECTED]> > > Date: Wed, 14 Dec 2005 19:39:37 -0800 > > > > > I think we need a global receive pool and per-socket send pool

Re: [RFC][PATCH 0/3] TCP/IP Critical socket communication mechanism

2005-12-14 Thread Stephen Hemminger
On Wed, 14 Dec 2005 21:23:09 -0800 (PST) "David S. Miller" <[EMAIL PROTECTED]> wrote: > From: Matt Mackall <[EMAIL PROTECTED]> > Date: Wed, 14 Dec 2005 21:02:50 -0800 > > > There needs to be two rules: > > > > iff global memory critical flag is set > > - allocate from the global critical receive

Re: [RFC][PATCH 0/3] TCP/IP Critical socket communication mechanism

2005-12-14 Thread Matt Mackall
On Wed, Dec 14, 2005 at 09:23:09PM -0800, David S. Miller wrote: > From: Matt Mackall <[EMAIL PROTECTED]> > Date: Wed, 14 Dec 2005 21:02:50 -0800 > > > There needs to be two rules: > > > > iff global memory critical flag is set > > - allocate from the global critical receive pool on receive > > -

Re: [RFC][PATCH 0/3] TCP/IP Critical socket communication mechanism

2005-12-14 Thread Nick Piggin
David S. Miller wrote: From: Matt Mackall <[EMAIL PROTECTED]> Date: Wed, 14 Dec 2005 21:02:50 -0800 There needs to be two rules: iff global memory critical flag is set - allocate from the global critical receive pool on receive - return packet to global pool if not destined for a socket with

Re: [RFC][PATCH 0/3] TCP/IP Critical socket communication mechanism

2005-12-14 Thread Andi Kleen
On Wed, Dec 14, 2005 at 08:30:23PM -0800, David S. Miller wrote: > From: Matt Mackall <[EMAIL PROTECTED]> > Date: Wed, 14 Dec 2005 19:39:37 -0800 > > > I think we need a global receive pool and per-socket send pools. > > Mind telling everyone how you plan to make use of the global receive > pool

Re: [RFC][PATCH 0/3] TCP/IP Critical socket communication mechanism

2005-12-14 Thread David S. Miller
From: Matt Mackall <[EMAIL PROTECTED]> Date: Wed, 14 Dec 2005 21:02:50 -0800 > There needs to be two rules: > > iff global memory critical flag is set > - allocate from the global critical receive pool on receive > - return packet to global pool if not destined for a socket with an > attached s

Re: [RFC][PATCH 0/3] TCP/IP Critical socket communication mechanism

2005-12-14 Thread Matt Mackall
On Wed, Dec 14, 2005 at 08:30:23PM -0800, David S. Miller wrote: > From: Matt Mackall <[EMAIL PROTECTED]> > Date: Wed, 14 Dec 2005 19:39:37 -0800 > > > I think we need a global receive pool and per-socket send pools. > > Mind telling everyone how you plan to make use of the global receive > pool

Re: [RFC][PATCH 0/3] TCP/IP Critical socket communication mechanism

2005-12-14 Thread David S. Miller
From: Matt Mackall <[EMAIL PROTECTED]> Date: Wed, 14 Dec 2005 19:39:37 -0800 > I think we need a global receive pool and per-socket send pools. Mind telling everyone how you plan to make use of the global receive pool when the allocation happens in the device driver and we have no idea which sock

Re: [RFC][PATCH 0/3] TCP/IP Critical socket communication mechanism

2005-12-14 Thread Matt Mackall
On Wed, Dec 14, 2005 at 09:55:45AM -0800, Sridhar Samudrala wrote: > On Wed, 2005-12-14 at 10:22 +0100, Andi Kleen wrote: > > > I would appreciate any feedback or comments on this approach. > > > > Maybe I'm missing something but wouldn't you need an own critical > > pool (or at least reservation)

Re: [RFC][PATCH 0/3] TCP/IP Critical socket communication mechanism

2005-12-14 Thread Mitchell Blank Jr
James Courtier-Dutton wrote: > When I had the conversation with Matt at KS, the problem we were trying > to solve was "Memory pressure with network attached swap space". s/swap space/writable filesystems/ You can hit these problems even if you have no swap. Too much of the memory becomes filled

Re: [RFC][PATCH 0/3] TCP/IP Critical socket communication mechanism

2005-12-14 Thread Sridhar Samudrala
On Wed, 2005-12-14 at 14:39 -0800, Ben Greear wrote: > James Courtier-Dutton wrote: > > > Have you actually thought about what would happen in a real world senario? > > There is no real world requirement for this sort of user land feature. > > In memory pressure mode, you don't care about user app

Re: [RFC][PATCH 0/3] TCP/IP Critical socket communication mechanism

2005-12-14 Thread Ben Greear
James Courtier-Dutton wrote: Have you actually thought about what would happen in a real world senario? There is no real world requirement for this sort of user land feature. In memory pressure mode, you don't care about user applications. In fact, under memory pressure no user applications are

Re: [RFC][PATCH 0/3] TCP/IP Critical socket communication mechanism

2005-12-14 Thread James Courtier-Dutton
Sridhar Samudrala wrote: On Wed, 2005-12-14 at 20:49 +, James Courtier-Dutton wrote: Jesper Juhl wrote: On 12/14/05, Sridhar Samudrala <[EMAIL PROTECTED]> wrote: These set of patches provide a TCP/IP emergency communication mechanism that could be used to guarantee high priority commun

Re: [RFC][PATCH 0/3] TCP/IP Critical socket communication mechanism

2005-12-14 Thread Sridhar Samudrala
On Wed, 2005-12-14 at 20:49 +, James Courtier-Dutton wrote: > Jesper Juhl wrote: > > On 12/14/05, Sridhar Samudrala <[EMAIL PROTECTED]> wrote: > > > >>These set of patches provide a TCP/IP emergency communication mechanism that > >>could be used to guarantee high priority communications over a

Re: [RFC][PATCH 0/3] TCP/IP Critical socket communication mechanism

2005-12-14 Thread James Courtier-Dutton
Jesper Juhl wrote: On 12/14/05, Sridhar Samudrala <[EMAIL PROTECTED]> wrote: These set of patches provide a TCP/IP emergency communication mechanism that could be used to guarantee high priority communications over a critical socket to succeed even under very low memory conditions that last for

Re: [RFC][PATCH 0/3] TCP/IP Critical socket communication mechanism

2005-12-14 Thread Ben Greear
Jesper Juhl wrote: To be a little serious, it sounds like something that could be used to cause trouble and something that will lose its usefulness once enough people start using it (for valid or invalid reasons), so what's the point... It could easily be a user-configurable option in an appli

Re: [RFC][PATCH 0/3] TCP/IP Critical socket communication mechanism

2005-12-14 Thread Jesper Juhl
On 12/14/05, Sridhar Samudrala <[EMAIL PROTECTED]> wrote: > > These set of patches provide a TCP/IP emergency communication mechanism that > could be used to guarantee high priority communications over a critical socket > to succeed even under very low memory conditions that last for a couple of >

Re: [RFC][PATCH 0/3] TCP/IP Critical socket communication mechanism

2005-12-14 Thread David Stevens
> It has a lot > more users that compete true, but likely the set of GFP_CRITICAL users > would grow over time too and it would develop the same problem. No, because the critical set is determined by the user (by setting the socket flag). The receive side has some things marked as

Re: [RFC][PATCH 0/3] TCP/IP Critical socket communication mechanism

2005-12-14 Thread Andi Kleen
> Here we are assuming that the pre-allocated critical page pool is big enough > to satisfy the requirements of all the critical sockets. That seems like a lot of assumptions. Is it really better than the existing GFP_ATOMIC which works basically the same? It has a lot more users that compete tr

Re: [RFC][PATCH 0/3] TCP/IP Critical socket communication mechanism

2005-12-14 Thread Sridhar Samudrala
On Wed, 2005-12-14 at 10:22 +0100, Andi Kleen wrote: > > I would appreciate any feedback or comments on this approach. > > Maybe I'm missing something but wouldn't you need an own critical > pool (or at least reservation) for each socket to be safe against deadlocks? > > Otherwise if a critical s

Re: [RFC][PATCH 0/3] TCP/IP Critical socket communication mechanism

2005-12-14 Thread Andi Kleen
> I would appreciate any feedback or comments on this approach. Maybe I'm missing something but wouldn't you need an own critical pool (or at least reservation) for each socket to be safe against deadlocks? Otherwise if a critical sockets needs e.g. 2 pages to finish something and 2 critical sock

[RFC][PATCH 0/3] TCP/IP Critical socket communication mechanism

2005-12-14 Thread Sridhar Samudrala
These set of patches provide a TCP/IP emergency communication mechanism that could be used to guarantee high priority communications over a critical socket to succeed even under very low memory conditions that last for a couple of minutes. It uses the critical page pool facility provided by Matt's