Ouch. I can say that our implementation doesn't seem to suffer from this problem. Could be there's an issue in the use of PRUS_* v. the socket state we use. The code in my kernel looks like:
in sosend():
if (dontroute)
so->so_options |= SO_DONTROUTE;
if (resid > 0)
so->so_state |= SS_MORETOCOME;
s = splnet(); /* XXX */
error = (*so->so_proto->pr_usrreq)(so,
(flags & MSG_OOB) ? PRU_SENDOOB : PRU_SEND,
top, addr, control);
splx(s);
if (dontroute)
so->so_options &= ~SO_DONTROUTE;
so->so_state &= ~SS_MORETOCOME;
and in tcp_output():
if (len) {
if (len == tp->t_maxseg)
goto send;
if (!(so->so_state & SS_MORETOCOME)) {
if ((idle || tp->t_flags & TF_NODELAY) &&
len + off >= so->so_snd.sb_cc)
goto send;
}
if (tp->t_force)
goto send;
We've subjected this to countless (well, some, I'm sure, can count them :-}) hours of thrashing for web server, file server, and other-server types of uses, and haven't seen any (reports of) leakage like this.
I'll look more closely at the results we see, to verify that we don't have a problem.
Regards,
Justin
From:
Date: 1999-02-11 01:51:17 -0800
To:
Subject: Re: Serious mbuf cluster leak..
Cc:
In-reply-to: "Your message of Wed, 10 Feb 1999 17:49:20 CST."<19990210234920.2a1...@friley-185-205.res.iastate.edu>
X-Mailer: exmh version 2.0.2 2/24/98
X-Loop:
After a while, I have determined the cause of the leak to be the
following commit. Although, I can't seem to find any reason why
it would cause this behavior--reverting these files fixes it.
Any thoughts?
fenner 1999/01/20 09:32:01 PST
Modified files:
sys/kern uipc_socket.c
sys/netinet tcp_output.c tcp_usrreq.c tcp_var.h
sys/sys protosw.h
Log:
Add a flag, passed to pru_send routines, PRUS_MORETOCOME. This
flag means that there is more data to be put into the socket buffer.
Use it in TCP to reduce the interaction between mbuf sizes and the
Nagle algorithm.
Based on: "Justin C. Walker" <jus...@apple.com>'s description of Apple's
fix for this problem.
Revision Changes Path
1.50 +4 -2 src/sys/kern/uipc_socket.c
1.32 +3 -2 src/sys/netinet/tcp_output.c
1.40 +7 -2 src/sys/netinet/tcp_usrreq.c
1.49 +18 -17 src/sys/netinet/tcp_var.h
1.26 +2 -1 src/sys/sys/protosw.h
>I have been seeing a nasty cluster leak in both 3.0 stable and 4.0
>current as of today. Until now, I thougt maybe it was something in
>my driver, athough after much careful looking over my code, it
>simply does not look possible. Also, I downgraded to current of
>Dec 12, and the problem dissappears.
>The odd thing is that the clusters that leak don't seem to be
>attached to mbufs. Or at least there is not a 1-1 ratio. Following
>is netstat output after a while of running netpipe in streaming
>mode. (NPtcp -s; see ftp://ftp.scl.ameslab.gov/pub/netpipe) Also,
>the leak only becomes apparent when the send write size is very
>large--several hundred K to several megabytes.
>Does anyone have any idea what this may be? I really am not sure
>where to look aside from trying prorgressively newer kernels. Also,
>I only have alphas to test on right now..
>puck:~> netstat -m
>211/416 mbufs in use:
> 116 mbufs allocated to data
> 95 mbufs allocated to packet headers
>1674/1688/2048 mbuf clusters in use (current/peak/max)
>3480 Kbytes allocated to network (97% in use)
>0 requests for memory denied
>0 requests for memory delayed
>0 calls to protocol drain routines
>Chris Csanady
>To Unsubscribe: send mail to majord...@freebsd.org
>with "unsubscribe freebsd-net" in the body of the message
To Unsubscribe: send mail to majord...@freebsd.org
with "unsubscribe freebsd-net" in the body of the message
To Unsubscribe: send mail to majord...@freebsd.org with "unsubscribe freebsd-current" in the body of the message
