Re: FreeBSD random I/O performance issues

2000-04-04 Thread Kun Limfjordsporter
On Wed, 5 Apr 19100, Matthew Dillon wrote: > :performance -- innd was still unresponsive during the time that the > :random history database files needed updating for somewhere around > :10 to 15 seconds when handling a full feed. > > The innd problem is a different issue entirely, one relat

Re: FreeBSD random I/O performance issues

2000-04-04 Thread Matthew Dillon
:Woo. Okay, so I took the bait and snarfed the k0deZ from midday GMT :02.April, and have built a suitable SMP kernel and world, and that's :what I've been running for one news peering swerver for a spell. : :... : :As I noted in the earlier message I just sent, disabling the write- :behind via t

Re: FreeBSD random I/O performance issues

2000-04-04 Thread Kun Limfjordsporter
On Sun, 2406 Sep 1993, Matthew Dillon wrote: > I've committed an 80% fix for the random seek / write > performance issue. The rest of the fix will come later > when Kirk commits his shared-lock-buffer-cache idea. > > I've committed it into -current and will MFC it into > -st

Re: FreeBSD random I/O performance issues

2000-04-04 Thread tele danmarQ kvindeservice
On Thu, 23 Mar 19100 (not like I'm slow or anything), Matthew Dillon wrote: [regarding random-access database files, such as are used for the INN history files, and FreeBSD 4.0...] > For INN there are several things you can tune in 4.0. First and > foremost you can try turning off the w

Re: FreeBSD random I/O performance issues

2000-04-03 Thread Matthew Dillon
: : : :On Sat, 01 Apr 2000 17:03:24 PST, Matthew Dillon wrote: : :> I've committed it into -current and will MFC it into :> -stable in a week if there aren't any problems. I :> do not intend to MFC it into 3.x. : :Hi Matt, : :I'd like to suggest that a week is not enough. Given that

Re: FreeBSD random I/O performance issues

2000-04-03 Thread Sheldon Hearn
On Sat, 01 Apr 2000 17:03:24 PST, Matthew Dillon wrote: > I've committed it into -current and will MFC it into > -stable in a week if there aren't any problems. I > do not intend to MFC it into 3.x. Hi Matt, I'd like to suggest that a week is not enough. Given that we seem to be

Re: FreeBSD random I/O performance issues

2000-04-01 Thread Matthew Dillon
I've committed an 80% fix for the random seek / write performance issue. The rest of the fix will come later when Kirk commits his shared-lock-buffer-cache idea. I've committed it into -current and will MFC it into -stable in a week if there aren't any problems. I do not

Re: FreeBSD random I/O performance issues

2000-03-26 Thread Richard Wendland
> This is one of the things that made us do so badly > in the benchmarks against NT/Linux last year. If the benchmarks included random I/O I would think so. By creating a small synthetic program exhibiting the problem, I may have obscured the scale of the real-world consequences of this problem.

Re: FreeBSD random I/O performance issues

2000-03-23 Thread Brad Knowles
At 1:17 AM +0100 2000/3/23, FREENIX IS OVERRATED wrote: > Without really tuning anything, after a bit of time, the time needed > to do history lookups drops to microseconds, and as long as a `sync' > isn't needed, innd doesn't get stuck. Theoretically, a sync, where > you are in fact seeking

Re: FreeBSD random I/O performance issues

2000-03-22 Thread Julian Elischer
This is one of the things that made us do so badly in the benchmarks against NT/Linux last year. OBVIOUSLY one should be able to re-read this infoirmation without affecting a pending write. Mike Smith wrote: > > > effects of the I/O being in-progress. If a user program doesn't access > >

Re: FreeBSD random I/O performance issues

2000-03-22 Thread Matthew Dillon
:> out. :> :> What the write-behind code tries to do is to prevent the buffer cache :> from being saturated with dirty buffers and to smooth out disk write :> I/O. It makes the assumption that write-behind data is not typically :> accessed by the program immediately after b

Re: FreeBSD random I/O performance issues

2000-03-22 Thread FREENIX IS OVERRATED
On Wed, 2395 Sep 1993, Matthew Dillon wrote: > It is perfectly ok for dirty blocks to remain in the buffer cache. In > fact, it's *optimal* to leave them in the buffer cache as long as the > buffer cache does not get saturated with them. The buffer cache is > perfectly capable o

Re: FreeBSD random I/O performance issues

2000-03-22 Thread Matthew Dillon
: :> effects of the I/O being in-progress. If a user program doesn't access :> any of the information it recently wrote the whole mechanism winds up :> operating asynchronously in the background. If a user program does, :> then the write behind mechanism breaks down and you get

Re: FreeBSD random I/O performance issues

2000-03-22 Thread Mike Smith
> effects of the I/O being in-progress. If a user program doesn't access > any of the information it recently wrote the whole mechanism winds up > operating asynchronously in the background. If a user program does, > then the write behind mechanism breaks down and you get a stal

Re: FreeBSD random I/O performance issues

2000-03-22 Thread Richard Wendland
> > With sync it's ~20,000 operations matching the total of reads & > > writes. This demonstrates another aspect of the bug, sync behaviour > > should cause 10,000 operations; the reads aren't being cached. > > This isn't quite true. It's 20,000 *write* operations. I put this down > to the mtime

Re: FreeBSD random I/O performance issues

2000-03-21 Thread Chris Wasser
On Wed, Mar 22, 2000 at 12:22:42AM +, Richard Wendland wrote: > Any views gratefully received. A fix would be much better :-) Not sure if my meager setup helps any, but in the interests in providing results to help the cause so-to-speak, I ran the test on my own machine (followed the instruc

Re: FreeBSD random I/O performance issues

2000-03-21 Thread Matthew Dillon
:written immediately which is 8750/1 writes. : :When the write size drops below the filesystem block size then the :clustering code never gets called because the buffers are just marked :dirty and cached. : :I think if we fixed the issue of writing out full blocks this behviour :would stop but

Re: FreeBSD random I/O performance issues

2000-03-21 Thread Paul Richards
Richard Wendland wrote: > I spent a bit of time analysing these results when I first saw them. I don't think it has anything to do with the cache, it has to do with how we write out blocks. > One interesting observation is that for non sync, async or noclusterw > mounts ~8750 I/O operations are

Re: FreeBSD random I/O performance issues

2000-03-21 Thread Matthew Dillon
:Paul Richards said in "Re: patches for test / review": : :> Richard, do you want to post a summary of your tests? : :Well I'd best post the working draft of my report on the issues :I've seen, as I'm not going to have time to work on it in the near :future, and it raises serious performance issu

FreeBSD random I/O performance issues

2000-03-21 Thread Richard Wendland
Paul Richards said in "Re: patches for test / review": > Richard, do you want to post a summary of your tests? Well I'd best post the working draft of my report on the issues I've seen, as I'm not going to have time to work on it in the near future, and it raises serious performance issues that