> Date: Mon, 8 Feb 2010 08:58:46 -0500 > From: Ted Unangst <ted.unan...@gmail.com> > > On Sun, Feb 7, 2010 at 6:11 PM, Theo de Raadt <dera...@cvs.openbsd.org> > wrote: > > I think adding big chunks of sysctl code to such specific applications > > is very un-unix. > > > > If choosing a buffer size is going to be a common operation, it should > > be an API called to "ask what the buffer size should be". If that is > > the case, it will soon over-commit the resources of the machine. So > > then it starts being something you want a system call for. The unix > > practice is to undercommit, and punish userland because userland is > > commonly unefficient, while the kernel is trying to be efficient (as > > much as it can). This scaling undoes that. cp will have it's > > priority increased since it is "efficient", and an un-modified one > > using the standard buffer sizes will be demoted. > > > > So I guess it begs for an extension to the UNIX system calls. > > > > That is a road that requires long consideration, to avoid overcommit. > > You would not want a bunch of applications asking "how much should I > > use", and then doing the allocation later. > > > > That is why I don't like what you are doing in cp. I think your > > heuristic is hasty. > > So how do we move forward?
For one thing, I'd like to see some real benchmarks. Does using a larger buffer really speed up cp? You claim moving a "head" between reads and writes takes time, but so does moving it between reads. And modern drives have caches that may influence the behaviour. Also, your origional diff has a problem. The current cp implementation creates "holes" if it encounters a block of zeroes. The size of that block is coupled to the size of the buffer it uses for reads and writes. Since you made that buffer larger, you'll probably create less holes and waste more space.
