I'm not sure this would be of any help, but perhaps you should look into UBC ( http://wagerlabs.com/blog/2008/03/04/hacking-the-mac-osx-unified-buffer-cache/ ).
On Thu, Mar 5, 2015 at 1:45 AM, <[email protected]> wrote: > Send Coreaudio-api mailing list submissions to > [email protected] > > To subscribe or unsubscribe via the World Wide Web, visit > https://lists.apple.com/mailman/listinfo/coreaudio-api > or, via email, send a message with subject or body 'help' to > [email protected] > > You can reach the person managing the list at > [email protected] > > When replying, please edit your Subject line so it is more specific > than "Re: Contents of Coreaudio-api digest..." > > > Today's Topics: > > 1. large scale (audio) file I/O on OS X : help or insight > requested (Paul Davis) > > > ---------------------------------------------------------------------- > > Message: 1 > Date: Tue, 03 Mar 2015 18:20:31 -0600 > From: Paul Davis <[email protected]> > To: coreaudio-api <[email protected]> > Subject: large scale (audio) file I/O on OS X : help or insight > requested > Message-ID: > < > cafa_ckm+y0bb5_wdhfz1x_y0zkz95khkqdbcvck3jgmt0ny...@mail.gmail.com> > Content-Type: text/plain; charset="utf-8" > > Many readers of the list will know who I am, but for those who don't let me > start by just mentioning that I'm the lead developer and designer of > Ardour, an open source GPL'ed DAW for Linux, OS X and Windows, and also the > original author of JACK, a cross-platform API for low latency, realtime > audio and MIDI, including inter-application and network routing. > > Over in Ardour-land, we've wrestled for a couple of years now with less > than ideal support for OS X and are just finally beginning to resolve > many/most of the areas where we were falling down. > > But there's an area where we've really run into a brick wall, despite the > extremely deep and amazingly talented pool of people we have as developers, > people who know *nix operating systems inside out and upside down. > > That area is: getting acceptable disk I/O bandwidth on OS X when reading > many files. For comparison: we can easily handle 1024 tracks on a single > spinning disk on Linux. We can get close to similar performance on Windows. > We can even (gasp!) get in the same ballpark on Windows running as a guest > OS on OS X (i.e. the entire Windows filesystem is just one file from OS X's > perspective). > > But on OS X itself, we are unable to get consistent, reliable performance > when reading many files at once. Our code is, of course, entirely > cross-platform, and so we are not using any Apple specific APIs (or > Linux-specific or Windows-specific). The same code works well for recording > (writing) files (though OS X does still perform worse than other > platforms). > > We've used various dtrace-based tools to try to analyse what is going on: > these only confirmed for us that we have a problem, but provided no > insights into what. > > Things became so inexplicable that we decided to break out the problem from > Ardour itself and write a small test application that would let us easily > collect data from many different systems. We've now tested on the order of > a half-dozen different OS X systems, and they all show the same basic bad > behaviour: > > * heavy dependence of sustained streaming bandwidth on the number of > files being read > (i.e. sustained streaming bandwidth is high when reading 10 > files, but can be very low > when reading 128 files; This dependence is low on Windows > and non-existent > on Linux) > > * periodic drops of sustained streaming bandwidth of as much a factor > of 50, which can > last for several seconds (e.g a disk that can peak at 100MB/sec > fails to deliver > better than 5MB/sec for a noticeable period). > > * a requirement to read larger blocksizes to get the same bandwidth > than on other platforms > > Our test application is small, less than 130 lines of code. It uses the > POSIX API to read a specified blocksize from each of N files, and reports > on the observed I/O bandwidth. It comes with a companion shell script > (about 60 lines) which sets up the files to be read and then runs the > executable with each of series of (specified) blocksizes. > > I've attached both files (source code for the executable, plus the shell > script). The executable has an unfortunate reliance right now on glib in > order to get the cross-platform g_get_monotonic_time(). If you want to run > them, the build command is at the top of the executable, and then you would > just do: > > ./run-readtest.sh -d FOLDER-FOR-TEST-FILES -n NUMBER-OF-FILES -f > FILESIZE-IN-BYTES blocksize1 blocksize2 .... > > (You'll also need some pretty large-ish chunk of diskspace available for > the test files). The blocksize list is typically powers of 2 starting at > 16348 and going up to 4MB. We only see problems with NUMBER-OF-FILES is on > the order of 128 or above. To be a useful test, the filesizes need to be at > least 10MB or so. The full test takes several (2-20) minutes depending on > the overall speed of your disk. > > But you don't have to run them: you could just read the source code > executable to see if any lightbulbs go off. Why would this code fail so > badly once the number of files gets up to 100+? Why can we predictably > make any version of OS X able to get barely 5MB/sec from a disk that can > sustain 100MB/sec? Are there some tricks to making this sort of thing work > on OS X that anyone is aware of? Keep in mind that this same code works > reliably, solidly and predictably on Windows and Linux (and even works > reliably on Windows as a guest OS on OS X). > > We do know, by simple inspection, that other DAWs, from Reaper to Logic, > appear to have solved this problem. Alas, unlike Ardour, they don't > generally share their code, so we have no idea whether they did something > clever, or whether we are doing something stupid. > > I, and rest of the Ardour community, would be very grateful for any > insights anyone can offer. > > thanks, > --p > -------------- next part -------------- > An HTML attachment was scrubbed... > URL: < > https://lists.apple.com/mailman/private/coreaudio-api/attachments/20150303/ec2826b2/attachment.html > > > -------------- next part -------------- > A non-text attachment was scrubbed... > Name: readtest.c > Type: text/x-csrc > Size: 4120 bytes > Desc: not available > URL: < > https://lists.apple.com/mailman/private/coreaudio-api/attachments/20150303/ec2826b2/attachment.bin > > > -------------- next part -------------- > A non-text attachment was scrubbed... > Name: run-readtest.sh > Type: application/x-sh > Size: 1737 bytes > Desc: not available > URL: < > https://lists.apple.com/mailman/private/coreaudio-api/attachments/20150303/ec2826b2/attachment.sh > > > > ------------------------------ > > _______________________________________________ > Coreaudio-api mailing list > [email protected] > https://lists.apple.com/mailman/listinfo/coreaudio-api > > End of Coreaudio-api Digest, Vol 12, Issue 43 > ********************************************* >
_______________________________________________ Do not post admin requests to the list. They will be ignored. Coreaudio-api mailing list ([email protected]) Help/Unsubscribe/Update your Subscription: https://lists.apple.com/mailman/options/coreaudio-api/archive%40mail-archive.com This email sent to [email protected]
