Re: [fluid-dev] parallel loading of soundfont samples

2021-01-19 Thread Tom M. via fluid-dev
A PR is now ready for this: https://github.com/FluidSynth/fluidsynth/pull/746 Feedback is welcome! Tom ___ fluid-dev mailing list fluid-dev@nongnu.org https://lists.nongnu.org/mailman/listinfo/fluid-dev

Re: [fluid-dev] parallel loading of soundfont samples

2020-10-31 Thread Hamish Moffatt
On 30/10/20 10:57 pm, Tom M. via fluid-dev wrote: I would need to compile with Visual Studio Pls, why do you *need* to compile with that ancient Microsoft product? Couldn't you use MinGW, CygWin, Clang or ICC? To be precise I need to compile my app with MSVC, but I can compile Fluidsynth to

Re: [fluid-dev] parallel loading of soundfont samples

2020-10-31 Thread Tom M. via fluid-dev
> So in my opinion, doing the work to make the sample cache and maybe even the whole soundfont loading mechanism work in parallel seems like a good idea and worth the effort. We need to be careful, as the soundfont loading is exposed via the API. Breaking changes should be avoided. Also, we could

Re: [fluid-dev] parallel loading of soundfont samples

2020-10-31 Thread Marcus Weseloh
Hi, Am Fr., 30. Okt. 2020 um 02:09 Uhr schrieb Hamish Moffatt : > Beautiful! Over 6 times faster. Great, thanks for testing it. > In order to get the best out of modern CPUs we have to use > more threads as the cores aren't getting much faster. I think this is probably the best reason to consi

Re: [fluid-dev] parallel loading of soundfont samples

2020-10-30 Thread Tom M. via fluid-dev
> I would need to compile with Visual Studio Pls, why do you *need* to compile with that ancient Microsoft product? Couldn't you use MinGW, CygWin, Clang or ICC? Tom ___ fluid-dev mailing list fluid-dev@nongnu.org https://lists.nongnu.org/mailman/listi

Re: [fluid-dev] parallel loading of soundfont samples

2020-10-29 Thread Hamish Moffatt
On 30/10/20 11:12 am, Hamish Moffatt wrote: So now I have #3 it's not really important for me to speed up the loading any further, but I posted this just for interest's sake and I'm glad it's generated some discussion, and I think other applications can benefit too. In order to get the best out

Re: [fluid-dev] parallel loading of soundfont samples

2020-10-29 Thread Hamish Moffatt
On 30/10/20 5:42 am, Marcus Weseloh wrote: Hi, for anybody who wants to try the OpenMP parallel loading, I've just pushed the fix for the vorbis load to use separate file pointers to Toms branch here: https://github.com/FluidSynth/fluidsynth/tree/parallelize-sf3-loading Hamish, would be great t

Re: [fluid-dev] parallel loading of soundfont samples

2020-10-29 Thread Marcus Weseloh
Hi, for anybody who wants to try the OpenMP parallel loading, I've just pushed the fix for the vorbis load to use separate file pointers to Toms branch here: https://github.com/FluidSynth/fluidsynth/tree/parallelize-sf3-loading Hamish, would be great to see how that performs on your 6 core machin

Re: [fluid-dev] parallel loading of soundfont samples

2020-10-29 Thread Ben Gonzales via fluid-dev
Hi all. I've been watching this thread from the side-lines. I don't use FluidSynth much now, as I had great difficulty finding high quality soundfonts that fitted my needs and ended up buying a DynaSample XpressO. I like to hear of the continuing development work so I've maintained my members

Re: [fluid-dev] parallel loading of soundfont samples

2020-10-29 Thread Marcus Weseloh
Hi, in addition to what JJC said, we should also take into account that sample entries can point to arbitrary locations in the sample data chunk. So it's theoretically possible that two samples point to the same encoded ogg data, but use different parameters like loop start/end. A parallel loading

Re: [fluid-dev] parallel loading of soundfont samples

2020-10-28 Thread Hamish Moffatt
On 29/10/20 12:50 am, Marcus Weseloh wrote: Hi, Am Mi., 28. Okt. 2020 um 13:10 Uhr schrieb Ceresa Jean-Jacques : Now we should try the suggested test to run 2 fluidsynth console applications (a1, a2) at the same time (not sequencially) and measure the total time. That's a really good idea to

Re: [fluid-dev] parallel loading of soundfont samples

2020-10-28 Thread Marcus Weseloh
Am Mi., 28. Okt. 2020 um 20:11 Uhr schrieb Tom M. via fluid-dev : > 1. The samplecache is guarded by a mutex that prevents naïve > parallelization. Yes, and that is a difficult lock to remove, I think. But as a quick check for a possible speedup factor with parallel loading, your branch is a good

Re: [fluid-dev] parallel loading of soundfont samples

2020-10-28 Thread Ceresa Jean-Jacques
>Currently, there are two problems: 1. The samplecache is guarded by a mutex >that prevents naïve parallelization. 2. If we relax that mutex we get a >strange race condition,... The whole fluid_samplecache_load() need to be atomic and isn't thread safe without the mutex -get_samplecache_entry(

Re: [fluid-dev] parallel loading of soundfont samples

2020-10-28 Thread Tom M. via fluid-dev
An idea to parallelize loading would be to use openMP Tasks. I've made a quick'n dirty implementation of that: https://github.com/FluidSynth/fluidsynth/tree/parallelize-sf3-loading Currently, there are two problems: 1. The samplecache is guarded by a mutex that prevents naïve parallelization. 2.

Re: [fluid-dev] parallel loading of soundfont samples

2020-10-28 Thread Ceresa Jean-Jacques
Hi, >So it looks like parallel loading would be beneficial. Yes, as we notice that with 1 or 2 simultaneous loading the total time is quite the same (6,1s   6,6s) However with 3 and 4 simultaneous loading we see that the total time doesn't stay constant but is growing (8,8s and 10,4s), probaly

Re: [fluid-dev] parallel loading of soundfont samples

2020-10-28 Thread Marcus Weseloh
Hi again, just to highlight the much faster speed of SF2 loading, here is the previous test run again, but this time with the same Soundfont in SF2 format (converted with Polyphone). The resulting SF2 is 204 MB (SF3 was 40MB): $ time ./test.sh 1 real 0m0,366s user 0m0,093s sys 0m0,163s $ time ./

Re: [fluid-dev] parallel loading of soundfont samples

2020-10-28 Thread Marcus Weseloh
Hi, Am Mi., 28. Okt. 2020 um 13:10 Uhr schrieb Ceresa Jean-Jacques : > Now we should try the suggested test to run 2 fluidsynth console applications > (a1, a2) at the same time (not sequencially) and measure the total time. That's a really good idea to quickly test if parallel loading has merit.

Re: [fluid-dev] parallel loading of soundfont samples

2020-10-28 Thread Ceresa Jean-Jacques
Hi, >The SF3 files are taking ~5 seconds each to load. The SF2 files take >400-500ms, despite being 7-10x the size, so the bottleneck is not disk I/O.   Ok, assuming one SF3 file x taking 5 seconds to load using one fluidsynth console application a1. Now we should try the suggested test to ru

Re: [fluid-dev] parallel loading of soundfont samples

2020-10-27 Thread Hamish Moffatt
On 27/10/20 8:37 pm, Marcus Weseloh wrote: Hi again, Am Di., 27. Okt. 2020 um 10:13 Uhr schrieb Hamish Moffatt : I could convert to SF2 on first load, if I had for that. Using a regular compressor like zip on the SF2 is not very effective. Have a look at this pull-request for the MuseScore sf

Re: [fluid-dev] parallel loading of soundfont samples

2020-10-27 Thread Hamish Moffatt
On 28/10/20 2:53 am, Ceresa Jean-Jacques wrote: Hi, >I did look through the sfloader code and I can see it's not easy to restructure it to use multiple threads. >Still, I think parallel loading of the soundfonts would be a good optimization that would benefit every application using Fluidsyn

Re: [fluid-dev] parallel loading of soundfont samples

2020-10-27 Thread Tom M. via fluid-dev
JJC is right, we need to profile it. I can test this with the Intel VTune Amplifier. Which SF3 to use? Is the MuseScore_General.sf3 (38MB) a realistic test-case? Or do we need another (bigger) one? Tom ___ fluid-dev mailing list fluid-dev@nongnu.org htt

Re: [fluid-dev] parallel loading of soundfont samples

2020-10-27 Thread Ceresa Jean-Jacques
Hi, >I did look through the sfloader code and I can see it's not easy to >restructure it to use multiple threads. >Still, I think parallel loading of the soundfonts would be a good optimization that would benefit every application using Fluidsynth.   Before doing any attempt to restructure it

Re: [fluid-dev] parallel loading of soundfont samples

2020-10-27 Thread Marcus Weseloh
Hi again, Am Di., 27. Okt. 2020 um 10:13 Uhr schrieb Hamish Moffatt : > I could convert to SF2 on first load, if I had for that. Using a regular > compressor like zip on the SF2 is not very effective. Have a look at this pull-request for the MuseScore sftools. It adds SF3 to SF2 conversion: http

Re: [fluid-dev] parallel loading of soundfont samples

2020-10-27 Thread Hamish Moffatt
On 27/10/20 6:27 pm, Marcus Weseloh wrote: Thinking about this some more... If I interpret your question as "What is the fastest way to load a whole Soundfont on app startup?", then my answer would be: simply load an SF2 file. Hi Marcus, thanks for considering my idea. As you guessed, I'm usin

Re: [fluid-dev] parallel loading of soundfont samples

2020-10-27 Thread Reinhold Hoffmann
2020 08:28 An: FluidSynth mailing list Betreff: Re: [fluid-dev] parallel loading of soundfont samples Thinking about this some more... If I interpret your question as "What is the fastest way to load a whole Soundfont on app startup?", then my answer would be: simply load an SF2 file.

Re: [fluid-dev] parallel loading of soundfont samples

2020-10-27 Thread Marcus Weseloh
Thinking about this some more... If I interpret your question as "What is the fastest way to load a whole Soundfont on app startup?", then my answer would be: simply load an SF2 file. Loading an SF2 file is really *much!* faster than loading Vorbis-encoded SF3 files. The SF2 loader grabs all samp

Re: [fluid-dev] parallel loading of soundfont samples

2020-10-26 Thread Marcus Weseloh
Hi Hamish, Am Mo., 26. Okt. 2020 um 23:51 Uhr schrieb Hamish Moffatt : > While we're on the topic of soundfont loading, is there any possibility > of speeding it up by using multiple cores? That would require a fairly large code restructuring, the whole loader was written with sequential single-t

[fluid-dev] parallel loading of soundfont samples

2020-10-26 Thread Hamish Moffatt
While we're on the topic of soundfont loading, is there any possibility of speeding it up by using multiple cores? I'm loading two soundfonts in SF3 (vorbis) format, and they take a few seconds each to load. This is now the majority of my app's startup time. Two things would help; either allo