I work with images with a lot of processing done in C code. Quite often I allocate memory there up to several gigs in chunks of 10-15 Mb each plus hundreds of protected dims, names etc. I had a similar problem only once when due to some erroneous use of an external library, internally created objects were not freed correctly. Otherwise, after correcting this, I never have seen any slow down on large number of objects created and manipulated. And then, it was so difficult to track the memory leak that I would really suggest to double and triple check all the memory allocations. Your code does not use any temp files? This could be a real pain. Oleg
Dirk Eddelbuettel wrote: > On 25 May 2007 at 19:12, Michael Braun wrote: > | So I'm stuck. Can anyone help? > > It sounds like a memory issue. Your memory may just get fragmented. One tool > that may help you find leaks is valgrind -- see the 'R Extensions' manual. I > can also recommend the visualisers like kcachegrind (part of KDE). > > But it may not be a leak. I found that R just doesn't cope well with many > large memory allocations and releases -- I often loop over data request that > I subset and process. This drives my 'peak' memory use to 1.5 or 1.7gb on > 32bit/multicore machine with 4gb, 6gb or 8gb (but 32bit leading to the hard > 3gb per process limit) . And I just can't loop over many such task. So I > now use the littler frontend to script this, dump the processed chunks as > Rdata files and later re-read the pieces. That works reliably. > > So one think you could try is to dump your data in 'gsl ready' format from R, > quit R, leave it out of the equation and then see if what happens if you do > the iterations in only GSL and your code. > > Hth, Dirk > -- Dr Oleg Sklyar | EBI-EMBL, Cambridge CB10 1SD, UK | +44-1223-494466 ______________________________________________ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
