Re: [R] bam (mgcv) not using the specified number of cores

2014-08-25 Thread Simon Wood
oops, I just realised that the fix referred to below is in mgcv 1.8-3 - not yet on CRAN. On 25/08/14 13:20, Simon Wood wrote: Hi Andrew, In some of the shots you sent then top was reporting several cores working. I think the problem here may be to do with the way bam is parallelized - at prese

Re: [R] bam (mgcv) not using the specified number of cores

2014-08-25 Thread Simon Wood
Hi Andrew, In some of the shots you sent then top was reporting several cores working. I think the problem here may be to do with the way bam is parallelized - at present only part of the computation is in parallel - the model matrix QR decomposition part. The smoothing parameter selection is

Re: [R] bam (mgcv) not using the specified number of cores

2014-08-21 Thread Andrew Crane-Droesch
Hi Simon, (resending with all images as imgur so as to not bounce from list) Thanks for the reply. I've tried to reproduce the error, but I don't know how to show output from `top` any other way than with screenshots, so please excuse that. Here are screenshots of what happens when I run wi

Re: [R] bam (mgcv) not using the specified number of cores

2014-08-21 Thread Simon Wood
Hi Andrew, Could you provide a bit more information, please. In particular the results of sessionInfo() and the code that caused this weird behaviour (+ an indication of dataset size). best, Simon On 21/08/14 12:53, Andrew Crane-Droesch wrote: I am getting strange behavior when trying to fi

[R] bam (mgcv) not using the specified number of cores

2014-08-21 Thread Andrew Crane-Droesch
I am getting strange behavior when trying to fit models to large datasets using bam. I am working on a 4-core machine, but I think that there may be 2 physical cores that the computer uses as 4 cores in some sense that I don't understand. When I run the bam using makeCluster(3), the model run