Looking at parLapplyLB, one sees that it takes in X and then passes splitList(X, length(cl)) to clusterApplyLB, which then calls dynamicClusterApply. Thus while dynamicClusterApply does handle tasks in a load-balancing fashion, sending out individual tasks as previous tasks complete, parLapplyLB preempts that by splitting up the tasks in advance into as many groups of tasks as there are cluster processes. This seems to defeat the purpose of load-balancing and of the manner in which dynamicClusterApply is coded.
This question basically repeats a question posed in 2013 -- see http://r.789695.n4.nabble.com/parLapplyLB-Load-balancing-tt4671848.html I'm reposting because there doesn't seem to have been any response to the previous posting, and it looks like the issue is still present in R 3.1 so it seems asking again if this is the intended behavior of parLapplyLB. I'm using R 3.1.1 and the 3.1.1 version of the parallel package under Ubuntu 14.04, but the code appears to be the same in R-devel. Chris ---------------------------------------------------------------------------------------------- Chris Paciorek Statistical Computing Consultant Statistical Computing Facility, Econometrics Laboratory, Berkeley Research Computing Office: 495 Evans Hall Email: pacio...@stat.berkeley.edu Mailing Address: Voice: 510-842-6670 Department of Statistics Fax: 510-642-7892 367 Evans Hall Skype: cjpaciorek University of California, Berkeley WWW: www.stat.berkeley.edu/~paciorek Berkeley, CA 94720 USA Permanent forward: pacio...@alumni.cmu.edu ______________________________________________ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
