Thanks for this martin. I'll start retooling and let you know how it goes.
Ben Caldwell
Graduate fellow
On Apr 24, 2013 4:34 PM, "Martin Morgan" <mtmor...@fhcrc.org> wrote:
> On 04/24/2013 02:50 PM, Benjamin Caldwell wrote:
>
>> Dear R help,
>>
>> I've what I think is a fairly simple parallel problem, and am getting
>> bogged down in documentation and packages for much more complex
>> situations.
>>
>> I have a big matrix (30^5,5]. I have a function that will act on each row
>> of that matrix sequentially and output the 'best' result from the whole
>> matrix (it compares the result from each row to the last and keeps the
>> 'better' result). I would like to divide that first large matrix into
>> chunks equal to the number of cores I have available to me, and work
>> through each chunk, then output the results from each chunk.
>>
>> I'm really having trouble making head or tail of how to do this on a
>> windows machine - lots of different false starts on several different
>> packages now. Basically, I have the function, and I can of course easily
>> divide the matrix into chunks. I just need a way to process each chunk
>> in parallel (other than opening new R sessions for each core manually).
>>
>> Any help much appreciated - after two days of trying to get this to work
>> I'm pretty burnt out.
>>
>
> Hi Ben -- in your code from this morning you had a function
>
> fitting <- function(ndx.grd=two,dt.grd=**one,ind.vr='ind',rsp.vr='res') {
> ## ... setup
> for(i in 1:length(ndx.grd[,1])){
> ## ... do work
> }
> ## ... collate results
> }
>
> that you're trying to run in parallel. Obviously the ## ... represent
> lines I've removed. When you say something like
>
> y <- foreach(icount(length(two))) %dopar% fitting()
>
> its saying that you want to run fitting() length(two) times. So you're
> actually doing the same thing length(two) times, whereas you really want to
> divide the work thats inside fitting() into chunks, and do those on
> separate cores!
>
> Conceptually what you'd like to do is
>
> fit_one <- function(idx, ndx.grd, dt.grd, ind.vr, rsp.vr) {
> ## ... do work on row idx _ONLY_
> }
>
> and then evaluate with
>
> ## ... setup
> y <-
> foreach (idx = icount(nrow(two)) %dopar% one_fit(idx, two, one, "ind",
> "res")
> ## ... collate
>
> so that fit_one fits just one of your combinations. foreach will worry
> about distributing the work. Make sure that fit_one works first, before
> trying to run this in parallel; your use of try(), trying to fit different
> data types (character, integer, numeric) into a matrix rather than
> data.frame, and the type coercions all indicate that you're fighting with R
> rather than working with it.
>
> Hope that helps,
>
> Martin
>
>
>> Thanks
>>
>> *Ben Caldwell*
>>
>> [[alternative HTML version deleted]]
>>
>> ______________________________**________________
>> R-help@r-project.org mailing list
>> https://stat.ethz.ch/mailman/**listinfo/r-help<https://stat.ethz.ch/mailman/listinfo/r-help>
>> PLEASE do read the posting guide http://www.R-project.org/**
>> posting-guide.html <http://www.R-project.org/posting-guide.html>
>> and provide commented, minimal, self-contained, reproducible code.
>>
>>
>
> --
> Computational Biology / Fred Hutchinson Cancer Research Center
> 1100 Fairview Ave. N.
> PO Box 19024 Seattle, WA 98109
>
> Location: Arnold Building M1 B861
> Phone: (206) 667-2793
>
[[alternative HTML version deleted]]
______________________________________________
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
