Re: [R] Appropriateness of R functions for multicore

2013-08-23 Thread R. Michael Weylandt
On Mon, Aug 19, 2013 at 2:08 PM, Patrick Connolly wrote: > On Sat, 17-Aug-2013 at 05:09PM -0700, Jeff Newmiller wrote: > > > |> In most threaded multitasking environments it is not safe to > |> perform IO in multiple threads. In general you will have difficulty > |> performing IO in parallel proce

Re: [R] Appropriateness of R functions for multicore

2013-08-20 Thread Hadley Wickham
> For tasks which don't involve I/O but fail with mclapply, how does one > work out where the problem is? The handy browser() function which > allows for interactive diagnosis won't work with parallel jobs. > > What other approaches can one use? ?dump.frames - interactive debugging after the fac

Re: [R] Appropriateness of R functions for multicore

2013-08-19 Thread Prof Brian Ripley
On 20/08/2013 03:13, Hopkins, Bill wrote: I wrap functions to run via multicore with tryCatch() to gather stats on failure rate and capture state. I'm still interested in how/whether core fuctions were verified as being threadsafe. What does 'threads' have to do with this? Multicore forks

Re: [R] Appropriateness of R functions for multicore

2013-08-19 Thread Hopkins, Bill
I wrap functions to run via multicore with tryCatch() to gather stats on failure rate and capture state. I'm still interested in how/whether core fuctions were verified as being threadsafe. Bill Hopkins Written using a virtual Android keyboard... -- Original message -- From: Jef

Re: [R] Appropriateness of R functions for multicore

2013-08-19 Thread Daniel Nordlund
f Of Jeff Newmiller > Sent: Monday, August 19, 2013 4:19 PM > To: Patrick Connolly > Cc: r-help@R-project.org; Hopkins,Bill > Subject: Re: [R] Appropriateness of R functions for multicore > > I don't know... I suppose it depends how it fails. I recommend that you > restr

Re: [R] Appropriateness of R functions for multicore

2013-08-19 Thread Jeff Newmiller
I don't know... I suppose it depends how it fails. I recommend that you restrict yourself to using only the data that was passed as parameters to your parallel function. You may be able to tackle parts of the task and return only those partial results to confirm how far through the code you can

Re: [R] Appropriateness of R functions for multicore

2013-08-19 Thread Patrick Connolly
On Sat, 17-Aug-2013 at 05:09PM -0700, Jeff Newmiller wrote: |> In most threaded multitasking environments it is not safe to |> perform IO in multiple threads. In general you will have difficulty |> performing IO in parallel processing so it is best to let the |> master hand out data to worker tas

Re: [R] Appropriateness of R functions for multicore

2013-08-17 Thread Jeff Newmiller
In most threaded multitasking environments it is not safe to perform IO in multiple threads. In general you will have difficulty performing IO in parallel processing so it is best to let the master hand out data to worker tasks and gather results from them for storage. Keep in mind that just bec