Re: [R] Parallel Scan of Large File

2010-12-09 Thread Luedde, Mirko
Hi Ryan, the "Getting Started with doMC and foreach" manual tells me that you might have forgotten a registerDoMC() in the example you provided. Best, Mirko __ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE

Re: [R] Parallel Scan of Large File

2010-12-08 Thread Ryan Garner
Hi Jim, Thanks for your insight. I used Linux split to split my large file into smaller partitions. On the server I work on, multipath I/O access is enabled and we use RAID for storage; thus, I don't think I can put each partition on a spindle. I'm able to open multiple files at a time into stdin

Re: [R] Parallel Scan of Large File

2010-12-08 Thread jim holtman
10 17:22:57 -0800 >> From: ryan.steven.gar...@gmail.com >> To: r-help@r-project.org >> Subject: [R] Parallel Scan of Large File >> >> >> Is it possible to parallel scan a large file into a character vector in 1M >> chunks using scan() with the "doMC"

Re: [R] Parallel Scan of Large File

2010-12-08 Thread Mike Marchywka
> Date: Tue, 7 Dec 2010 17:22:57 -0800 > From: ryan.steven.gar...@gmail.com > To: r-help@r-project.org > Subject: [R] Parallel Scan of Large File > > > Is it possible to parallel scan a large file into a character vector in 1

[R] Parallel Scan of Large File

2010-12-07 Thread Ryan Garner
Is it possible to parallel scan a large file into a character vector in 1M chunks using scan() with the "doMC" package? Furthermore, can I specify the tasks for each child? i.e. I'm working on a Linux box with 8 cores and would like to scan in 8M records at time (all 8 cores scan 1M records at a