In case anyone is concerned that this regression will affect them, the code was reverted to the 2.14.x behavior by
------------------------------------------------------------------------ r58842 | ripley | 2012-03-26 08:12:43 -0400 (Mon, 26 Mar 2012) | 1 line Changed paths: M /branches/R-2-15-branch/doc/NEWS.Rd M /branches/R-2-15-branch/src/library/parallel/R/unix/forkCluster.R M /branches/R-2-15-branch/src/library/parallel/R/unix/mcfork.R revert to XDR serialization for 2.15.0 ------------------------------------------------------------------------ Thanks, Ben > I am experiencing a problem related to serialization behavior in > 2.15.0 beta (binary installed from Debian unstable) and 2.16.0 (from > svn) that is not present in 2.14.2 (binary from Debian testing). > > I don't fully understand the problem. Also, I tried but have not yet > been able to create a small, self-contained example that reproduces > the problem. However, I do have a large, not self-contained example, > which requires an alpha version (not yet on CRAN) of the mi package > (the mi package on CRAN would not exhibit this issue). Anyone > interested in reproducing the problem can follow the readme.txt file > in this directory: > > http://www.columbia.edu/~bg2382/mi/serialization/ > > I track r-devel with git-svn and was able to git bisect to svn commit r58219 > > commit 799102bd9d0266fe89c3120981decf0b1f17ef11 > Author: ripley <ripley at 00db46b3-68df-0310-9c12-caf00c1e9a41> > Date: Sat Jan 28 15:02:34 2012 +0000 > > make use of non-xdr serialization;. > > although this commit could merely expose the problem rather than cause it. > > The problem occurs when the FUN called by mclapply() in the parallel > package returns a S4 object that contains a slot (called X) that is a > large matrix, specifically a "model matrix" similar to that produced > by glm(). Some columns of this matrix get corrupted with wrong values > (usually zero, but sometimes NaN or 10^300ish), which can be seen by > examining X right before FUN returns (to mclapply()'s environment) and > comparing to the "same" X after mclapply() returns to the calling > environment. > > Part of svn commit r58219 is this hunk > > diff --git a/src/library/parallel/R/unix/mcfork.R > b/src/library/parallel/R/unix/mcfork.R > index 8e27534..4f92193 100644 > --- a/src/library/parallel/R/unix/mcfork.R > +++ b/src/library/parallel/R/unix/mcfork.R > @@ -82,7 +82,8 @@ mckill <- function(process, signal = 2L) > ## used by mcparallel, mclapply > sendMaster <- function(what) > { > - if (!is.raw(what)) what <- serialize(what, NULL, FALSE) > + # This is talking to the same machine, so no point in using xdr. > + if (!is.raw(what)) what <- serialize(what, NULL, xdr = FALSE) > .Call(C_mc_send_master, what, PACKAGE = "parallel") > } > > Contrary to the comment, I have found that if I specify xdr = TRUE, I > get the expected (non-corrupted X slot) behavior in 2.16.0, even > though it is forking locally on my 64bit Debian laptop with a little > endian i7 processor, whose specs are > > goodrich at CYBERPOWERPC:/tmp/serialization$ cat /proc/cpuinfo > processor : 0 > vendor_id : GenuineIntel > cpu family : 6 > model : 42 > model name : Intel(R) Core(TM) i7-2630QM CPU @ 2.00GHz > stepping : 7 > microcode : 0x17 > cpu MHz : 800.000 > cache size : 6144 KB > physical id : 0 > siblings : 8 > core id : 0 > cpu cores : 4 > apicid : 0 > initial apicid : 0 > fpu : yes > fpu_exception : yes > cpuid level : 13 > wp : yes > flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge > mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe > syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl > xtopology nonstop_tsc aperfmperf pni pclmulqdq dtes64 monitor ds_cpl > vmx est tm2 ssse3 cx16 xtpr pdcm pcid sse4_1 sse4_2 x2apic popcnt > tsc_deadline_timer xsave avx lahf_lm ida arat epb xsaveopt pln pts dts > tpr_shadow vnmi flexpriority ept vpid > bogomips : 3990.83 > clflush size : 64 > cache_alignment : 64 > address sizes : 36 bits physical, 48 bits virtual > power management: > > ... > > processor : 7 > [same as processor 0] > > So, to summarize I get the good behavior on R 2.14.2 when using > mclapply(), on 2.15.0 beta when using lapply(), and on 2.16.0 using > mclapply() iff I patch in xdr = TRUE in sendMaster(). I get the bad > behavior on 2.15.0 beta and unpatched 2.16.0 when using mclapply(). > > My session info: > >> sessionInfo() > R version 2.15.0 beta (2012-03-16 r58769) > Platform: x86_64-pc-linux-gnu (64-bit) > > locale: > [1] LC_CTYPE=en_US.UTF-8 LC_NUMERIC=C > [3] LC_TIME=en_US.UTF-8 LC_COLLATE=en_US.UTF-8 > [5] LC_MONETARY=en_US.UTF-8 LC_MESSAGES=en_US.UTF-8 > [7] LC_PAPER=C LC_NAME=C > [9] LC_ADDRESS=C LC_TELEPHONE=C > [11] LC_MEASUREMENT=en_US.UTF-8 LC_IDENTIFICATION=C > > attached base packages: > [1] stats4 stats graphics grDevices utils datasets methods > [8] base > > other attached packages: > [1] mi_0.9-83 bigmemory_4.2.11 arm_1.5-03 foreign_0.8-49 > [5] abind_1.4-0 R2WinBUGS_2.1-18 coda_0.14-5 lme4_0.999375-42 > [9] Matrix_1.0-4 lattice_0.20-0 MASS_7.3-17 > > loaded via a namespace (and not attached): > [1] grid_2.15.0 nlme_3.1-103 > > Thanks, > Ben ______________________________________________ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
