Re: [Rd] image adds lines
Yes, thanks. I think I know where this is coming from and will discuss with Duncan Murdoch who was working on this topic recently. Best, Uwe On 13.07.2011 20:06, Robert J. Hijmans wrote: There seems to be a bug in "image" in R 13.1 (on windows 32& 64 bits) and on R-devel that is not present in R 13.0 and before. The below creates a plot with many white lines, horizontal and vertical, more or less regularly spaced. The effect is particularly dramatic when the plotting window is made very small. image(t(volcano)[ncol(volcano):1,]) Robert Hijmans University of California, Davis sessionInfo() R version 2.13.1 (2011-07-08) Platform: x86_64-pc-mingw32/x64 (64-bit) locale: [1] LC_COLLATE=English_United States.1252 [2] LC_CTYPE=English_United States.1252 [3] LC_MONETARY=English_United States.1252 [4] LC_NUMERIC=C [5] LC_TIME=English_United States.1252 attached base packages: [1] stats graphics grDevices utils datasets methods base [[alternative HTML version deleted]] __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
[Rd] Manipulating single-precision (float) arrays in .Call functions
I am writing a wrapper function in C++ that calls a GPU kernel. My array type for the GPU kernel is float, so I would like my wrapper function to receive float arrays from R. I understand that I can use 'as.single' in R to copy a double-precision vector from R in single-precision format while using the '.C' interface, but is there any way to do something similar for '.Call'? Given that the latter passes pointers from R to C/C++, I'm guessing this may be impossible, but I wanted to double-check. If you can suggest a solution, a small code sample would be much appreciated. The reason I prefer '.Call' to '.C' is because the former passes pointers and therefore creates less data transfer overhead as opposed to the latter which copies data. Is this argument controversial? -Alireza -- View this message in context: http://r.789695.n4.nabble.com/Manipulating-single-precision-float-arrays-in-Call-functions-tp3675684p3675684.html Sent from the R devel mailing list archive at Nabble.com. __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] Confusing inheritance problem
I've packaged the test library up as a tar file at ftp.mayo.edu directory therneau, file ktest.tar login username: mayoftp password: KPlLiFoz This will disappear in 3 days (Mayo is very fussy about outside access). In response to Uwe's comments 1. "2.13.0" is not recent It's not the latest, but it is recent. This is for machines at work where where upgrades happen more infrequently 2. "Matrix not loaded" The sessionInfo was only to show what version we have. Forgetting to load Matrix isn't the problem -- when I do that the error is quick and obvious. Thanks in advance for any pointers. Terry T. On Sat, 2011-07-16 at 19:27 +0200, Uwe Ligges wrote: > > On 15.07.2011 23:23, Terry Therneau wrote: > > I have library in development with a function that works when called > > from the top level, but fails under R CMD check. The paricular line of > > failure is > > rsum<- rowSums(kmat>0) > > where kmat is a dsCMatrix object. > > > >I'm currently stumped and looking for some ideas. > > > >I've created a stripped down library "ktest" that has only 3 > > functions: pedigree.R to create a pedigree or pedigreeList object, > >kinship.R with "kinship" methods for the two objects > >one small compute function called by the others > > along with the minimal amount of other information such that a call to > > R --vanilla CMD check ktest > > gives no errors until the fatal one. > > > > There are two test cases. A 3 line one that creates a dsCMatrix and > > call rowSums at the top level works fine, but the same call inside the > > kmat.pedigreeList function gives an error > > 'x' must be an array of at least two dimensions > > Adding a print statement above the rowSums call shows that the argument > > is a 14 by 14 dsCMatrix. > > > > I'm happy to send the library to anyone else to try and duplicate. > > Terry Therneau > > > > tmt% R --vanilla > > > >> sessionInfo() > > R version 2.13.0 (2011-04-13) > > Platform: x86_64-unknown-linux-gnu (64-bit) > > > > locale: > > [1] LC_CTYPE=en_US.UTF-8 LC_NUMERIC=C > > [3] LC_TIME=en_US.UTF-8LC_COLLATE=C > > [5] LC_MONETARY=C LC_MESSAGES=en_US.UTF-8 > > [7] LC_PAPER=en_US.UTF-8 LC_NAME=C > > [9] LC_ADDRESS=C LC_TELEPHONE=C > > [11] LC_MEASUREMENT=en_US.UTF-8 LC_IDENTIFICATION=C > > > > attached base packages: > > [1] stats graphics grDevices utils datasets methods > > base > > > > Terry, > > 1. Your R is not recent. > 2. You do this without having Matrix loaded (according to > sessionInfo())? This may already be the cause of your problems. > 3. You may want to make your package available on some website. I am > sure there are people who will take a look (including me, but not today). > > Best wishes, > Uwe > > > > > > > > > > > __ > > R-devel@r-project.org mailing list > > https://stat.ethz.ch/mailman/listinfo/r-devel __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] Manipulating single-precision (float) arrays in .Call functions
On 18/07/2011 11:52 AM, Alireza Mahani wrote: I am writing a wrapper function in C++ that calls a GPU kernel. My array type for the GPU kernel is float, so I would like my wrapper function to receive float arrays from R. I understand that I can use 'as.single' in R to copy a double-precision vector from R in single-precision format while using the '.C' interface, but is there any way to do something similar for '.Call'? Given that the latter passes pointers from R to C/C++, I'm guessing this may be impossible, but I wanted to double-check. If you can suggest a solution, a small code sample would be much appreciated. The reason I prefer '.Call' to '.C' is because the former passes pointers and therefore creates less data transfer overhead as opposed to the latter which copies data. Is this argument controversial? R has no native type holding singles. It exports a double to a single vector in .C if you ask it to, but that's not available in .Call. You'll need to do the copying yourself. Duncan Murdoch __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
[Rd] Fwd: Understanding R's "Environment" concept
*Initially, I posted this topic in R-help however, folks there suggested me to post this in R-devel forum. Here is my problem* Hi all, I am trying to understand the R's "environment" concept however the underlying help files look quite technical to me. Can experts here provide me some more intuitive ideas behind this concept like, why it is there, what exactly it is doing in R's architecture etc.? I mainly need some non-technical intuitive explanation. Thanks, __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] Fwd: Understanding R's "Environment" concept
On 2011-07-18, at 11:52 AM, Nipesh Bajaj wrote: > Hi all, I am trying to understand the R's "environment" concept > however the underlying help files look quite technical to me. Can > experts here provide me some more intuitive ideas behind this concept > like, why it is there, what exactly it is doing in R's architecture > etc.? The "official" definition in the language manual (http://cran.r-project.org/doc/manuals/R-lang.html#Environment-objects) is quite technical, but this sentence makes intuitive sense as well: When R looks up the value for a symbol the frame is examined and if a matching symbol is found its value will be returned. In other words, the environment holds all the defined values (variables, functions, etc.) and lets you refer to them by name. When you assign a value to a variable, say "x <- 10", this is stored in the environment. Then when the program is executed, when you refer to the name "x", it is looked up in the environment and replaced by its value there. (The reality is a little more complicated than that, but this is a close enough simplification.) As for "why and what it's doing there", you will need to understand some programming language concepts first -- I don't think it's possible to explain this "non-technically". Davor __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] Fwd: Understanding R's "Environment" concept
Here is an attempt at the general concept without getting technical. How many people in the world answer to the name/title "Dad"? Yet based on context we can usually tell who someone is talking about when they use "Dad". It is the same in programming, I may write a function which includes a variable named "length", but my function may call another function that also uses a variable named "length" which could be very different from my "length", the user that calls my function may have their own variable called "length". How is the computer to know which "length" to use when (and not replace the user's "length" with the one in the function)? By context of the functions, which context is called environments. Each function has its own environment and will create and use variables within that environment without affecting variables of the same name in other environments. Environments can be used for other things as well, but that starts getting more technical. -- Gregory (Greg) L. Snow Ph.D. Statistical Data Center Intermountain Healthcare greg.s...@imail.org 801.408.8111 > -Original Message- > From: r-devel-boun...@r-project.org [mailto:r-devel-bounces@r- > project.org] On Behalf Of Nipesh Bajaj > Sent: Monday, July 18, 2011 12:52 PM > To: r-devel@r-project.org > Subject: [Rd] Fwd: Understanding R's "Environment" concept > > *Initially, I posted this topic in R-help however, folks there > suggested me to post this in R-devel forum. Here is my > problem* > > > Hi all, I am trying to understand the R's "environment" concept > however the underlying help files look quite technical to me. Can > experts here provide me some more intuitive ideas behind this concept > like, why it is there, what exactly it is doing in R's architecture > etc.? > > I mainly need some non-technical intuitive explanation. > > Thanks, > > __ > R-devel@r-project.org mailing list > https://stat.ethz.ch/mailman/listinfo/r-devel __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
[Rd] filter(x, method="recursive") when there are NA's in x messes up
If x contains an NA then filter(x, f, method="recursive") puts NA's and 0.0's in the output in an odd pattern. It puts NA's in the length(f) positions ending at the position of the input NA and it puts a run of length(f) 0.0's starting at the position immediately after the input NA. The values after the 0.0's look like the recursion was restarted there. I'd naively expect that all output values at and after the position of the input NA to be NA, but it looks like it is trying to restart the recursion. I would not expect the output values before the position of the input NA to depend on future values. Are the NA's misplaced? > x <- c(1:4, NA, 6:9) > cbind(x, "1"=filter(x, 0.5, method="recursive"), + "2"=filter(x, c(0.5, 0.0), method="recursive"), + "3"=filter(x, c(0.5, 0.0, 0.0), method="recursive")) Time Series: Start = 1 End = 9 Frequency = 1 x 1 2 3 1 1 1.000 1.00 1.0 2 2 2.500 2.50 2.5 3 3 4.250 4.25 NA 4 4 6.125NA NA 5 NA NANA NA 6 6 0.000 0.00 0.0 7 7 7.000 0.00 0.0 8 8 11.500 8.00 0.0 9 9 14.750 13.00 9.0 Bill Dunlap Spotfire, TIBCO Software wdunlap tibco.com __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] [R] Problem compiling in extra/xdr
1) R-help is the wrong list: see the posting guide. I've moved this to R-devel. 2) A glibc system should not be compiling in that directory. glibc 2.14 is rather recent and NEWS does say * The RPC implementation in libc is obsoleted. Old programs keep working but new programs cannot be linked with the routines in libc anymore. Programs in need of RPC functionality must be linked against TI-RPC. The TI-RPC implementation is IPv6 enabled and there are other benefits. Visible changes of this change include (obviously) the inability to link programs using RPC functions without referencing the TI-RPC library and the removal of the RPC headers from the glibc headers. Implemented by Ulrich Drepper. So the answer seems to be that your libc is too new. On Mon, 18 Jul 2011, Allin Cottrell wrote: I'm building R 2.13.1 on i686-pc-linux-gnu, using gcc 4.6.1 and with glibc 2.14. I get this error: In file included from xdr.c:61:0: ./rpc/types.h:63:14: error: conflicting types for 'malloc' make[4]: *** [xdr.o] Error 1 I can make the build proceed some by commenting out the declaration "extern char *malloc();" in xdr/rpc/types.h, but then I get a slew of other errors: xdr_float.c: In function 'xdr_float': xdr_float.c:119:21: error: storage size of 'is' isn't known xdr_float.c:120:20: error: storage size of 'vs' isn't known and so on. config.log is rather big to post here; I'm putting it at http://www.wfu.edu/~cottrell/tmp/R.config.log . -- Allin Cottrell Department of Economics Wake Forest University, NC __ r-h...@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code. -- Brian D. Ripley, rip...@stats.ox.ac.uk Professor of Applied Statistics, http://www.stats.ox.ac.uk/~ripley/ University of Oxford, Tel: +44 1865 272861 (self) 1 South Parks Road, +44 1865 272866 (PA) Oxford OX1 3TG, UKFax: +44 1865 272595 __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] Manipulating single-precision (float) arrays in .Call functions
Duncan, Thank you for your reply. This is a rather unfortunate limitation, because for large data sizes there is a significant difference between the performance of '.C' and '.Call'. I will have to do some tests to see what sort of penalty I incur for copying from double to float inside my C++ code (so I can use '.Call'). I won't be able to use memset(); rather, I have to have an explicit loop and use casting. Is there a more efficient option? -Alireza -- View this message in context: http://r.789695.n4.nabble.com/Manipulating-single-precision-float-arrays-in-Call-functions-tp3675684p3676667.html Sent from the R devel mailing list archive at Nabble.com. __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] Manipulating single-precision (float) arrays in .Call functions
On Jul 18, 2011, at 6:15 PM, Alireza Mahani wrote:
> Duncan,
>
> Thank you for your reply. This is a rather unfortunate limitation, because
> for large data sizes there is a significant difference between the
> performance of '.C' and '.Call'.
I think you may have missed the main point - R does NOT have any objects that
are in "float" (single precision) representations because that is insufficient
precision, so one way or another you'll have to do something like
SEXP foo(SEXP bar) {
double *d = REAL(bar);
int i, n = LENGTH(bar);
float *f = (float*) R_alloc(sizeof(float), n);
for (i = 0; i < n; i++) f[i] = d[i];
// continue with floats ..
There is simply no other way as, again, there are no floats in R anywhere. This
has nothing to do with .C/.Call, either way floats will need to be created from
the input vectors.
If you make up your own stuff, you could use raw vectors to store floats if
only your functions work on it (or external pointers if you want to keep track
of the memory yourself).
> I will have to do some tests to see what
> sort of penalty I incur for copying from double to float inside my C++ code
> (so I can use '.Call'). I won't be able to use memset(); rather, I have to
> have an explicit loop and use casting. Is there a more efficient option?
>
I'm not aware of any, if you use floats, you incur a penalty regardless (one
for additional storage, another for converting).
One of the reasons that GPU processing has not caught much traction in stat
computing is because it's (practically) limited to single precision
computations (and we even need quad precision occasionally). Although some GPUs
support double precision, they are still not fast enough to be a real threat to
CPUs. (I'm talking about generic usability - for very specialized tasks GPU can
deliver big speedups, but those require low-level exploitation of the
architecture).
Cheers,
Simon
__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] Manipulating single-precision (float) arrays in .Call functions
Simon, Thank you for elaborating on the limitations of R in handling float types. I think I'm pretty much there with you. As for the insufficiency of single-precision math (and hence limitations of GPU), my personal take so far has been that double-precision becomes crucial when some sort of error accumulation occurs. For example, in differential equations where boundary values are integrated to arrive at interior values, etc. On the other hand, in my personal line of work (Hierarchical Bayesian models for quantitative marketing), we have so much inherent uncertainty and noise at so many levels in the problem (and no significant error accumulation sources) that single vs double precision issue is often inconsequential for us. So I think it really depends on the field as well as the nature of the problem. Regards, Alireza -- View this message in context: http://r.789695.n4.nabble.com/Manipulating-single-precision-float-arrays-in-Call-functions-tp3675684p3677232.html Sent from the R devel mailing list archive at Nabble.com. __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] [R] Problem compiling in extra/xdr
On Mon, 18 Jul 2011, Allin Cottrell wrote: On Mon, 18 Jul 2011, Prof Brian Ripley wrote: 1) R-help is the wrong list: see the posting guide. I've moved this to R-devel. 2) A glibc system should not be compiling in that directory. glibc 2.14 is rather recent and NEWS does say * The RPC implementation in libc is obsoleted. Old programs keep working but new programs cannot be linked with the routines in libc anymore. Programs in need of RPC functionality must be linked against TI-RPC. The TI-RPC implementation is IPv6 enabled and there are other benefits. Visible changes of this change include (obviously) the inability to link programs using RPC functions without referencing the TI-RPC library and the removal of the RPC headers from the glibc headers. Implemented by Ulrich Drepper. So the answer seems to be that your libc is too new. OK, thanks. I should have remembered the info about RPC in the glibc-2.14 news. Then there will presumably be a problem building current R on current Fedora? What is 'current Fedora'? glibc 2.14 postdates the current release, Fedora 15, which uses 2.13. I do not know what Fedora 16 will use in several months The main problem will be that the xdr included in R is only for platforms with 32-bit longs -- but that may be true for your i686 Linux. It needs _X86_ defined to compile for i686: I would have expected that to be true on your platform, but am testing a cleaned-up version. If that works it will appear in R-patched within 24 hours. On Mon, 18 Jul 2011, Allin Cottrell wrote: I'm building R 2.13.1 on i686-pc-linux-gnu, using gcc 4.6.1 and with glibc 2.14. I get this error: In file included from xdr.c:61:0: ./rpc/types.h:63:14: error: conflicting types for 'malloc' make[4]: *** [xdr.o] Error 1 I can make the build proceed some by commenting out the declaration "extern char *malloc();" in xdr/rpc/types.h, but then I get a slew of other errors: xdr_float.c: In function 'xdr_float': xdr_float.c:119:21: error: storage size of 'is' isn't known xdr_float.c:120:20: error: storage size of 'vs' isn't known and so on. config.log is rather big to post here; I'm putting it at http://www.wfu.edu/~cottrell/tmp/R.config.log . -- Allin Cottrell Department of Economics Wake Forest University, NC __ r-h...@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code. -- Allin Cottrell Department of Economics Wake Forest University -- Brian D. Ripley, rip...@stats.ox.ac.uk Professor of Applied Statistics, http://www.stats.ox.ac.uk/~ripley/ University of Oxford, Tel: +44 1865 272861 (self) 1 South Parks Road, +44 1865 272866 (PA) Oxford OX1 3TG, UKFax: +44 1865 272595 __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] Manipulating single-precision (float) arrays in .Call functions
On Mon, 18 Jul 2011, Alireza Mahani wrote: Simon, Thank you for elaborating on the limitations of R in handling float types. I think I'm pretty much there with you. As for the insufficiency of single-precision math (and hence limitations of GPU), my personal take so far has been that double-precision becomes crucial when some sort of error accumulation occurs. For example, in differential equations where boundary values are integrated to arrive at interior values, etc. On the other hand, in my personal line of work (Hierarchical Bayesian models for quantitative marketing), we have so much inherent uncertainty and noise at so many levels in the problem (and no significant error accumulation sources) that single vs double precision issue is often inconsequential for us. So I think it really depends on the field as well as the nature of the problem. The main reason to use only double precision in R was that on modern CPUs double precision calculations are as fast as single-precision ones, and with 64-bit CPUs they are a single access. So the extra precision comes more-or-less for free. You also under-estimate the extent to which stability of commonly used algorithms relies on double precision. (There are stable single-precision versions, but they are no longer commonly used. And as Simon said, in some cases stability is ensured by using extra precision where available.) I disagree slightly with Simon on GPUs: I am told by local experts that the double-precision on the latest GPUs (those from the last year or so) is perfectly usable. See the performance claims on http://en.wikipedia.org/wiki/Nvidia_Tesla of about 50% of the SP performance in DP. Regards, Alireza -- View this message in context: http://r.789695.n4.nabble.com/Manipulating-single-precision-float-arrays-in-Call-functions-tp3675684p3677232.html Sent from the R devel mailing list archive at Nabble.com. __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel -- Brian D. Ripley, rip...@stats.ox.ac.uk Professor of Applied Statistics, http://www.stats.ox.ac.uk/~ripley/ University of Oxford, Tel: +44 1865 272861 (self) 1 South Parks Road, +44 1865 272866 (PA) Oxford OX1 3TG, UKFax: +44 1865 272595 __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
