[Rd] Why doesn't R have a float data type?
This is strictly a curiosity question. I am aware the R doesn't possess a float data type. I also don't mean to request that such functionality be implemented as I'm sure it would require a large amount of work with potential back compatibility conflicts. But I wanted to know why R has never had a float data type available? Regards, Charles [[alternative HTML version deleted]] __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] Why doesn't R have a float data type?
Hi Greg, I was referring to the single precision type. Your points were what I expected. I just wanted to ask the R community if there was any other reason than 'there wasn't any reason to implement it'. Thanks, Charles On Tue, Jun 30, 2015 at 12:29 PM, Greg Snow <538...@gmail.com> wrote: > My understanding is that R does have a float type, it is just called > "double" instead of "float". > > If you are referring to a single precision floating point type, then R > does have the "as.single" function, but that does not really change > the way the number is stored, just sets a flag so that the proper > conversion is done when passing to the ".C" or ".fortran" functions. > The original S language and S+ would store things in single precision > if needed, but for computations these values were almost always > converted to doubles for precision. By the time R was developed the > memory saving of using single precision instead of double precision > was not as big an issue, so I expect that nobody ever considered it > worth the effort to fully implement the single precision storage. > > If you mean something else other than the above by "float data type" > then please give us more details so that we can better answer the > question. > > On Tue, Jun 30, 2015 at 10:42 AM, Charles Determan > wrote: > > This is strictly a curiosity question. I am aware the R doesn't possess > a > > float data type. I also don't mean to request that such functionality be > > implemented as I'm sure it would require a large amount of work with > > potential back compatibility conflicts. But I wanted to know why R has > > never had a float data type available? > > > > Regards, > > Charles > > > > [[alternative HTML version deleted]] > > > > __ > > R-devel@r-project.org mailing list > > https://stat.ethz.ch/mailman/listinfo/r-devel > > > > -- > Gregory (Greg) L. Snow Ph.D. > 538...@gmail.com > [[alternative HTML version deleted]] __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] [R] TensorFlow in R
Hi Axel, Looks like the only thing right now is rflow ( https://github.com/terrytangyuan/rflow). It appears to simply wrap around the python bindings. I am not aware of any others. Be interesting to keep an eye on. Regards, Charles On Fri, Apr 1, 2016 at 11:32 AM, Axel Urbiz wrote: > Hi All, > > I didn't have much success through my Google search in finding any active > R-related projects to create a wrapper around TensorFlow in R. Anyone know > if this is on the go? > > Thanks, > Axel. > > [[alternative HTML version deleted]] > > __ > r-h...@r-project.org mailing list -- To UNSUBSCRIBE and more, see > https://stat.ethz.ch/mailman/listinfo/r-help > PLEASE do read the posting guide > http://www.R-project.org/posting-guide.html > and provide commented, minimal, self-contained, reproducible code. > [[alternative HTML version deleted]] __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
[Rd] Single-threaded aspect
R Developers, Could someone help explain what it means that R is single threaded? I am trying to understand what is actually going on inside R when users want to parallelize code. For example, using mclapply or foreach (with some backend) somehow allows users to benefit from multiple CPUs. Similarly there is the RcppParallel package for RMatrix/RVector objects. But none of these address the general XPtr objects in Rcpp. Some readers here may recognize my question on SO ( http://stackoverflow.com/questions/37167479/rcpp-parallelize-functions-that-return-xptr) where I was curious about parallel calls to C++/Rcpp functions that return XPtr objects. I am being a little more persistent here as this limitation provides a very hard stop on the development on one of my packages that heavily uses XPtr objects. It's not meant to be a criticism or intended to be rude, I just want to fully understand. I am willing to accept that it may be impossible currently but I want to at least understand why it is impossible so I can explain to future users why parallel functionality is not available. Which just echos my original question, what does it mean that R is single threaded? Kind Regards, Charles [[alternative HTML version deleted]] __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] Single-threaded aspect
Thanks for the replies. Regarding the answer by Dirk, I didn't feel like I still understood the reasoning why mclapply or foreach cannot handle XPtr objects. Instead of cluttering the SO question with comments I was getting the impression that this was a limitation inherited with R objects (which XPtr is supposed to be a proxy for an R object according to Dirk's comment). If this is not the case, I could repost this on Rcpp-devel unless it could be migrated. Regards, Charles On Thu, May 12, 2016 at 8:11 AM, Mark van der Loo wrote: > Charles, > > 1. Perhaps this question is better directed at the R-help or > R-pacakge-devel mailinglist. > > 2. It basically means that R itself can only evaluate one R expression at > the time. > > The parallel package circumvents this by starting multiple R-sessions and > dividing workload. > > Compiled code called by R (such as C++ code through RCpp or C-code through > base R's interface) can execute multi-threaded code for internal purposes, > using e.g. openMP. A limitation is that compiled code cannot call R's C API > from multiple threads (in many cases). For example, it is not thread-safe > to create R-variables from multiple threads running in C. (R's variable > administration is such that the order of (un)making them from compiled code > matters). > > I am not very savvy on Rcpp or XPtr objects, but it appears that Dirk > provided answers about that in your SO-question. > > Best, > Mark > > > > > > > > > > > Op do 12 mei 2016 om 14:46 schreef Charles Determan >: > >> R Developers, >> >> Could someone help explain what it means that R is single threaded? I am >> trying to understand what is actually going on inside R when users want to >> parallelize code. For example, using mclapply or foreach (with some >> backend) somehow allows users to benefit from multiple CPUs. >> >> Similarly there is the RcppParallel package for RMatrix/RVector objects. >> But none of these address the general XPtr objects in Rcpp. Some readers >> here may recognize my question on SO ( >> >> http://stackoverflow.com/questions/37167479/rcpp-parallelize-functions-that-return-xptr >> ) >> where I was curious about parallel calls to C++/Rcpp functions that return >> XPtr objects. I am being a little more persistent here as this limitation >> provides a very hard stop on the development on one of my packages that >> heavily uses XPtr objects. It's not meant to be a criticism or intended >> to >> be rude, I just want to fully understand. >> >> I am willing to accept that it may be impossible currently but I want to >> at >> least understand why it is impossible so I can explain to future users why >> parallel functionality is not available. Which just echos my original >> question, what does it mean that R is single threaded? >> >> Kind Regards, >> Charles >> >> [[alternative HTML version deleted]] >> >> __ >> R-devel@r-project.org mailing list >> https://stat.ethz.ch/mailman/listinfo/r-devel >> > [[alternative HTML version deleted]] __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] Single-threaded aspect
Thank you Simon for the detailed reply. That explains much more of what I was looking for from the R side. Dirk, I'm sorry if I seem hung up on anything here but I am trying to understand the details. My reply about XPtr or XPtr on arma/Eigen was to confirm my understanding was correct, which it appears it was. I was not aware the RVector/RMatrix objects don't connect to R as I am just now familiarizing myself with the package, that explains more of my confusion. I will look at doing work within the compiled code as you have suggested. Regards, Charles On Thu, May 12, 2016 at 9:18 AM, Dirk Eddelbuettel wrote: > > On 12 May 2016 at 13:11, Mark van der Loo wrote: > | Charles, > | > | 1. Perhaps this question is better directed at the R-help or > | R-pacakge-devel mailinglist. > | > | 2. It basically means that R itself can only evaluate one R expression at > | the time. > | > | The parallel package circumvents this by starting multiple R-sessions and > | dividing workload. > | > | Compiled code called by R (such as C++ code through RCpp or C-code > through > | base R's interface) can execute multi-threaded code for internal > purposes, > | using e.g. openMP. A limitation is that compiled code cannot call R's C > API > | from multiple threads (in many cases). For example, it is not thread-safe > | to create R-variables from multiple threads running in C. (R's variable > | administration is such that the order of (un)making them from compiled > code > | matters). > > Well put. > > | I am not very savvy on Rcpp or XPtr objects, but it appears that Dirk > | provided answers about that in your SO-question. > > Charles seems to hang himself up completely about a small detail, failing > to > see the forest for the trees. > > There are (many) working examples of parallel (compiled) code with R. All > of > them stress (and I simplify here) that can you touch R objects, or call > back > into R, for fear of any assignment or allocation triggering an R event. R > being single-threaded it cannot do this. > > My answer to this problem is to only use non-R data structures. That is > what > RcpParallel does in the actual parallel code portions in all examples -- > types RVector and RMatrix do NOT connect back to R. There are several > working > examples. That is also what the OpenMP examples at the Rcpp Gallery do. > > Charles seems to be replying 'but I use XPtr' or 'I use XPtr on arma::mat > or > Eigen::Matrixxd' and seems to forget that these are proxy objects to SEXPs. > XPtr just wrap the SEXP for external pointers; Arma's and Eigen's matrices > are performant via RcppArmadillo and RcppEigen because we use R memory via > proxies. All of that is 'too close to R' for comfort. > > So the short answer is: enter compiled code from R, set a mutex (either > conceptually or explicitly), _copy_ your data in to plain C++ data > structures > and go to town in parallel via OpenMP and other multithreaded approaches. > Then collect the result, release the mutex and move back up. > > I hope this help. > > Dirk > > | > | Best, > | Mark > | > | > | > | > | > | > | > | > | > | > | Op do 12 mei 2016 om 14:46 schreef Charles Determan < > cdeterma...@gmail.com>: > | > | > R Developers, > | > > | > Could someone help explain what it means that R is single threaded? I > am > | > trying to understand what is actually going on inside R when users > want to > | > parallelize code. For example, using mclapply or foreach (with some > | > backend) somehow allows users to benefit from multiple CPUs. > | > > | > Similarly there is the RcppParallel package for RMatrix/RVector > objects. > | > But none of these address the general XPtr objects in Rcpp. Some > readers > | > here may recognize my question on SO ( > | > > | > > http://stackoverflow.com/questions/37167479/rcpp-parallelize-functions-that-return-xptr > | > ) > | > where I was curious about parallel calls to C++/Rcpp functions that > return > | > XPtr objects. I am being a little more persistent here as this > limitation > | > provides a very hard stop on the development on one of my packages that > | > heavily uses XPtr objects. It's not meant to be a criticism or > intended to > | > be rude, I just want to fully understand. > | > > | > I am willing to accept that it may be impossible currently but I want > to at > | > least understand why it is impossible so I can explain to future users > why > | > parallel functionality is not available. Which just echos my original > | > question, what do
Re: [Rd] R external pointer and GPU memory leak problem
Hi Yuan,
I think this is likely more appropriate for the r-sig-hpc mailing list.
However, regarding you design and comment about R's 'current' GPU package
(I don't what you consider this, gputools?) I think you should look at two
other packages. I believe the gmatrix (
https://cran.r-project.org/web/packages/gmatrix/index.html) implements
exactly what you are trying to do for NVIDIA specific code. There is also
the gpuR package (https://cran.r-project.org/web/packages/gpuR/index.html)
package which also implements the object 'on GPU' functionality you desire
but in OpenCL so it works for 'all' GPUs.
If you really want to continue your development I strongly recommend you
look in to using Rcpp and the XPtr objects for external pointers. They
handle the pointer protection and finalizer without you needing to worry
about them.
Regards,
Charles
On Sat, May 14, 2016 at 10:43 AM, Yuan Li wrote:
> My question is based on a project I have partially done, but there is
> still something I'm not clear.
>
> My goal is to create a R package contains GPU functions (some are from
> Nividia cuda library, some are my self-defined CUDA functions)
>
> My design is quite different from current R's GPU package, I want to
> create a R object (external pointer) point to GPU address, and run my GPU
> function direct on GPU side without transferring forth and back between CPU
> and GPU.
>
> I used the R external pointer to implement my design. But I found I have
> memory leak problems on GPU side, I can still fix it by running gc()
> function explicitly in R side, but I'm just wondering if I missed something
> in my C code. Would you please indicate my mistake, because this is my
> first time write a R package, and I could possibly made some terrible
> mistakes.
>
> actually, I have wrote bunch of GPU functions which can run on GPU side
> with the object created by following create function, but the memory leak
> kills me if I need to deal with some huge dataset.
>
> Here is my create function, I create a gpu pointer x, and allocate GPU
> memory for x, then make a R external pointer ext based on x, and copy the
> cpu vector input to my gpu external pointer ext,
>
>
> /*
> define function to create a vector in GPU
> by transferring a R's vector to GPU.
> input is R's vector and its length,
> output is a R external pointer
> pointing to GPU vector(device)
> */
> SEXP createGPU(SEXP input, SEXP n)
> {
> int *lenth = INTEGER(n);
>PROTECT (input = AS_NUMERIC (input));
>double * temp;
>temp = REAL(input);
> double *x; ##here is the step which causes the memory leak
> cudacall(cudaMalloc((void**)&x, *lenth * sizeof(double)));
> //protect the R external pointer from finalizer
> SEXP ext = PROTECT(R_MakeExternalPtr(x, R_NilValue, R_NilValue));
> R_RegisterCFinalizerEx(ext, _finalizer, TRUE);
>
> //copying CPU to GPU
> cublascall(cublasSetVector(*lenth, sizeof(double), temp, 1,
> R_ExternalPtrAddr(ext), 1));
>UNPROTECT(2);
> return ext;
> }
>
>
>
> here is my finalized for my create function,
>
> /*
> define finalizer for R external pointer
> input is R external pointer, function will finalize the pointer
> when it is not in use.
> */
> static void _finalizer(SEXP ext)
> {
> if (!R_ExternalPtrAddr(ext))
> return;
>double * ptr= (double *) R_ExternalPtrAddr(ext);
> Rprintf("finalizer invoked once \n");
> cudacall(cudaFree(ptr));
> R_ClearExternalPtr(ext);
> }
>
>
> My create function can run smoothly, but if I run the create function too
> many times, it shows out of memory for my GPU device, which clearly implies
> memory leak problem. Can anybody help? Help alot in advance!
> __
> R-devel@r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-devel
>
[[alternative HTML version deleted]]
__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
[Rd] example fails during R CMD CHECK but works interactively?
Greetings, I am collaborating with developing the bigmemory package and have run in to a strange problem when we run R CMD CHECK. For some reason that isn't clear to us one of the examples crashes stating: Error: memory could not be allocated for instance of type big.matrix You can see the output on the Travis CI page at https://travis-ci.org/kaneplusplus/bigmemory where the error starts at line 1035. This is completely reproducible when running devtools::check(args='--as-cran') locally. The part that is confusing is that the calls work perfectly when called interactively. Hadley comments on the 'check' page of his R packages website ( http://r-pkgs.had.co.nz/check.html) regarding test failing following R CMD check: Occasionally you may have a problem where the tests pass when run interactively with devtools::test(), but fail when in R CMD check. This usually indicates that you’ve made a faulty assumption about the testing environment, and it’s often hard to figure it out. Any thoughts on how to troubleshoot this problem? I have no idea what assumption we could have made. Regards, Charles [[alternative HTML version deleted]] __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] example fails during R CMD CHECK but works interactively?
Thank you Dan but it isn't my tests that are failing (all of them pass without problem) but one of the examples from the inst/examples directory. I did try, however, to start R with the environmental variables as you suggest but it had no effect on my tests. Charles On Wed, May 13, 2015 at 1:51 PM, Dan Tenenbaum wrote: > > > - Original Message ----- > > From: "Charles Determan" > > To: r-devel@r-project.org > > Sent: Wednesday, May 13, 2015 11:31:36 AM > > Subject: [Rd] example fails during R CMD CHECK but works interactively? > > > > Greetings, > > > > I am collaborating with developing the bigmemory package and have run > > in to > > a strange problem when we run R CMD CHECK. For some reason that > > isn't > > clear to us one of the examples crashes stating: > > > > Error: memory could not be allocated for instance of type big.matrix > > > > You can see the output on the Travis CI page at > > https://travis-ci.org/kaneplusplus/bigmemory where the error starts > > at line > > 1035. This is completely reproducible when running > > devtools::check(args='--as-cran') locally. The part that is > > confusing is > > that the calls work perfectly when called interactively. > > > > Hadley comments on the 'check' page of his R packages website ( > > http://r-pkgs.had.co.nz/check.html) regarding test failing following > > R CMD > > check: > > > > Occasionally you may have a problem where the tests pass when run > > interactively with devtools::test(), but fail when in R CMD check. > > This > > usually indicates that you’ve made a faulty assumption about the > > testing > > environment, and it’s often hard to figure it out. > > > > Any thoughts on how to troubleshoot this problem? I have no idea > > what > > assumption we could have made. > > Note that R CMD check runs R with environment variables set as follows (at > least on my system; you can check $R_HOME/bin/check to see what it does on > yours): > > R_DEFAULT_PACKAGES= LC_COLLATE=C > > So try staring R like this: > > R_DEFAULT_PACKAGES= LC_COLLATE=C R > > And see if that reproduces the test failure. The locale setting could > affect tests of sort order, and the default package setting could > potentially affect other things. > > Dan > > > > > > > Regards, > > Charles > > > > [[alternative HTML version deleted]] > > > > __ > > R-devel@r-project.org mailing list > > https://stat.ethz.ch/mailman/listinfo/r-devel > > > [[alternative HTML version deleted]] __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] example fails during R CMD CHECK but works interactively?
Does anyone else have any thoughts about troubleshooting the R CMD check environment? Charles On Wed, May 13, 2015 at 1:57 PM, Charles Determan wrote: > Thank you Dan but it isn't my tests that are failing (all of them pass > without problem) but one of the examples from the inst/examples directory. > I did try, however, to start R with the environmental variables as you > suggest but it had no effect on my tests. > > Charles > > On Wed, May 13, 2015 at 1:51 PM, Dan Tenenbaum > wrote: > >> >> >> - Original Message - >> > From: "Charles Determan" >> > To: r-devel@r-project.org >> > Sent: Wednesday, May 13, 2015 11:31:36 AM >> > Subject: [Rd] example fails during R CMD CHECK but works interactively? >> > >> > Greetings, >> > >> > I am collaborating with developing the bigmemory package and have run >> > in to >> > a strange problem when we run R CMD CHECK. For some reason that >> > isn't >> > clear to us one of the examples crashes stating: >> > >> > Error: memory could not be allocated for instance of type big.matrix >> > >> > You can see the output on the Travis CI page at >> > https://travis-ci.org/kaneplusplus/bigmemory where the error starts >> > at line >> > 1035. This is completely reproducible when running >> > devtools::check(args='--as-cran') locally. The part that is >> > confusing is >> > that the calls work perfectly when called interactively. >> > >> > Hadley comments on the 'check' page of his R packages website ( >> > http://r-pkgs.had.co.nz/check.html) regarding test failing following >> > R CMD >> > check: >> > >> > Occasionally you may have a problem where the tests pass when run >> > interactively with devtools::test(), but fail when in R CMD check. >> > This >> > usually indicates that you’ve made a faulty assumption about the >> > testing >> > environment, and it’s often hard to figure it out. >> > >> > Any thoughts on how to troubleshoot this problem? I have no idea >> > what >> > assumption we could have made. >> >> Note that R CMD check runs R with environment variables set as follows >> (at least on my system; you can check $R_HOME/bin/check to see what it does >> on yours): >> >> R_DEFAULT_PACKAGES= LC_COLLATE=C >> >> So try staring R like this: >> >> R_DEFAULT_PACKAGES= LC_COLLATE=C R >> >> And see if that reproduces the test failure. The locale setting could >> affect tests of sort order, and the default package setting could >> potentially affect other things. >> >> Dan >> >> >> >> > >> > Regards, >> > Charles >> > >> > [[alternative HTML version deleted]] >> > >> > __ >> > R-devel@r-project.org mailing list >> > https://stat.ethz.ch/mailman/listinfo/r-devel >> > >> > > [[alternative HTML version deleted]] __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
[Rd] modifying some package code
Greetings, I am working on modifying some code from the nlme package. I have had many discussions on the mixed models mailing list and have been directed to simply 'hack' the source code to have the degrees of freedom generated by one function to use in the output of another function that doesn't generate them. My current holdup is an error regarding a .c file called 'inner_perc_table' called by the .C function. The error states that the object 'inner_perc_table' is not found. My confusion lies in the fact that when I run the original script, it recognizes the part just fine. At no point is the object defined and I cannot currently find such a code in the package's source. Perhaps someone here is familiar with the nlme package and could assist me in some form. If you need further information, please ask as I don't know if there is a general answer for this type of question or if you will need the actual code. Regards, Charles [[alternative HTML version deleted]] __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] modifying some package code
Simon, Thank you for this valuable information. However, you must forgive some ignorance on my part. If R-registerRoutines defines the native function, how should I go about fixing this issue? Would I copy the init.c to the base package (where I have the new function)? Thanks, Charles On Thu, May 24, 2012 at 11:58 AM, Simon Urbanek wrote: > > On May 24, 2012, at 12:25 PM, Charles Determan Jr wrote: > > > Greetings, > > > > I am working on modifying some code from the nlme package. I have had > many > > discussions on the mixed models mailing list and have been directed to > > simply 'hack' the source code to have the degrees of freedom generated by > > one function to use in the output of another function that doesn't > generate > > them. My current holdup is an error regarding a .c file called > > 'inner_perc_table' called by the .C function. The error states that the > > object 'inner_perc_table' is not found. My confusion lies in the fact > that > > when I run the original script, it recognizes the part just fine. At no > > point is the object defined and I cannot currently find such a code in > the > > package's source. Perhaps someone here is familiar with the nlme package > > and could assist me in some form. If you need further information, > please > > ask as I don't know if there is a general answer for this type of > question > > or if you will need the actual code. > > > > The (unexported) object contains cached reference to the native function > (see ?getNativeSymbolInfo) and is defined by R_registerRoutines in > src/init.c. This is a typical optimization in R packages to avoid costly > lookup of symbols and to provide check for native arguments. > > Cheers, > Simon > > > > [[alternative HTML version deleted]] __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
