[Rd] Problem with accessing internal variable in package.
I am writing a package, which contains several functions and variables. The variables are for internal use by functions. Functions are ment to be callable by a user. However, the function cannot access these variables. The package contains only R code, and was created using package.skeleton package.skeleton(name="ac9", list=c(ls(pattern="(AB2)|(ac9)"), "Chl","Pro","LP","SP","syn","unk","Y","Nh", "plot.C","read.AC9.data","EstimC","plot.AC9"), namespace=TRUE,force=TRUE) Among others, the package contains the functions EstimC, AB2C, AB2C.S, and AB2C.HK, and variables ac9nw, ac9wl, ac9aw, ... Functions use these variables for calculations, and I would like to hide them from users. The file NAMESPACE contains the single line: export(AB2C.HK,AB2C,AB2C.S,EstimC,plot.C,read.AC9.data,plot.AC9) I am getting the following error: Error in AB2C(a, b, model.type) : object "ac9nw" not found > traceback() 2: AB2C(a, b, model.type) 1: EstimC(i, plot.base.name = paste(plt.base.name, "S_", sep = ""), model.type = "S") > sessionInfo() R version 2.5.0 Patched (2007-05-13 r41549) i386-pc-mingw32 locale: LC_COLLATE=Russian_Russia.1251;LC_CTYPE=Russian_Russia.1251;LC_MONETARY=Russian_Russia.1251;LC_NUMERIC=C;LC_TIME=Russian_Russia.1251 attached base packages: [1] "datasets" "utils" "stats" "graphics" "grDevices" "methods" [7] "base" other attached packages: ac9 xlsReadWrite matlab MASS "0.1" "1.3.2" "0.8-1" "7.2-33" -- View this message in context: http://www.nabble.com/Problem-with-accessing-internal-variable-in-package.-tf3814649.html#a10798364 Sent from the R devel mailing list archive at Nabble.com. __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] Problem with accessing internal variable in package.
This is my first package, and clearly, I'm missing something. After reading again manuals about name spaces and hooks, I have defined the function .onLoad, which implicitly assigns all internal variables with <<- Then I have deleted previous package directory and changed the call to package.skeleton package.skeleton(name="ac9", list=c(ls(pattern="(AB2)"), "plot.C","read.AC9.data","EstimC","plot.AC9",".onLoad"), namespace=TRUE,force=TRUE) It has created file ac9/R/ac9-internal.R, containing the function .onLoad, however, its body completely differs from that I have defined before. Vladimir Eremeev wrote: > > I am writing a package, which contains several functions and variables. > The variables are for internal use by functions. > Functions are ment to be callable by a user. > However, the function cannot access these variables. > > The package contains only R code, and was created using package.skeleton > > package.skeleton(name="ac9", > list=c(ls(pattern="(AB2)|(ac9)"), > "Chl","Pro","LP","SP","syn","unk","Y","Nh", > "plot.C","read.AC9.data","EstimC","plot.AC9"), > namespace=TRUE,force=TRUE) > > Among others, the package contains the functions EstimC, AB2C, AB2C.S, and > AB2C.HK, and variables ac9nw, ac9wl, ac9aw, ... > Functions use these variables for calculations, and I would like to hide > them from users. > > The file NAMESPACE contains the single line: > export(AB2C.HK,AB2C,AB2C.S,EstimC,plot.C,read.AC9.data,plot.AC9) > > I am getting the following error: > Error in AB2C(a, b, model.type) : object "ac9nw" not found > >> traceback() > 2: AB2C(a, b, model.type) > 1: EstimC(i, plot.base.name = paste(plt.base.name, "S_", sep = ""), >model.type = "S") > >> sessionInfo() > R version 2.5.0 Patched (2007-05-13 r41549) > i386-pc-mingw32 > > locale: > LC_COLLATE=Russian_Russia.1251;LC_CTYPE=Russian_Russia.1251;LC_MONETARY=Russian_Russia.1251;LC_NUMERIC=C;LC_TIME=Russian_Russia.1251 > > attached base packages: > [1] "datasets" "utils" "stats" "graphics" "grDevices" "methods" > [7] "base" > > other attached packages: > ac9 xlsReadWrite matlab MASS >"0.1" "1.3.2" "0.8-1" "7.2-33" > -- View this message in context: http://www.nabble.com/Problem-with-accessing-internal-variable-in-package.-tf3814649.html#a10798787 Sent from the R devel mailing list archive at Nabble.com. __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
[Rd] C code for qbeta
Can somebody help me? I need the C/C++ code for the R's qbeta function. Can you send it to me a [EMAIL PROTECTED] ? PS: I'm italian, i can't speak english very well... ___ __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] C code for qbeta
On Friday 25 May 2007 05:02, Gatsu wrote: > Can somebody help me? > > I need the C/C++ code for the R's qbeta function. R is open-source, so you can simply download the source and look at whatever parts you like. Sean __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
[Rd] private variables in package.
I am sorry, my previous questions about hiding variables from users in a package were obscure and vague, and would like to clarify them. This should be a basic question. I am writing my first package. It has several functions, which should be callable by users. These functions use several variables, and I would like to hide these variables from users. How can I do it? For example, the package has a function AB2C, which uses the variable ac9nw. If I add the file NAMESPACE, exporting only AB2C, then I get the error: Error in AB2C(a, b, model.type) : object "ac9nw" not found I also tried to include in the .onLoad function the code, defining these variables with <<- However, this code has created all internal variables in the global environment. -- View this message in context: http://www.nabble.com/private-variables-in-package.-tf3815306.html#a10800416 Sent from the R devel mailing list archive at Nabble.com. __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] private variables in package.
Thank you for the answer.
Here is everything.
ac9wl<-c(412,440,488,510,532,555,650,676,715)
ac9nw<-length(ac9wl)
AB2C <-function(a,b,model.type="S") {
nza<-dim(a)[1]
nwa<-dim(a)[2]
nzb<-dim(b)[1]
nwb<-dim(b)[2]
if(nza!=nzb || nwa!=ac9nw || nwb!=ac9nw) {
warning('AB2C: Dimensions of the input matrices are invalid')
return(NULL);
}
[ further calculations, they do not use ac9nw ]
}
ripley-3 wrote:
>
> R has debugging features: see 'Writing R Extensions'. Please make use of
> them, e.g. options(error=recover) will enable you to explore the
> environments that are visible.
>
> We can only guess at this, not having function AB2C.
>
> On Fri, 25 May 2007, Vladimir Eremeev wrote:
>
>
>> I am sorry, my previous questions about hiding variables from users in a
>> package were obscure and vague, and would like to clarify them.
>>
>> This should be a basic question.
>> I am writing my first package.
>> It has several functions, which should be callable by users.
>> These functions use several variables, and I would like to hide these
>> variables from users.
>> How can I do it?
>>
>> For example, the package has a function AB2C, which uses the variable
>> ac9nw.
>> If I add the file NAMESPACE, exporting only AB2C, then I get the error:
>>
>> Error in AB2C(a, b, model.type) : object "ac9nw" not found
>
--
View this message in context:
http://www.nabble.com/private-variables-in-package.-tf3815306.html#a10801165
Sent from the R devel mailing list archive at Nabble.com.
__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] private variables in package.
On 5/25/2007 7:27 AM, Vladimir Eremeev wrote:
> Thank you for the answer.
>
> Here is everything.
>
> ac9wl<-c(412,440,488,510,532,555,650,676,715)
> ac9nw<-length(ac9wl)
>
> AB2C <-function(a,b,model.type="S") {
> nza<-dim(a)[1]
> nwa<-dim(a)[2]
> nzb<-dim(b)[1]
> nwb<-dim(b)[2]
>
> if(nza!=nzb || nwa!=ac9nw || nwb!=ac9nw) {
> warning('AB2C: Dimensions of the input matrices are invalid')
> return(NULL);
> }
>
> [ further calculations, they do not use ac9nw ]
> }
That would work, so my guess is that you aren't using that function. If
you have a function named AB2C in your global environment, R will find
it before this one, and it may not be able to see the ac9nw variable.
Duncan Murdoch
__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] private variables in package.
I do use this function.
Here is the example session, run from the newly created directory, without
.Rdata and .Rhistory files.
=== begin
R version 2.5.0 Patched (2007-05-13 r41549)
Copyright (C) 2007 The R Foundation for Statistical Computing
ISBN 3-900051-07-0
R is free software and comes with ABSOLUTELY NO WARRANTY.
You are welcome to redistribute it under certain conditions.
Type 'license()' or 'licence()' for distribution details.
R is a collaborative project with many contributors.
Type 'contributors()' for more information and
'citation()' on how to cite R or R packages in publications.
Type 'demo()' for some demos, 'help()' for on-line help, or
'help.start()' for an HTML browser interface to help.
Type 'q()' to quit R.
> library(ac9)
> AB2C(matrix(rnorm(90),ncol=9),matrix(rnorm(90),ncol=9))
Error in AB2C(matrix(rnorm(90), ncol = 9), matrix(rnorm(90), ncol = 9)) :
object "ac9nw" not found
> options(error=recover)
> AB2C(matrix(rnorm(90),ncol=9),matrix(rnorm(90),ncol=9))
Error in AB2C(matrix(rnorm(90), ncol = 9), matrix(rnorm(90), ncol = 9)) :
object "ac9nw" not found
Enter a frame number, or 0 to exit
1: AB2C(matrix(rnorm(90), ncol = 9), matrix(rnorm(90), ncol = 9))
Selection: 1
Called from: eval(expr, envir, enclos)
Browse[1]> ls()
[1] "a" "b" "model.type" "nwa""nwb"
[6] "nza""nzb"
Browse[1]> search()
[1] ".GlobalEnv""package:ac9" "package:stats"
[4] "package:graphics" "package:grDevices" "package:utils"
[7] "package:datasets" "package:methods" "Autoloads"
[10] "package:base"
Browse[1]> n
Enter a frame number, or 0 to exit
1: AB2C(matrix(rnorm(90), ncol = 9), matrix(rnorm(90), ncol = 9))
Selection: 1
Browse[1]> n
Enter a frame number, or 0 to exit
1: AB2C(matrix(rnorm(90), ncol = 9), matrix(rnorm(90), ncol = 9))
Selection: 0
=== Pause of the session, don't quit from R.
The function AB2C tries to work, if I use the data function.
But this data function makes the ac9nw visible in the global environment.
=== Continuing the sample session:
> data(ac9nw)
> ls()
[1] "ac9nw"
> ac9nw
[1] 9
> AB2C(matrix(rnorm(90),ncol=9),matrix(rnorm(90),ncol=9))
Error in eval(expr, envir, enclos) : object "Pro" not found
Enter a frame number, or 0 to exit
1: AB2C(matrix(rnorm(90), ncol = 9), matrix(rnorm(90), ncol = 9))
2: NextMethod("AB2C")
3: AB2C.S(matrix(rnorm(90), ncol = 9), matrix(rnorm(90), ncol = 9))
4: lm(t(cbind(a, b)) ~ Pro + syn + unk + Y + Nh + 0, offset = c(ac9aw,
ac9bw))
5: eval(mf, parent.frame())
6: eval(expr, envir, enclos)
7: model.frame(formula = t(cbind(a, b)) ~ Pro + syn + unk + Y + Nh + 0,
offset = c(ac9a
8: model.frame.default(formula = t(cbind(a, b)) ~ Pro + syn + unk + Y + Nh
+ 0, offset
9: eval(predvars, data, env)
10: eval(expr, envir, enclos)
Selection: 0
The function AB2C now sees ac9nw, and gives similar errors, related to other
hidden variables.
Duncan Murdoch-2 wrote:
>
> On 5/25/2007 7:27 AM, Vladimir Eremeev wrote:
>> Thank you for the answer.
>>
>> Here is everything.
>>
>> ac9wl<-c(412,440,488,510,532,555,650,676,715)
>> ac9nw<-length(ac9wl)
>>
>> AB2C <-function(a,b,model.type="S") {
>> nza<-dim(a)[1]
>> nwa<-dim(a)[2]
>> nzb<-dim(b)[1]
>> nwb<-dim(b)[2]
>>
>> if(nza!=nzb || nwa!=ac9nw || nwb!=ac9nw) {
>> warning('AB2C: Dimensions of the input matrices are invalid')
>> return(NULL);
>> }
>>
>> [ further calculations, they do not use ac9nw ]
>> }
>
>
> That would work, so my guess is that you aren't using that function. If
> you have a function named AB2C in your global environment, R will find
> it before this one, and it may not be able to see the ac9nw variable.
>
> Duncan Murdoch
>
>
--
View this message in context:
http://www.nabble.com/private-variables-in-package.-tf3815306.html#a10801823
Sent from the R devel mailing list archive at Nabble.com.
__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
[Rd] solved.
By removing the data directory (which package.skeleton has created, and where
it has put all my variables in files with .rda extension) and adding one
more file to the R directory, containing the variable assignments.
Vladimir Eremeev wrote:
>
> I do use this function.
> Here is the example session, run from the newly created directory, without
> .Rdata and .Rhistory files.
>
> === begin
> R version 2.5.0 Patched (2007-05-13 r41549)
> Copyright (C) 2007 The R Foundation for Statistical Computing
> ISBN 3-900051-07-0
>
> R is free software and comes with ABSOLUTELY NO WARRANTY.
> You are welcome to redistribute it under certain conditions.
> Type 'license()' or 'licence()' for distribution details.
>
> R is a collaborative project with many contributors.
> Type 'contributors()' for more information and
> 'citation()' on how to cite R or R packages in publications.
>
> Type 'demo()' for some demos, 'help()' for on-line help, or
> 'help.start()' for an HTML browser interface to help.
> Type 'q()' to quit R.
>
>> library(ac9)
>> AB2C(matrix(rnorm(90),ncol=9),matrix(rnorm(90),ncol=9))
> Error in AB2C(matrix(rnorm(90), ncol = 9), matrix(rnorm(90), ncol = 9)) :
> object "ac9nw" not found
>> options(error=recover)
>> AB2C(matrix(rnorm(90),ncol=9),matrix(rnorm(90),ncol=9))
> Error in AB2C(matrix(rnorm(90), ncol = 9), matrix(rnorm(90), ncol = 9)) :
> object "ac9nw" not found
>
> Enter a frame number, or 0 to exit
>
> 1: AB2C(matrix(rnorm(90), ncol = 9), matrix(rnorm(90), ncol = 9))
>
> Selection: 1
> Called from: eval(expr, envir, enclos)
> Browse[1]> ls()
> [1] "a" "b" "model.type" "nwa""nwb"
> [6] "nza""nzb"
> Browse[1]> search()
> [1] ".GlobalEnv""package:ac9" "package:stats"
> [4] "package:graphics" "package:grDevices" "package:utils"
> [7] "package:datasets" "package:methods" "Autoloads"
> [10] "package:base"
> Browse[1]> n
>
> Enter a frame number, or 0 to exit
>
> 1: AB2C(matrix(rnorm(90), ncol = 9), matrix(rnorm(90), ncol = 9))
>
> Selection: 1
> Browse[1]> n
>
> Enter a frame number, or 0 to exit
>
> 1: AB2C(matrix(rnorm(90), ncol = 9), matrix(rnorm(90), ncol = 9))
> Selection: 0
>
>
> === Pause of the session, don't quit from R.
>
> The function AB2C tries to work, if I use the data function.
> But this data function makes the ac9nw visible in the global environment.
>
> === Continuing the sample session:
>
>> data(ac9nw)
>> ls()
> [1] "ac9nw"
>> ac9nw
> [1] 9
>
>> AB2C(matrix(rnorm(90),ncol=9),matrix(rnorm(90),ncol=9))
> Error in eval(expr, envir, enclos) : object "Pro" not found
>
> Enter a frame number, or 0 to exit
>
> 1: AB2C(matrix(rnorm(90), ncol = 9), matrix(rnorm(90), ncol = 9))
> 2: NextMethod("AB2C")
> 3: AB2C.S(matrix(rnorm(90), ncol = 9), matrix(rnorm(90), ncol = 9))
> 4: lm(t(cbind(a, b)) ~ Pro + syn + unk + Y + Nh + 0, offset = c(ac9aw,
> ac9bw))
> 5: eval(mf, parent.frame())
> 6: eval(expr, envir, enclos)
> 7: model.frame(formula = t(cbind(a, b)) ~ Pro + syn + unk + Y + Nh + 0,
> offset = c(ac9a
> 8: model.frame.default(formula = t(cbind(a, b)) ~ Pro + syn + unk + Y +
> Nh + 0, offset
> 9: eval(predvars, data, env)
> 10: eval(expr, envir, enclos)
>
> Selection: 0
>
> The function AB2C now sees ac9nw, and gives similar errors, related to
> other hidden variables.
>
>
> Duncan Murdoch-2 wrote:
>>
>> On 5/25/2007 7:27 AM, Vladimir Eremeev wrote:
>>> Thank you for the answer.
>>>
>>> Here is everything.
>>>
>>> ac9wl<-c(412,440,488,510,532,555,650,676,715)
>>> ac9nw<-length(ac9wl)
>>>
>>> AB2C <-function(a,b,model.type="S") {
>>> nza<-dim(a)[1]
>>> nwa<-dim(a)[2]
>>> nzb<-dim(b)[1]
>>> nwb<-dim(b)[2]
>>>
>>> if(nza!=nzb || nwa!=ac9nw || nwb!=ac9nw) {
>>> warning('AB2C: Dimensions of the input matrices are invalid')
>>> return(NULL);
>>> }
>>>
>>> [ further calculations, they do not use ac9nw ]
>>> }
>>
>>
>> That would work, so my guess is that you aren't using that function. If
>> you have a function named AB2C in your global environment, R will find
>> it before this one, and it may not be able to see the ac9nw variable.
>>
>> Duncan Murdoch
>>
>>
>
>
--
View this message in context:
http://www.nabble.com/private-variables-in-package.-tf3815306.html#a10802257
Sent from the R devel mailing list archive at Nabble.com.
__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] private variables in package.
R has debugging features: see 'Writing R Extensions'. Please make use of them, e.g. options(error=recover) will enable you to explore the environments that are visible. We can only guess at this, not having function AB2C. On Fri, 25 May 2007, Vladimir Eremeev wrote: > > I am sorry, my previous questions about hiding variables from users in a > package were obscure and vague, and would like to clarify them. > > This should be a basic question. > I am writing my first package. > It has several functions, which should be callable by users. > These functions use several variables, and I would like to hide these > variables from users. > How can I do it? > > For example, the package has a function AB2C, which uses the variable ac9nw. > If I add the file NAMESPACE, exporting only AB2C, then I get the error: > > Error in AB2C(a, b, model.type) : object "ac9nw" not found > > I also tried to include in the .onLoad function the code, defining these > variables with <<- > However, this code has created all internal variables in the global > environment. > > > -- Brian D. Ripley, [EMAIL PROTECTED] Professor of Applied Statistics, http://www.stats.ox.ac.uk/~ripley/ University of Oxford, Tel: +44 1865 272861 (self) 1 South Parks Road, +44 1865 272866 (PA) Oxford OX1 3TG, UKFax: +44 1865 272595 __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
[Rd] documented/undocumented behavior of as.double(formatC(x, digits=17))
Some days ago, there was a discussion about the command formatC(exp(1),digits=100,width=-1) Converting a double value to a string, from which the double may be reconstructed exactly, may be useful. So, I did some experimentation with it in my linux installation of R-2.5.0. I generated a vector x of a large number of random doubles (random sign, random mantissa with 53 significant bits and binary exponent from -1074:1023) and did the following y <- formatC(x,digits=17,width=-1) z <- as.double(y) all(x == z) # TRUE On Wed, May 23, 2007 at 06:32:36PM +0100, Prof Brian Ripley wrote: > However, the C99 standard does make clear that [sf]printf is not required > (even as 'recommended practice') to be accurate to more than *_DIG places, > which as ?as.character has pointed out is 15 on the machines R runs on. > > It really is the case that writing out a number to > 15 significant digits > and reading it back in again is not required to give exactly the same > IEC60559 (sic) number, and furthermore there are R platforms for which > this does not happen. I think that this implies that preserving the exact double value in formatC does not follow from C99 standard. Is there some other (e.g. platform specific) documentation that implies this or it has to be used as an undocumented feature? (Well, I do know that the R sources or gcc sources are such a documentation, but I am trying to find some other). Petr. __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
[Rd] Recent changes in R related to CHARSXPs
Hello all,
I want to highlight a recent change in R-devel to the larger developeR
community. As of r41495, R maintains a global cache of CHARSXPs such
that each unique string is stored only once in memory. For many
common use cases, such as dimnames of matrices and keys in
environments, the result is a significant savings in memory (and time
under some circumstances).
A result of these changes is that CHARSXPs must be treated as read
only objects and must never be modified in-place by assigning to the
char* returned by CHAR(). If you maintain a package that manipulates
CHARSXPs, you should check to see if you make such in-place
modifications. If you do, the general solution is as follows:
If you need a temp char buffer, you can allocate one with a new
helper macro like this:
/* CallocCharBuf takes care of the +1 for the \0,
so the size argument is the length of your string.
*/
char *tmp = CallocCharBuf(n);
/* manipulate tmp */
SEXP schar = mkChar(tmp);
Free(tmp);
You can also use R_alloc which has the advantage of not having to
free it in a .Call function.
The mkChar function now consults the global CHARSXP cache and will
return an already existing CHARSXP if one with a matching string
exists. Otherwise, it will create a new one and add it to the cache
before returning it.
In a discussion with Herve Pages, he suggested that the return type of
CHAR(), at least for package code, be modified from (char *) to (const
char *). I think this is an excellent suggestion because it will
allow the compiler to alert us to package C code that might be
modifying CHARSXPs in-place. This hasn't happened yet, but I'm hoping
that a patch for this will be applied soon (unless better suggestions
for improvement arise through this discussion :-)
One other thing is worth mentioning: at present, not all CHARSXPs are
captured by the cache. I think the goal is to refine things so that
all CHARSXPs _are_ in the cache. At that point, strcmp calls can be
replaced with pointer comparisons which should provide some nice
speed ups. So part of the idea is that the way to get CHARSXPs is via
mkChar or mkString and that one should not use allocString, etc.
Finally, here is a comparison of time and memory for loading all the
environments (hash tables) in Bioconductor's GO annotation data
package.
## unpatched
> gc()
used (Mb) gc trigger (Mb) max used (Mb)
Ncells 168891 9.1 35 18.7 35 18.7
Vcells 115731 0.9 786432 6.0 425918 3.3
> library("GO")
> system.time(for (e in ls(2)) get(e))
user system elapsed
51.919 1.168 53.228
> gc()
used (Mb) gc trigger (Mb) max used (Mb)
Ncells 17879072 954.9 19658017 1049.9 18683826 997.9
Vcells 31702823 241.9 75190268 573.7 53912452 411.4
## patched
> gc()
used (Mb) gc trigger (Mb) max used (Mb)
Ncells 154717 8.3 35 18.7 35 18.7
Vcells 133613 1.1 786432 6.0 483138 3.7
> library("GO")
> system.time(for (e in ls(2)) get(e))
user system elapsed
31.166 0.736 31.998
> gc()
used (Mb) gc trigger (Mb) max used (Mb)
Ncells 5837253 311.86910418 369.1 6193578 330.8
Vcells 16831859 128.5 45712717 348.8 39456690 301.1
Best Wishes,
+ seth
--
Seth Falcon | Computational Biology | Fred Hutchinson Cancer Research Center
http://bioconductor.org
__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] C code for qbeta
On Fri, 25 May 2007, Sean Davis wrote: > On Friday 25 May 2007 05:02, Gatsu wrote: >> Can somebody help me? >> >> I need the C/C++ code for the R's qbeta function. > > R is open-source, so you can simply download the source and look at whatever > parts you like. > Finding the qbeta function may take a little effort,though. The qbeta code is at https://svn.r-project.org/R/trunk/src/nmath/qbeta.c -thomas __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
[Rd] R scripts slowing down after repeated called to compiled code
Thanks in advance to anyone that might be able to help me with this problem. I have not been able to find a reference to it in the documentation on online sources, so I am turning to this group. I am running R 2.4.1 under Red Hat Enterprise Linux 4, on an x86_64 platform (multi-core Intel Xeon processors, 3.6Ghx, 8GB of RAM). I have some rather complicated code (so I won't attach it here), but it is an iterative algorithm that takes data in the form of an R list, passes the entire list to some compiled C code, converts list items to GSL matrices and vectors, performs its various operations, and sends back the result to R. That result is then sent back to the compiled code until some kind of convergence (I won't bore the group with more details). The problem is that every .Call to the compiled code runs slower (I print the iteration time using Sys.time() and difftime() ). There is no logical reason for this (i.e., it's not a feature of the algorithm itself). I am using about 20% of my machine's available RAM (it's a large dataset and a memory-intensive algorithm), but there does not appear to be any swapping of memory to disk. I am sure that I am UNPROTECTING the SEXP's that I created, and I am freeing all of the GSL objects at the end of each function. The total RAM used does seem to increase, slowly but steadily, but the speed decrease occurs well before I start coming close to running out of RAM. Also, it is not just the compiled call that slows down. EVERYTHING slows down, even those that consist only of standard R functions. The time for each of these function calls is roughly proportional to the time of the .Call to the C function. Another observation is that when I terminate the algorithm, do a rm (list=ls()), and then a gc(), not all of the memory is returned to the OS. It is not until I terminate the R session that I get all of the memory back. In my C code, I am not doing anything to de-allocate the SEXP's I create, relying on the PROTECT/UNPROTECT mechanism instead (is this right?). I spent most of the day thinking I have a memory leak, but that no longer appears to be the case. I tried using Rprof(), but that only gives me the aggregated relative time spent in each function (more than 80% of the time, it's in the .Call). So I'm stuck. Can anyone help? Thanks, Michael -- Michael Braun Assistant Professor of Marketing MIT Sloan School of Management One Amherst St., E40-169 Cambridge, MA 02142 (617) 253-3436 [EMAIL PROTECTED] __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] R scripts slowing down after repeated called to compiled code
On Friday 25 May 2007 7:12 pm, Michael Braun wrote: > Thanks in advance to anyone that might be able to help me with this > > Also, it is not just the compiled call that slows down. EVERYTHING > slows down, even those that consist only of standard R functions. The > time for each of these function calls is roughly proportional to the > time of the .Call to the C function. > > Another observation is that when I terminate the algorithm, do a rm > (list=ls()), and then a gc(), not all of the memory is returned to the > OS. It is not until I terminate the R session that I get all of the > memory back. In my C code, I am not doing anything to de-allocate the > SEXP's I create, relying on the PROTECT/UNPROTECT mechanism instead (is > this right?). > > I spent most of the day thinking I have a memory leak, but that no > longer appears to be the case. I tried using Rprof(), but that only > gives me the aggregated relative time spent in each function (more than > 80% of the time, it's in the .Call). One possibility is that you are somehow creating a lot of R objects (say by calling assign() or missing UNPROTECT()) and this slows garbage collector down. The garbage collector running time will grow with the number of objects you have - their total size does not have to be large. Could you try printing numbers from gc() call and checking whether the numbers of allocated objects grow a lot ? best Vladimir Dergachev > > So I'm stuck. Can anyone help? > > Thanks, > > Michael __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] R scripts slowing down after repeated called to compiled code
Vladimir: Thanks for your response. I have done as you requested (with a smaller dataset--the patterns are still there). Here is the gc()-laden output. I don;t fully understand the meaning of all parts of these reports. But it does look like the number of objects grows (and it grows faster on the larger, slower problem). But how can I use this information to narrow down the cause? As far as your other suggestions, my PROTECTs and UNPROTECTs are all balanced (I'd get a stack imbalance if they weren't). One thing I am doing is duplicating the SEXP's that are passed from R, and working with the copies. I unprotect the copies at the end of the function. There are other, various VECTOR_ELT's in my code. So where do I go from there? Michael Dataset B06 Iteration: 1 Garbage collection 67 = 33+9+25 (level 2) ... 778324 cons cells free (42%) 32.4 Mbytes of heap free (28%) Dataset B06 Iteration: 2 Garbage collection 92 = 52+11+29 (level 2) ... 778303 cons cells free (42%) 43.9 Mbytes of heap free (34%) Dataset B06 Iteration: 3 Garbage collection 112 = 69+12+31 (level 2) ... 778303 cons cells free (42%) 43.9 Mbytes of heap free (34%) Dataset B06 Iteration: 4 Garbage collection 126 = 79+13+34 (level 2) ... 778303 cons cells free (42%) 50.7 Mbytes of heap free (38%) Dataset B06 Iteration: 5 Garbage collection 139 = 89+14+36 (level 2) ... 778303 cons cells free (42%) 50.7 Mbytes of heap free (38%) Dataset B06 Iteration: 6 Garbage collection 149 = 95+15+39 (level 2) ... 778303 cons cells free (42%) 58.0 Mbytes of heap free (41%) Dataset B06 Iteration: 7 Garbage collection 157 = 101+15+41 (level 2) ... 778303 cons cells free (42%) 58.0 Mbytes of heap free (41%) Dataset B06 Iteration: 8 Garbage collection 165 = 107+15+43 (level 2) ... 778303 cons cells free (42%) 58.0 Mbytes of heap free (41%) Dataset B06 Iteration: 9 Garbage collection 173 = 112+16+45 (level 2) ... 778303 cons cells free (42%) 58.0 Mbytes of heap free (41%) Dataset B06 Iteration: 10 Garbage collection 181 = 118+16+47 (level 2) ... 778303 cons cells free (42%) 58.0 Mbytes of heap free (41%) Dataset B06 Iteration: 11 Garbage collection 188 = 123+16+49 (level 2) ... 778303 cons cells free (42%) 58.0 Mbytes of heap free (41%) Dataset B06 Iteration: 12 Garbage collection 195 = 128+16+51 (level 2) ... 778303 cons cells free (42%) 58.0 Mbytes of heap free (41%) Dataset B06 Iteration: 13\ Garbage collection 202 = 132+17+53 (level 2) ... 778303 cons cells free (42%) 58.0 Mbytes of heap free (41%) Dataset B06 Iteration: 14 Garbage collection 209 = 137+17+55 (level 2) ... 778303 cons cells free (42%) 58.0 Mbytes of heap free (41%) On Fri, 2007-05-25 at 19:29 -0400, Vladimir Dergachev wrote: > On Friday 25 May 2007 7:12 pm, Michael Braun wrote: > > Thanks in advance to anyone that might be able to help me with this > > > > Also, it is not just the compiled call that slows down. EVERYTHING > > slows down, even those that consist only of standard R functions. The > > time for each of these function calls is roughly proportional to the > > time of the .Call to the C function. > > > > Another observation is that when I terminate the algorithm, do a rm > > (list=ls()), and then a gc(), not all of the memory is returned to the > > OS. It is not until I terminate the R session that I get all of the > > memory back. In my C code, I am not doing anything to de-allocate the > > SEXP's I create, relying on the PROTECT/UNPROTECT mechanism instead (is > > this right?). > > > > I spent most of the day thinking I have a memory leak, but that no > > longer appears to be the case. I tried using Rprof(), but that only > > gives me the aggregated relative time spent in each function (more than > > 80% of the time, it's in the .Call). > > One possibility is that you are somehow creating a lot of R objects (say by > calling assign() or missing UNPROTECT()) and this slows garbage collector > down. The garbage collector running time will grow with the number of objects > you have - their total size does not have to be large. > > Could you try printing numbers from gc() call and checking whether the > numbers > of allocated objects grow a lot ? > > best > > Vladimir Dergachev > > > > > So I'm stuck. Can anyone help? > > > > Thanks, > > > > Michael > > > The information contained in this transmission may contain privileged and > confidential information. It is intended only for the use of the person(s) > named above. If you are not the intended recipient, you are hereby notified > that any review, dissemination, distribution or duplication of this > communication is strictly prohibited. If you are not the intended recipient, > please contact the sender by reply email and destroy all copies of the > original message. > > -- Michael Braun Assistant Professor of Marketing MIT Sloan School of Management One Amher
Re: [Rd] R scripts slowing down after repeated called to compiled code
On 25 May 2007 at 19:12, Michael Braun wrote: | So I'm stuck. Can anyone help? It sounds like a memory issue. Your memory may just get fragmented. One tool that may help you find leaks is valgrind -- see the 'R Extensions' manual. I can also recommend the visualisers like kcachegrind (part of KDE). But it may not be a leak. I found that R just doesn't cope well with many large memory allocations and releases -- I often loop over data request that I subset and process. This drives my 'peak' memory use to 1.5 or 1.7gb on 32bit/multicore machine with 4gb, 6gb or 8gb (but 32bit leading to the hard 3gb per process limit) . And I just can't loop over many such task. So I now use the littler frontend to script this, dump the processed chunks as Rdata files and later re-read the pieces. That works reliably. So one think you could try is to dump your data in 'gsl ready' format from R, quit R, leave it out of the equation and then see if what happens if you do the iterations in only GSL and your code. Hth, Dirk -- Hell, there are no rules here - we're trying to accomplish something. -- Thomas A. Edison __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
