Re: [Rd] Shapiro Test P Value Incorrect? (PR#9768)
This is not a bug. The algorithm uses different approximation of the p-value for n=3 (exact value), 4<=n<=11 and n>=12 as seen in src/library/stats/src/swilk.c below the line 202 /* Calculate significance level for W */ The W statistic monotonically decreases in the presented example. Petr. > Full_Name: Jason Polak > Version: R version 2.5.0 (2007-04-23) > OS: Xubuntu 7.04 > Submission from: (NULL) (137.122.144.35) > > > Dear R group, > > I have noticed a strange anomaly with the shapiro.test() function. > Unfortunately > I do not know how to calculate the shapiro test P values manually so I don't > know if this is an actual bug. > > So, to produce the results, run the following code: > > pvalues = 0; > for (i in 1:17) > { > j = 1:(i+3); > pvalues[i]=shapiro.test(j)$p; > } > > plot(pvalues); > print(pvalues); > > Now I just made the graph to illustrate that the p-values are strictly > decreasing. To me this makes intuitive sense: we are using the Shapiro test to > test normality of (1,2,3,4),(1,2,3,4,5), and so on. So the p-value should > decrease. > > These are the p-values: > [1] 0.9718776 0.9671740 0.9605557 0.9492892 0.9331653 0.9135602 0.8923668 > [8] 0.8698419 0.8757315 0.8371814 0.7964400 0.7545289 0.7123167 0.6704457 > [15] 0.6294307 0.5896464 0.5513749 > > However, there is an increase in p-value when you go from (1,..,11) to > (1,..,12). Is this just a quirk of the Shapiro test, or is there an error in > the > calculation algorithm? > > __ > R-devel@r-project.org mailing list > https://stat.ethz.ch/mailman/listinfo/r-devel > __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
[Rd] How to disable R's C stack checking
Hi all, I'm developing an application on Mac OS X Darwin which embeds R. However, the application always crashes due to the C stack checking. I know that R_CStackLimit can be set to -1 to disable the stack checking, but I don't know where to put the code "R_CStackLimit=-1". Please give some instructions. Thank you very much! Here is my compiler infomation: $g++ -v Using built-in specs. Target: i686-apple-darwin8 Configured with: /private/var/tmp/gcc/gcc-5367.obj~1/src/configure --disable-checking -enable-werror --prefix=/usr --mandir=/share/man --enable-languages=c,objc,c++,obj-c++ --program-transform-name=/^[cg][^.-]*$/s/$/-4.0/ --with-gxx-include-dir=/include/c++/4.0.0 --with-slibdir=/usr/lib --build=powerpc-apple-darwin8 --with-arch=nocona --with-tune=generic --program-prefix= --host=i686-apple-darwin8 --target=i686-apple-darwin8 Thread model: posix gcc version 4.0.1 (Apple Computer, Inc. build 5367) My machine infomation: $uname -a Darwin xi-imac 8.9.1 Darwin Kernel Version 8.9.1 Whin. __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
[Rd] Hook for running a function before evaluation
Hello, I like much addTaskCallback() and friends. However, there are situations were we would like to have a function run BEFORE, not after every top-level tasks. I think specifically to reset options(width = ) to accommodate to the current width of the console, using something like: options(width = system("tput cols")) I am sure there are many other situations where this could be useful. Combining tasks run before and after evaluation, one could design a system to (remotely) indicate when a given R kernel is starting/evaluating/ending evaluation of some code (with Rpad in mind, for instance). How difficult would it be to implement such Tasks run before top-level ones? Best, Philippe -- ..<°}))>< ) ) ) ) ) ( ( ( ( (Prof. Philippe Grosjean ) ) ) ) ) ( ( ( ( (Numerical Ecology of Aquatic Systems ) ) ) ) ) Mons-Hainaut University, Belgium ( ( ( ( ( .. __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] Shapiro Test P Value Incorrect? (PR#9768)
On Fri, 6 Jul 2007, Petr Savicky wrote: > This is not a bug. The algorithm uses different approximation of the > p-value for n=3 (exact value), 4<=n<=11 and n>=12 as seen in > src/library/stats/src/swilk.c > below the line 202 > /* Calculate significance level for W */ > > The W statistic monotonically decreases in the presented example. The crucial piece missing from the help page (but in the references) is that (at least some of) the approximations used are only claimed to be adequate for p < 0.1. I've added a line to that effect. > Petr. > >> Full_Name: Jason Polak >> Version: R version 2.5.0 (2007-04-23) >> OS: Xubuntu 7.04 >> Submission from: (NULL) (137.122.144.35) >> >> >> Dear R group, >> >> I have noticed a strange anomaly with the shapiro.test() function. >> Unfortunately >> I do not know how to calculate the shapiro test P values manually so I don't >> know if this is an actual bug. >> >> So, to produce the results, run the following code: >> >> pvalues = 0; >> for (i in 1:17) >> { >> j = 1:(i+3); >> pvalues[i]=shapiro.test(j)$p; >> } >> >> plot(pvalues); >> print(pvalues); >> >> Now I just made the graph to illustrate that the p-values are strictly >> decreasing. To me this makes intuitive sense: we are using the Shapiro test >> to >> test normality of (1,2,3,4),(1,2,3,4,5), and so on. So the p-value should >> decrease. >> >> These are the p-values: >> [1] 0.9718776 0.9671740 0.9605557 0.9492892 0.9331653 0.9135602 0.8923668 >> [8] 0.8698419 0.8757315 0.8371814 0.7964400 0.7545289 0.7123167 0.6704457 >> [15] 0.6294307 0.5896464 0.5513749 >> >> However, there is an increase in p-value when you go from (1,..,11) to >> (1,..,12). Is this just a quirk of the Shapiro test, or is there an error in >> the >> calculation algorithm? >> >> __ >> R-devel@r-project.org mailing list >> https://stat.ethz.ch/mailman/listinfo/r-devel >> > > __ > R-devel@r-project.org mailing list > https://stat.ethz.ch/mailman/listinfo/r-devel > -- Brian D. Ripley, [EMAIL PROTECTED] Professor of Applied Statistics, http://www.stats.ox.ac.uk/~ripley/ University of Oxford, Tel: +44 1865 272861 (self) 1 South Parks Road, +44 1865 272866 (PA) Oxford OX1 3TG, UKFax: +44 1865 272595 __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] problem with findFun call from embedded R
Byron Ellis wrote: > Yeah, setting R_Interactive is probably fine. Great -- thanks for the confirmation. > As to the findFun issue, > there is a much more useful function (findVar1 maybe?) that wouldn't > die if the object didn't exist. OK, I'll poke around R sources some more. The function provided by Jeffrey Horner on a nearby post seems to work well too. Thanks, Joe __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] problem with findFun call from embedded R
Joe Conway wrote: Yes, if I patch RNamedCall.c with the attached (based on your function), everything seems to work. Sorry -- I managed to mess up that patch. This one should be better. Joe *** ../../../R-2.5.1/tests/Embedding/RNamedCall.c Mon Sep 4 07:21:12 2006 --- RNamedCall.c Fri Jul 6 10:19:46 2007 *** *** 2,7 --- 2,9 void bar1() ; void source(const char *name); + static SEXP MyfindFun(SEXP symb, SEXP envir); + /* Creates and evaluates a call to a function giving named arguments *** *** 51,58 SEXP e; PROTECT(e = allocVector(LANGSXP, 4)); ! fun = findFun(install("foo"), R_GlobalEnv); ! if(fun == R_NilValue) { fprintf(stderr, "No definition for function foo. Source foo.R and save the session.\n"); UNPROTECT(1); exit(1); --- 53,60 SEXP e; PROTECT(e = allocVector(LANGSXP, 4)); ! fun = MyfindFun(install("foo"), R_GlobalEnv); ! if(fun == R_UnboundValue) { fprintf(stderr, "No definition for function foo. Source foo.R and save the session.\n"); UNPROTECT(1); exit(1); *** *** 80,82 --- 82,104 UNPROTECT(1); } + static SEXP + MyfindFun(SEXP symb, SEXP envir) + { + SEXP fun; + SEXPTYPE t; + fun = findVar(symb,envir); + t = TYPEOF(fun); + + /* eval promise if need be */ + if (t == PROMSXP){ + int error=1; + fun = R_tryEval(fun,envir,&error); + if (error) return R_UnboundValue; + t = TYPEOF(fun); + } + + if (t == CLOSXP || t == BUILTINSXP || t == BUILTINSXP || t == SPECIALSXP) + return fun; + return R_UnboundValue; + } __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] problem with findFun call from embedded R
Jeffrey Horner wrote: Joe Conway wrote: I was debugging a problem reported to me regarding PL/R, and found that I can duplicate it using only R sources. It might be characterized as possibly a misuse of the findFun() function, but I leave that for the R devel experts to decide. My question is -- is the above an abuse of findFun() by RNamedCall.c (meaning I should avoid the same pattern)? Or maybe it should be wrapped in something similar to R_ToplevelExec? Yes, when one embeds R, findFun() will call error() which causes the longjmp() to the top level, so the embedding application shouldn't use findFun(). OK, thanks for the confirmation. I just wonder if RNamedCall.c ought to be changed then... Here's what I've written for the next version of RApache. Note that it's behavior is slightly different than the original findFun(), in that it doesn't search the enclosing environments. (Also, comments welcome as the implementation is based on the original, and I'm unsure if promise evaluation is needed.) /* This one doesn't longjmp when function not found */ static SEXP MyfindFun(SEXP symb, SEXP envir){ Yes, if I patch RNamedCall.c with the attached (based on your function), everything seems to work. Thanks, Joe *** ../../../R-2.5.1/tests/Embedding/RNamedCall.c Fri Jul 6 10:28:18 2007 --- RNamedCall.c Fri Jul 6 10:19:46 2007 *** *** 2,8 void bar1() ; void source(const char *name); ! SEXP MyfindFun(SEXP symb, SEXP envir); /* Creates and evaluates a call --- 2,8 void bar1() ; void source(const char *name); ! static SEXP MyfindFun(SEXP symb, SEXP envir); /* Creates and evaluates a call *** *** 82,88 UNPROTECT(1); } ! SEXP MyfindFun(SEXP symb, SEXP envir) { SEXP fun; --- 82,88 UNPROTECT(1); } ! static SEXP MyfindFun(SEXP symb, SEXP envir) { SEXP fun; __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] Hook for running a function before evaluation
Hi Philippe, This is exactly the approach being taken in RExecServer (which is in turn possibly connected to front ends via distributed objects). It's an OS X app so the model is my_ReadConsole(...) { didFinishEvaluationForInterpreter(...); internalReadConsole(...); (usually start the run loop and wait for an appropriate event) willBeginEvaluationForInterpreter(...); } technically there should also be shouldBeginEvaluationforInterpreter() to let us bail out of evaluation and head back to the reader loop. http://repo.or.cz/w/RExecServer.git On 7/6/07, Philippe Grosjean <[EMAIL PROTECTED]> wrote: > Hello, > I like much addTaskCallback() and friends. However, there are situations > were we would like to have a function run BEFORE, not after every > top-level tasks. I think specifically to reset options(width = ) to > accommodate to the current width of the console, using something like: > > options(width = system("tput cols")) > > I am sure there are many other situations where this could be useful. > Combining tasks run before and after evaluation, one could design a > system to (remotely) indicate when a given R kernel is > starting/evaluating/ending evaluation of some code (with Rpad in mind, > for instance). > > How difficult would it be to implement such Tasks run before top-level ones? > > Best, > > Philippe > -- > ..<°}))>< > ) ) ) ) ) > ( ( ( ( (Prof. Philippe Grosjean > ) ) ) ) ) > ( ( ( ( (Numerical Ecology of Aquatic Systems > ) ) ) ) ) Mons-Hainaut University, Belgium > ( ( ( ( ( > .. > > __ > R-devel@r-project.org mailing list > https://stat.ethz.ch/mailman/listinfo/r-devel > -- Byron Ellis ([EMAIL PROTECTED]) "Oook" -- The Librarian __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] Hook for running a function before evaluation
Hi Byron, Excellent! I am also on OS X (together with Win XP and Quantian using Parallels desktop ;-) So far, so good, I have the latest RExecServer running in a terminal. So, what can I do with it? Best, Philippe ..<°}))>< ) ) ) ) ) ( ( ( ( (Prof. Philippe Grosjean ) ) ) ) ) ( ( ( ( (Numerical Ecology of Aquatic Systems ) ) ) ) ) Mons-Hainaut University, Belgium ( ( ( ( ( .. Byron Ellis wrote: > Hi Philippe, > > This is exactly the approach being taken in RExecServer (which is in > turn possibly connected to front ends via distributed objects). It's > an OS X app so the model is > > my_ReadConsole(...) { > didFinishEvaluationForInterpreter(...); > internalReadConsole(...); (usually start the run loop and wait for an > appropriate event) > willBeginEvaluationForInterpreter(...); > } > > technically there should also be shouldBeginEvaluationforInterpreter() > to let us bail out of evaluation and head back to the reader loop. > > http://repo.or.cz/w/RExecServer.git > > On 7/6/07, Philippe Grosjean <[EMAIL PROTECTED]> wrote: >> Hello, >> I like much addTaskCallback() and friends. However, there are situations >> were we would like to have a function run BEFORE, not after every >> top-level tasks. I think specifically to reset options(width = ) to >> accommodate to the current width of the console, using something like: >> >> options(width = system("tput cols")) >> >> I am sure there are many other situations where this could be useful. >> Combining tasks run before and after evaluation, one could design a >> system to (remotely) indicate when a given R kernel is >> starting/evaluating/ending evaluation of some code (with Rpad in mind, >> for instance). >> >> How difficult would it be to implement such Tasks run before top-level >> ones? >> >> Best, >> >> Philippe >> -- >> ..<°}))>< >> ) ) ) ) ) >> ( ( ( ( (Prof. Philippe Grosjean >> ) ) ) ) ) >> ( ( ( ( (Numerical Ecology of Aquatic Systems >> ) ) ) ) ) Mons-Hainaut University, Belgium >> ( ( ( ( ( >> .. >> >> __ >> R-devel@r-project.org mailing list >> https://stat.ethz.ch/mailman/listinfo/r-devel >> > > __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] Hook for running a function before evaluation
Running from the Terminal, it acts as a simple shim for people who would prefer to use the Terminal or a terminal-like interface such as ESS. It provides a pretty basic graphics device implementation with a full event loop (without requiring CarbonEL or other workarounds) in that mode. That graphics device is also technically capable (though it doesn't actually write the file at the moment) of writing to a variety of bitmap formats as well as Quicktime movies without having to involve X11. You can also route PDF, though the built-in PDF device will likely give you smaller files. It's under heavy development (see http://statcomp.blogspot.com) at the moment so some things don't work often because there is an aqua-specific thing that needs to happen (viewing help for example) that we don't actually want to reroute away from the terminal (suspect functions usually contain if(Platform$GUI...) at the top). I'll get those all sorted out sooner rather than later. However, the real power and, I hope, primary use, isn't from the Terminal---although you can do some cute things like copy objects between any instance (they call themselves "R Execution Server 1," "R Execution Server 2" and so on or take a name you provide via environment variables). Each of the instances, unless told otherwise, is actually vending itself as a Distributed Object that can be attached from another process. The first client is obviously a standard R GUI (again, see the site above for a screenshot) that can run more than one instance of R at a time much the same way you'd just run from multiple xterms or what have you. It'll feel like a single GUI and have all of the current GUI's features w.r.t. script editing and so on. You'll just have more than one Console (this was actually Stefano's idea. I just wanted to see if it would work. :-) ). It's a pretty similar model to Simon's JGR stuff in spirit at least, though IIRC that is all still be running in the same process whereas Distributed Objects do not necessarily even need to be on the same machine. So, maybe a cross between Rserve and JGR then. :-) So far I'm actually quite surprised how quickly its come together. On 7/6/07, Philippe Grosjean <[EMAIL PROTECTED]> wrote: > Hi Byron, > > Excellent! I am also on OS X (together with Win XP and Quantian using > Parallels desktop ;-) > > So far, so good, I have the latest RExecServer running in a terminal. > So, what can I do with it? > Best, > > Philippe > > ..<°}))>< > ) ) ) ) ) > ( ( ( ( (Prof. Philippe Grosjean > ) ) ) ) ) > ( ( ( ( (Numerical Ecology of Aquatic Systems > ) ) ) ) ) Mons-Hainaut University, Belgium > ( ( ( ( ( > .. > > Byron Ellis wrote: > > Hi Philippe, > > > > This is exactly the approach being taken in RExecServer (which is in > > turn possibly connected to front ends via distributed objects). It's > > an OS X app so the model is > > > > my_ReadConsole(...) { > > didFinishEvaluationForInterpreter(...); > > internalReadConsole(...); (usually start the run loop and wait for an > > appropriate event) > > willBeginEvaluationForInterpreter(...); > > } > > > > technically there should also be shouldBeginEvaluationforInterpreter() > > to let us bail out of evaluation and head back to the reader loop. > > > > http://repo.or.cz/w/RExecServer.git > > > > On 7/6/07, Philippe Grosjean <[EMAIL PROTECTED]> wrote: > >> Hello, > >> I like much addTaskCallback() and friends. However, there are situations > >> were we would like to have a function run BEFORE, not after every > >> top-level tasks. I think specifically to reset options(width = ) to > >> accommodate to the current width of the console, using something like: > >> > >> options(width = system("tput cols")) > >> > >> I am sure there are many other situations where this could be useful. > >> Combining tasks run before and after evaluation, one could design a > >> system to (remotely) indicate when a given R kernel is > >> starting/evaluating/ending evaluation of some code (with Rpad in mind, > >> for instance). > >> > >> How difficult would it be to implement such Tasks run before top-level > >> ones? > >> > >> Best, > >> > >> Philippe > >> -- > >> ..<°}))>< > >> ) ) ) ) ) > >> ( ( ( ( (Prof. Philippe Grosjean > >> ) ) ) ) ) > >> ( ( ( ( (Numerical Ecology of Aquatic Systems > >> ) ) ) ) ) Mons-Hainaut University, Belgium > >> ( ( ( ( ( > >> .. > >> > >> __ > >> R-devel@r-project.org mailing list > >> https://stat.ethz.ch/mailman/listinfo/r-devel > >> > > > > > -- Byron Ellis ([EMAIL PROTECTED]) "Oook" -- The Librarian __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
[Rd] rbind.data.frame: bug?
Consider the following, which is new behaviour under R 2.5+: > df1 <- data.frame( x=2, y='cat') > df2 <- data.frame( x=3, y='dog') > rbind( df1[-1,], df2)$y == rbind( df1, df2)[-1,]$y Error in Ops.factor(rbind(df1[-1, ], df2)$y, rbind(df1, : Level sets of factors are different To me this seems illogical; it shouldn't matter whether you remove the first row of the data.frame before or after augmenting the rows (and didn't, in R prior to v2.5). And there's more! > levels( rbind( df1[-1,], df2)$y) [1] "dog" > levels( rbind( df1[-1,], df2[-1,])$y) [1] "cat" but why should rbind suddenly acknowledge the levels of its first argument in the second case and not the first? If the data.frames have more than one row, these issue don't arise; they occur because (as the documentation says) "The 'rbind' data frame method first drops all zero-column and zero-row arguments. (If that leaves none, it returns the first argument with columns otherwise a zero-column zero-row data frame.)...". So this behaviour is documented-- but isn't it nevertheless a bug? >From the release notes, I gather that the new zero-row-stripping behaviour was >introduced to get round a specific problem with 0*0 data.frames. Given the >above, though, wouldn't it be preferable to just include special code to deal >with the 0*0 case, leaving 0*N cases unaffected? Mark Bravington __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
[Rd] R graphics device for flash apps
Byron, I just read your blog (statcomp.blogspot.com, linked from your other post) and rand across your idea of an R/Flash graphics device. I've also been giving this some thought because of this amazing interactive flash app: http://tools.google.com/gapminder and others from http://www.gapminder.org/. I would love more than anything to work on this. The only hitch is that I'm so booked up with work (not to mention a new RApache release soon) that I don't have the time. And believe me, getting even minimal functionality to work, like 2D images with pop-ups labels and such... would be a tremendous development effort. Not to mention trying to keep up with an implementation for different OS's and flash player versions. So I have questions for the R development community: 1. Are there others interested in this? Is anyone working on an implementation? If not, is it too niche to even consider doing? 2. Could a project like this obtain funding, any kind, not just academic? 3. How does the community feel about a closed-source version? There are other closed-source packages available for R (the only one I know of is xlsReadWrite), so it seems to be a reasonable question. Comments, criticisms, flames... are all welcome. Email me off-list if you'd like. Jeff -- http://biostat.mc.vanderbilt.edu/JeffreyHorner __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] R graphics device for flash apps
Hi Jeffrey, On 7/6/07, Jeffrey Horner <[EMAIL PROTECTED]> wrote: > Byron, > > I just read your blog (statcomp.blogspot.com, linked from your other > post) and rand across your idea of an R/Flash graphics device. I've also > been giving this some thought because of this amazing interactive flash app: > > http://tools.google.com/gapminder > > and others from http://www.gapminder.org/. Ah, yes. Gotta love Hans Rosling (the guy can swallow swords too!). :-) When I proposed the idea I was thinking more of the Adobe Flex sort of applications where you're pushing more static graphics as SWF through to the Flash side of things. If you poke around Google Video for the Developer's Day stuff, one of the Flex guys had some nice graphs in a Flex application. He was showing off Gears and it was a pretty trivial CRM application, but you can imagine R coming into play pretty easily. > > I would love more than anything to work on this. The only hitch is that > I'm so booked up with work (not to mention a new RApache release soon) > that I don't have the time. And believe me, getting even minimal Yeah, I know the feeling. The RExecServer stuff I've been doing has been an extra-curricular activity just to do some different programming for a bit. Probably when I finally make the commits I've been threatening for a month now into the Bioconductor repository. :-) > functionality to work, like 2D images with pop-ups labels and such... > would be a tremendous development effort. Not to mention trying to keep > up with an implementation for different OS's and flash player versions. I think the best way to start would be implementing the traditional graphics device to push SWF. My thought that a lot of the gadgetry would be done by the UI designer folks and that the R generated stuff would be piped in by a surrounding Flash application. After that would come adding non-traditional elements to the plots, likely via another interface. I think Ming would probably be able to handle it, but compilation is non-trivial making portability difficult. > > So I have questions for the R development community: > > 1. Are there others interested in this? Is anyone working on an > implementation? If not, is it too niche to even consider doing? > > 2. Could a project like this obtain funding, any kind, not just academic? Good question. My guess is that it would be fundamentally tied to #3. (I wouldn't have high hopes for academic funding though...) But, hey, who knows? Only one way to find out I suppose. > > 3. How does the community feel about a closed-source version? > There are other closed-source packages available for R (the only one I > know of is xlsReadWrite), so it seems to be a reasonable question. > > Comments, criticisms, flames... are all welcome. Email me off-list if > you'd like. > > Jeff > -- > http://biostat.mc.vanderbilt.edu/JeffreyHorner > -- Byron Ellis ([EMAIL PROTECTED]) "Oook" -- The Librarian __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel