[Rd] Programming Tools CTV
Hi all, Sorry if this doesn't end up in the thread. Tobias Verbeke forwarded that e-mail to me, because he thought I would be interested in maintaining the Programming Tools CTV. I wasn't subscribed to R-devel yet, but I would indeed like to volunteer to maintain the Programming Tools CTV. It will be my first time creating a CTV, so some guidance on getting it setup will be appreciated. I myself am very interested in better/easier ways to develop faster and nicer code. Kind regards, Willem __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] Programming Tools CTV
Hi Willem thanks for volounteering. To the best of my knowledge (regarding the machinery side), if you're planning to use github (and maybe even if you don't) you can "stole" ideas from https://github.com/ropensci/webservices https://github.com/lbraglia/PackageDevelopmentTaskView (minor modifications from webservices) https://github.com/eddelbuettel/ctv-finance or https://github.com/eddelbuettel/ctv-hpc HTH, Luca 2015-01-23 11:13 GMT+01:00 Willem Ligtenberg : > Hi all, > > Sorry if this doesn't end up in the thread. > Tobias Verbeke forwarded that e-mail to me, because he thought I would be > interested in maintaining the Programming Tools CTV. > I wasn't subscribed to R-devel yet, but I would indeed like to volunteer to > maintain the Programming Tools CTV. > > It will be my first time creating a CTV, so some guidance on getting it setup > will be appreciated. > I myself am very interested in better/easier ways to develop faster and nicer > code. > > Kind regards, > > Willem > > __ > R-devel@r-project.org mailing list > https://stat.ethz.ch/mailman/listinfo/r-devel __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] Programming Tools CTV
... BTW you should install the ctv package and read the vignette at first :) ... 2015-01-23 12:49 GMT+01:00 Luca Braglia : > Hi Willem > > thanks for volounteering. > > To the best of my knowledge (regarding the machinery side), if you're > planning to use github (and maybe even if you don't) you can "stole" > ideas from > > https://github.com/ropensci/webservices > https://github.com/lbraglia/PackageDevelopmentTaskView (minor > modifications from webservices) > https://github.com/eddelbuettel/ctv-finance or > https://github.com/eddelbuettel/ctv-hpc > > > HTH, Luca > > 2015-01-23 11:13 GMT+01:00 Willem Ligtenberg > : >> Hi all, >> >> Sorry if this doesn't end up in the thread. >> Tobias Verbeke forwarded that e-mail to me, because he thought I would be >> interested in maintaining the Programming Tools CTV. >> I wasn't subscribed to R-devel yet, but I would indeed like to volunteer to >> maintain the Programming Tools CTV. >> >> It will be my first time creating a CTV, so some guidance on getting it >> setup will be appreciated. >> I myself am very interested in better/easier ways to develop faster and >> nicer code. >> >> Kind regards, >> >> Willem >> >> __ >> R-devel@r-project.org mailing list >> https://stat.ethz.ch/mailman/listinfo/r-devel __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] Programming Tools CTV
Dear Willem, Personally, I use the R-forge project for the distribution CTV : https://r-forge.r-project.org/projects/ctv/ It’s an alternative option to github. Regards, Christophe --- Christophe Dutang LMM, UdM, Le Mans, France web: http://dutangc.free.fr Le 23 janv. 2015 à 12:49, Luca Braglia a écrit : > Hi Willem > > thanks for volounteering. > > To the best of my knowledge (regarding the machinery side), if you're > planning to use github (and maybe even if you don't) you can "stole" > ideas from > > https://github.com/ropensci/webservices > https://github.com/lbraglia/PackageDevelopmentTaskView (minor > modifications from webservices) > https://github.com/eddelbuettel/ctv-finance or > https://github.com/eddelbuettel/ctv-hpc > > > HTH, Luca > > 2015-01-23 11:13 GMT+01:00 Willem Ligtenberg > : >> Hi all, >> >> Sorry if this doesn't end up in the thread. >> Tobias Verbeke forwarded that e-mail to me, because he thought I would be >> interested in maintaining the Programming Tools CTV. >> I wasn't subscribed to R-devel yet, but I would indeed like to volunteer to >> maintain the Programming Tools CTV. >> >> It will be my first time creating a CTV, so some guidance on getting it >> setup will be appreciated. >> I myself am very interested in better/easier ways to develop faster and >> nicer code. >> >> Kind regards, >> >> Willem >> >> __ >> R-devel@r-project.org mailing list >> https://stat.ethz.ch/mailman/listinfo/r-devel > > __ > R-devel@r-project.org mailing list > https://stat.ethz.ch/mailman/listinfo/r-devel __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] Programming Tools CTV
Dear Willem I maintain the MetaAnalysis CTV. I have found it quite practicable to do this without special tools. I use an editor for the XML. I use CRANberries to catch updates and I usually email people to check I have understood a new package. People also kindly email me occasionally with news about major changes. On the other hand since it is the programming tools CTV I imagine you will want to try out the latest all-singing, all-dancing technology. And why not. Michael On 23/01/2015 13:05, Christophe Dutang wrote: Dear Willem, Personally, I use the R-forge project for the distribution CTV : https://r-forge.r-project.org/projects/ctv/ It’s an alternative option to github. Regards, Christophe --- Christophe Dutang LMM, UdM, Le Mans, France web: http://dutangc.free.fr Le 23 janv. 2015 à 12:49, Luca Braglia a écrit : Hi Willem thanks for volounteering. To the best of my knowledge (regarding the machinery side), if you're planning to use github (and maybe even if you don't) you can "stole" ideas from https://github.com/ropensci/webservices https://github.com/lbraglia/PackageDevelopmentTaskView (minor modifications from webservices) https://github.com/eddelbuettel/ctv-finance or https://github.com/eddelbuettel/ctv-hpc HTH, Luca 2015-01-23 11:13 GMT+01:00 Willem Ligtenberg : Hi all, Sorry if this doesn't end up in the thread. Tobias Verbeke forwarded that e-mail to me, because he thought I would be interested in maintaining the Programming Tools CTV. I wasn't subscribed to R-devel yet, but I would indeed like to volunteer to maintain the Programming Tools CTV. It will be my first time creating a CTV, so some guidance on getting it setup will be appreciated. I myself am very interested in better/easier ways to develop faster and nicer code. Kind regards, Willem __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel - No virus found in this message. Checked by AVG - www.avg.com Version: 2015.0.5645 / Virus Database: 4260/8981 - Release Date: 01/23/15 -- Michael http://www.dewey.myzen.co.uk __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] Programming Tools CTV
I'd strongly second the notion of using github. The biggest advantage is that others can easily contribute changes through pull requests which lifts much of the burden from your shoulders. Hadley On Fri, Jan 23, 2015 at 3:49 AM, Luca Braglia wrote: > Hi Willem > > thanks for volounteering. > > To the best of my knowledge (regarding the machinery side), if you're > planning to use github (and maybe even if you don't) you can "stole" > ideas from > > https://github.com/ropensci/webservices > https://github.com/lbraglia/PackageDevelopmentTaskView (minor > modifications from webservices) > https://github.com/eddelbuettel/ctv-finance or > https://github.com/eddelbuettel/ctv-hpc > > > HTH, Luca > > 2015-01-23 11:13 GMT+01:00 Willem Ligtenberg > : >> Hi all, >> >> Sorry if this doesn't end up in the thread. >> Tobias Verbeke forwarded that e-mail to me, because he thought I would be >> interested in maintaining the Programming Tools CTV. >> I wasn't subscribed to R-devel yet, but I would indeed like to volunteer to >> maintain the Programming Tools CTV. >> >> It will be my first time creating a CTV, so some guidance on getting it >> setup will be appreciated. >> I myself am very interested in better/easier ways to develop faster and >> nicer code. >> >> Kind regards, >> >> Willem >> >> __ >> R-devel@r-project.org mailing list >> https://stat.ethz.ch/mailman/listinfo/r-devel > > __ > R-devel@r-project.org mailing list > https://stat.ethz.ch/mailman/listinfo/r-devel -- http://had.co.nz/ __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] Programming Tools CTV
Willem,
thanks for volunteering!
Sorry if this doesn't end up in the thread. Tobias Verbeke forwarded
that e-mail to me, because he thought I would be interested in
maintaining the Programming Tools CTV. I wasn't subscribed to R-devel
yet, but I would indeed like to volunteer to maintain the Programming
Tools CTV.
As discussed in the other thread: A programming tools task view would be
too broad and Max suggested splitting it up into sharper and more
manageable portions. Maybe you want to pick up one of these sub-topics?
It will be my first time creating a CTV, so some guidance on getting it
setup will be appreciated. I myself am very interested in better/easier
ways to develop faster and nicer code.
The process is usually the following:
- Someone proposes to set up a certain task view - often here in the list
or in a direct e-mail to me.
- If the topic is deemed feasible for a task view, then the
maintainer-to-be compiles a .ctv file following the advice in
vignette("ctv", package = "ctv") and first sends it to me.
- If the maintainer-to-be and myself are satisfied with the result so far,
we typically try to get some more feedback from the mailing list and then
release it to CRAN.
As for version control:
The "ctv" package and all task view reside on R-Forge. All maintainers are
invited to join the R-Forge project and thus get access to the SVN
repository so that they can change their .ctv file directly. Most
maintainers do so but a few maintainers have chose to either use no
version control themselves or use some separate system (e.g., Github). In
the latter cases, updates are sent by e-mail to me.
Best,
Z
Kind regards,
Willem
__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] :: and ::: as .Primitives?
On Thu, 22 Jan 2015, Henrik Bengtsson wrote: On Thu, Jan 22, 2015 at 11:44 AM, wrote: I'm not convinced that how to make :: faster is the right question. If you are finding foo::bar being called often enough to matter to your overall performance then to me the question is: why are you calling foo::bar more than once? Making :: a bit faster by making it a primitive will remove some overhead, but your are still left with a lot of work that shouldn't need to happen more than once. For default methods there ought to be a way to create those so the default method is computed at creation or load time and stored in an environment. For other cases if I want to use foo::bar many times, say in a loop, I would do foo_bar <- foo::bar and use foo_bar, or something along those lines. While you're on the line: Do you think this is an optimization that the 'compiler' package and it's cmpfun() byte compiler will be able to do in the future? Most likely, at least at reasonable optimization levels. Best, luke /Henrik When :: and ::: were introduce they were intended primarily for reflection and debugging, so speed was not an issue. ::: is still really only reliably usable that way, and making it faster may just encourage bad practice. :: is different and there are good arguments for using it in code, but I'm not yet seeing good arguments for use in ways that would be performance-critical, but I'm happy to be convinced otherwise. If there is a need for a faster :: then going to a SPECIALSXP is fine; it would also be good to make the byte code compiler aware of it, and possibly to work on ways to improve the performance further e.g. through cacheing. Best, luke On Thu, 22 Jan 2015, Peter Haverty wrote: Hi all, When S4 methods are defined on base function (say, "match"), the function becomes a method with the body "base::match(x,y)". A call to such a function often spends more time doing "::" than in the function itself. I always assumed that "::" was a very low-level thing, but it turns out to be a plain old function defined in base/R/namespace.R. What would you all think about making "::" and ":::" .Primitives? I have submitted some examples, timings, and a patch to the R bug tracker (https://bugs.r-project.org/bugzilla3/show_bug.cgi?id=16134). I'd be very interested to hear your thoughts on the matter. Regards, Pete Peter M. Haverty, Ph.D. Genentech, Inc. phave...@gene.com __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel -- Luke Tierney Ralph E. Wareham Professor of Mathematical Sciences University of Iowa Phone: 319-335-3386 Department of Statistics andFax: 319-335-3017 Actuarial Science 241 Schaeffer Hall email: luke-tier...@uiowa.edu Iowa City, IA 52242 WWW: http://www.stat.uiowa.edu __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel -- Luke Tierney Ralph E. Wareham Professor of Mathematical Sciences University of Iowa Phone: 319-335-3386 Department of Statistics andFax: 319-335-3017 Actuarial Science 241 Schaeffer Hall email: luke-tier...@uiowa.edu Iowa City, IA 52242 WWW: http://www.stat.uiowa.edu __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] :: and ::: as .Primitives?
On Thu, 22 Jan 2015, Michael Lawrence wrote: On Thu, Jan 22, 2015 at 11:44 AM, wrote: For default methods there ought to be a way to create those so the default method is computed at creation or load time and stored in an environment. We had considered that, but we thought the definition of the function would be easier to interpret if it explicitly specified the namespace, instead of using tricks with environments. The same applies for memoizing the lookup in front of a loop. interpret in what sense (human reader or R interpreter)? In either case I'm not convinced. The implementation of these functions is almost simpler in C than it is in R, so there is relatively little risk to this change. But I agree the benefits are also somewhat minor. I don't disagree, but it remains that even calling the C version has costs that should not need to be paid. But maybe we can leave that to the compiler/byte code engine. Optimizing references to symbols resolved statically to name spaces and imports is on the to do list, and with a little care that mechanism should work for foo::bar uses as well. Best, luke For other cases if I want to use foo::bar many times, say in a loop, I would do foo_bar <- foo::bar and use foo_bar, or something along those lines. When :: and ::: were introduce they were intended primarily for reflection and debugging, so speed was not an issue. ::: is still really only reliably usable that way, and making it faster may just encourage bad practice. :: is different and there are good arguments for using it in code, but I'm not yet seeing good arguments for use in ways that would be performance-critical, but I'm happy to be convinced otherwise. If there is a need for a faster :: then going to a SPECIALSXP is fine; it would also be good to make the byte code compiler aware of it, and possibly to work on ways to improve the performance further e.g. through cacheing. Best, luke On Thu, 22 Jan 2015, Peter Haverty wrote: Hi all, When S4 methods are defined on base function (say, "match"), the function becomes a method with the body "base::match(x,y)". A call to such a function often spends more time doing "::" than in the function itself. I always assumed that "::" was a very low-level thing, but it turns out to be a plain old function defined in base/R/namespace.R. What would you all think about making "::" and ":::" .Primitives? I have submitted some examples, timings, and a patch to the R bug tracker (https://bugs.r-project.org/bugzilla3/show_bug.cgi?id=16134). I'd be very interested to hear your thoughts on the matter. Regards, Pete Peter M. Haverty, Ph.D. Genentech, Inc. phave...@gene.com __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel -- Luke Tierney Ralph E. Wareham Professor of Mathematical Sciences University of Iowa Phone: 319-335-3386 Department of Statistics andFax: 319-335-3017 Actuarial Science 241 Schaeffer Hall email: luke-tier...@uiowa.edu Iowa City, IA 52242 WWW: http://www.stat.uiowa.edu __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel -- Luke Tierney Ralph E. Wareham Professor of Mathematical Sciences University of Iowa Phone: 319-335-3386 Department of Statistics andFax: 319-335-3017 Actuarial Science 241 Schaeffer Hall email: luke-tier...@uiowa.edu Iowa City, IA 52242 WWW: http://www.stat.uiowa.edu __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] speedbump in library
I think you can simplify a little by replacing this:
pkg %in% loadedNamespaces()
with this:
.getNamespace(pkg)
Whereas getNamespace(pkg) will load the package if it's not already
loaded, calling .getNamespace(pkg) (note the leading dot) won't load
the package.
I can't speak to whether there are any pitfalls in changing the
library path searching, though.
-Winston
On Thu, Jan 22, 2015 at 12:25 PM, Peter Haverty wrote:
> Hi all,
>
> Profiling turned up a bit of a speedbump in the library function. I
> submitted a patch to the R bug tracker as bug 16168 and I've also
> included it below. The alternate code is simpler and easier to
> read/maintain, I believe. Any thoughts on other ways to write this?
>
> Index: src/library/base/R/library.R
> ===
> --- src/library/base/R/library.R(revision 67578)
> +++ src/library/base/R/library.R(working copy)
> @@ -688,18 +688,8 @@
> out <- character()
>
> for(pkg in package) {
> -paths <- character()
> -for(lib in lib.loc) {
> -dirs <- list.files(lib,
> - pattern = paste0("^", pkg, "$"),
> - full.names = TRUE)
> -## Note that we cannot use tools::file_test() here, as
> -## cyclic namespace dependencies are not supported. Argh.
> -paths <- c(paths,
> - dirs[dir.exists(dirs) &
> -file.exists(file.path(dirs,
> - "DESCRIPTION"))])
> -}
> +paths <- file.path(lib.loc, pkg)
> +paths <- paths[ file.exists(file.path(paths, "DESCRIPTION")) ]
> if(use_loaded && pkg %in% loadedNamespaces()) {
> dir <- if (pkg == "base") system.file()
> else getNamespaceInfo(pkg, "path")
>
> Pete
>
>
> Peter M. Haverty, Ph.D.
> Genentech, Inc.
> phave...@gene.com
>
> __
> R-devel@r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-devel
__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] speedbump in library
Thanks Winston,
Yes, your version of that part is more direct. I guess it would need a
! is.null() too. I think we should use .getNamespace.
It It also occurred to me that this %in% check (which happens in a few
places) is kind of roundabout. It equates to
"foo" %in% ls(.Internal(getNamespaceRegistry()), all.names = TRUE)
We lack and R-level accessor for the namespace registry, but if we had
one we could do
getNamespaceRegistry()[["foo"]]
, which is just a hash lookup.
I'm getting a bit off topic here, but ...
"foo" %in% vector is a common pattern and reads well, but
any(vector == "foo")
is less work and much faster. I wonder if there is room for a fast
path there ...
Pete
Peter M. Haverty, Ph.D.
Genentech, Inc.
phave...@gene.com
On Fri, Jan 23, 2015 at 8:15 AM, Winston Chang wrote:
> I think you can simplify a little by replacing this:
> pkg %in% loadedNamespaces()
> with this:
> .getNamespace(pkg)
>
> Whereas getNamespace(pkg) will load the package if it's not already
> loaded, calling .getNamespace(pkg) (note the leading dot) won't load
> the package.
>
> I can't speak to whether there are any pitfalls in changing the
> library path searching, though.
>
> -Winston
>
>
> On Thu, Jan 22, 2015 at 12:25 PM, Peter Haverty
> wrote:
>> Hi all,
>>
>> Profiling turned up a bit of a speedbump in the library function. I
>> submitted a patch to the R bug tracker as bug 16168 and I've also
>> included it below. The alternate code is simpler and easier to
>> read/maintain, I believe. Any thoughts on other ways to write this?
>>
>> Index: src/library/base/R/library.R
>> ===
>> --- src/library/base/R/library.R(revision 67578)
>> +++ src/library/base/R/library.R(working copy)
>> @@ -688,18 +688,8 @@
>> out <- character()
>>
>> for(pkg in package) {
>> -paths <- character()
>> -for(lib in lib.loc) {
>> -dirs <- list.files(lib,
>> - pattern = paste0("^", pkg, "$"),
>> - full.names = TRUE)
>> -## Note that we cannot use tools::file_test() here, as
>> -## cyclic namespace dependencies are not supported. Argh.
>> -paths <- c(paths,
>> - dirs[dir.exists(dirs) &
>> -file.exists(file.path(dirs,
>> - "DESCRIPTION"))])
>> -}
>> +paths <- file.path(lib.loc, pkg)
>> +paths <- paths[ file.exists(file.path(paths, "DESCRIPTION")) ]
>> if(use_loaded && pkg %in% loadedNamespaces()) {
>> dir <- if (pkg == "base") system.file()
>> else getNamespaceInfo(pkg, "path")
>>
>> Pete
>>
>>
>> Peter M. Haverty, Ph.D.
>> Genentech, Inc.
>> phave...@gene.com
>>
>> __
>> R-devel@r-project.org mailing list
>> https://stat.ethz.ch/mailman/listinfo/r-devel
__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] :: and ::: as .Primitives?
I tend to use this (in my own internal code *only*):
exported <- function (pkg) {
if (pkg == "base") {
function (fun) {
fun <- as.character(substitute(fun))
res <- .BaseNamespaceEnv[[fun]]
if (is.null(res))
stop(fun, " is not found in package base")
res
}
} else {
ns <- getNamespace(pkg)
exports <- getNamespaceInfo(ns, "exports")
function (obj) {
obj <- as.character(substitute(obj))
exportedObj <- exports[[obj]]
if (is.null(exportedObj)) {
if (is.null(ns[[obj]])) {
stop(obj, " does not exists in package
", pkg)
} else {
stop(obj, " is not exported from
package ", pkg)
}
}
ns[[exportedObj]]
}
}
}
stats <- exported("stats")
stats(acf)
stats("[.acf")
stats("inexistant")
exported("base")(ls)
exported("base")(inexistant)
## Performance tests for what it’s worth
microbenchmark::microbenchmark(stats::acf, (stats <- exported("stats"))(acf),
stats(acf))
microbenchmark::microbenchmark(base::ls, (base <- exported("base"))(ls),
base(ls), .BaseNamespaceEnv$ls)
So, `::` is slow and I can get better speed results thanks to binding both the
namespace and the exports environments in the `stats` closure. Unless I miss
something, this is not much a problem for base package that is never unloaded.
Yet, .BaseNamespaceEnv$xxx, or baseenv()$xxx does the job faster and simpler.
However, there is a vicious problem with my exported() function, which is, to
say the least, dangerous under the hand of unaware users. Indeed:
stats <- exported(“stats”)
creates a new binding to both the namespace and the exports environments of the
stats package. So, if I do:
detach(“package:stats”, unload = TRUE), then library(“stats”), I got two
versions of the package in memory, and my `stats`closure refers to an outdated
version of the package. This is particularly problematic if the package was
recompiled in between (in the context of debugging).
Conclusion: much of the lost of performance in `::` is due to not caching the
environments. This is fully justified to keep the dynamism of the language at
full power and to avoid a messy state of R as described here above… Regarding
dynamism, even `stats::acf`remains discutable.
Moreover, it is possible to do many other crazy things with these environments,
once one got a grip on them. So, even getNamespace() and getNamespaceInfo() are
dangerous. Perhaps this should be emphasised in the ?getNamespace man page?
This is also why the code above is not released in the wild… Well, now it is :-(
Best,
Philippe
..<°}))><
) ) ) ) )
( ( ( ( (Prof. Philippe Grosjean
) ) ) ) )
( ( ( ( (Numerical Ecology of Aquatic Systems
) ) ) ) ) Mons University, Belgium
( ( ( ( (
..
> On 23 Jan 2015, at 16:01, luke-tier...@uiowa.edu wrote:
>
> On Thu, 22 Jan 2015, Michael Lawrence wrote:
>
>> On Thu, Jan 22, 2015 at 11:44 AM, wrote:
>>>
>>> For default methods there ought to be a way to create those so the
>>> default method is computed at creation or load time and stored in an
>>> environment.
>>
>> We had considered that, but we thought the definition of the function
>> would be easier to interpret if it explicitly specified the namespace,
>> instead of using tricks with environments. The same applies for
>> memoizing the lookup in front of a loop.
>
> interpret in what sense (human reader or R interpreter)? In either
> case I'm not convinced.
>
>> The implementation of these functions is almost simpler in C than it
>> is in R, so there is relatively little risk to this change. But I
>> agree the benefits are also somewhat minor.
>
> I don't disagree, but it remains that even calling the C version has
> costs that should not need to be paid. But maybe we can leave that to
> the compiler/byte code engine. Optimizing references to symbols
> resolved statically to name spaces and imports is on the to do list,
> and with a little care that mechanism should work for foo::bar uses as
> well.
>
> Best,
>
> luke
>
>>
>>> For other cases if I want to use foo::bar many times, say
>>> in a loop, I would do
>>>
>>> foo_bar <- foo::bar
>>>
>>> and use foo_bar, or something along those lines.
>>>
>>> When :: and ::: were introduce they were intended primarily for
>>> reflection and debugging, so speed was not an issue. ::: is still
>>> really only reliably usable that way, and making it faster may just
>>> encourage bad practice. :: is d
Re: [Rd] Programming Tools CTV
On 23 January 2015 at 05:55, Hadley Wickham wrote: | I'd strongly second the notion of using github. The biggest advantage | is that others can easily contribute changes through pull requests | which lifts much of the burden from your shoulders. That's "The Theory". "The Practice" for eight weeks having the High-Performance Computing and Finance Task Views there (in "source" and a rendered markdown, see [1]) is a single pull request fixing a one-char typo. So the jury may still be out on that one. Dirk [1] As already shown in the email thread: https://github.com/eddelbuettel/ctv-hpc https://github.com/eddelbuettel/ctv-finance -- http://dirk.eddelbuettel.com | @eddelbuettel | e...@debian.org __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] :: and ::: as .Primitives?
Hi,
On 01/23/2015 07:01 AM, luke-tier...@uiowa.edu wrote:
On Thu, 22 Jan 2015, Michael Lawrence wrote:
On Thu, Jan 22, 2015 at 11:44 AM, wrote:
For default methods there ought to be a way to create those so the
default method is computed at creation or load time and stored in an
environment.
We had considered that, but we thought the definition of the function
would be easier to interpret if it explicitly specified the namespace,
instead of using tricks with environments. The same applies for
memoizing the lookup in front of a loop.
interpret in what sense (human reader or R interpreter)? In either
case I'm not convinced.
From a developer perspective, especially when debugging, when we do
selectMethod("match", ...) and it turns out that this returns the
default method, it's good to see:
Method Definition (Class "derivedDefaultMethod"):
function (x, table, nomatch = NA_integer_, incomparables = NULL,
...)
base::match(x, table, nomatch = nomatch, incomparables = incomparables,
...)
Signatures:
x table
target "DataFrame" "ANY"
defined "ANY" "ANY"
rather than some obscure/uninformative body. I hope we can keep that.
The implementation of these functions is almost simpler in C than it
is in R, so there is relatively little risk to this change. But I
agree the benefits are also somewhat minor.
I don't disagree, but it remains that even calling the C version has
costs that should not need to be paid. But maybe we can leave that to
the compiler/byte code engine. Optimizing references to symbols
resolved statically to name spaces and imports is on the to do list,
and with a little care that mechanism should work for foo::bar uses as
well.
That would be great. Thanks!
H.
Best,
luke
For other cases if I want to use foo::bar many times, say
in a loop, I would do
foo_bar <- foo::bar
and use foo_bar, or something along those lines.
When :: and ::: were introduce they were intended primarily for
reflection and debugging, so speed was not an issue. ::: is still
really only reliably usable that way, and making it faster may just
encourage bad practice. :: is different and there are good arguments
for using it in code, but I'm not yet seeing good arguments for use in
ways that would be performance-critical, but I'm happy to be convinced
otherwise. If there is a need for a faster :: then going to a
SPECIALSXP is fine; it would also be good to make the byte code
compiler aware of it, and possibly to work on ways to improve the
performance further e.g. through cacheing.
Best,
luke
On Thu, 22 Jan 2015, Peter Haverty wrote:
Hi all,
When S4 methods are defined on base function (say, "match"), the
function becomes a method with the body "base::match(x,y)". A call to
such a function often spends more time doing "::" than in the function
itself. I always assumed that "::" was a very low-level thing, but it
turns out to be a plain old function defined in base/R/namespace.R.
What would you all think about making "::" and ":::" .Primitives? I
have submitted some examples, timings, and a patch to the R bug
tracker (https://bugs.r-project.org/bugzilla3/show_bug.cgi?id=16134).
I'd be very interested to hear your thoughts on the matter.
Regards,
Pete
Peter M. Haverty, Ph.D.
Genentech, Inc.
phave...@gene.com
__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
--
Luke Tierney
Ralph E. Wareham Professor of Mathematical Sciences
University of Iowa Phone: 319-335-3386
Department of Statistics andFax: 319-335-3017
Actuarial Science
241 Schaeffer Hall email: luke-tier...@uiowa.edu
Iowa City, IA 52242 WWW: http://www.stat.uiowa.edu
__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
--
Hervé Pagès
Program in Computational Biology
Division of Public Health Sciences
Fred Hutchinson Cancer Research Center
1100 Fairview Ave. N, M1-B514
P.O. Box 19024
Seattle, WA 98109-1024
E-mail: hpa...@fredhutch.org
Phone: (206) 667-5791
Fax:(206) 667-1319
__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
[Rd] issue with update.packages()
Hello,
I see the following issue in R-devel since 'both' has become the default
pkgType for binary platforms.
update.packages() fails when you set options(repos). Looks like it is trying to
download a tgz file from the src/contrib section of a repository (on a mac).
To reproduce this you need to have an older version of AnnotationDbi installed,
which I accomplished by faking it, installing pkgKitten and then doing:
library(pkgKitten)
kitten("AnnotationDbi")
Then exiting R and doing
R CMD INSTALL AnnotationDbi.
Here's the problem:
> packageVersion("AnnotationDbi")
[1] ‘1.0’
> options(repos=structure(c("http://bioconductor.org/packages/3.1/bioc";,
> "http://bioconductor.org/packages/3.1/data/annotation";,
+ "http://bioconductor.org/packages/3.1/data/experiment";,
"http://bioconductor.org/packages/3.1/extra";,
+ "http://cran.fhcrc.org";), .Names = c("BioCsoft", "BioCann", "BioCexp",
+ "BioCextra", "CRAN")))
> update.packages(oldPkgs="AnnotationDbi")
AnnotationDbi :
Version 1.0 installed in
/Library/Frameworks/R.framework.develMav/Versions/3.2/Resources/library
Version 1.29.17 available at http://bioconductor.org/packages/3.1/bioc
Update (y/N/c)? y
also installing the dependencies ‘IRanges’, ‘BiocGenerics’, ‘Biobase’,
‘GenomeInfoDb’, ‘DBI’, ‘RSQLite’, ‘S4Vectors’
trying URL
'http://bioconductor.org/packages/3.1/bioc/src/contrib/IRanges_2.1.35.tgz'
Error in download.file(url, destfile, method, mode = "wb", ...) :
cannot open URL
'http://bioconductor.org/packages/3.1/bioc/src/contrib/IRanges_2.1.35.tgz'
In addition: Warning message:
In download.file(url, destfile, method, mode = "wb", ...) :
cannot open: HTTP status was '404 Not Found'
Warning in download.packages(pkgs, destdir = tmpd, available = available, :
download of package ‘IRanges’ failed
trying URL
'http://bioconductor.org/packages/3.1/bioc/src/contrib/BiocGenerics_0.13.4.tgz'
Error in download.file(url, destfile, method, mode = "wb", ...) :
cannot open URL
'http://bioconductor.org/packages/3.1/bioc/src/contrib/BiocGenerics_0.13.4.tgz'
In addition: Warning message:
In download.file(url, destfile, method, mode = "wb", ...) :
cannot open: HTTP status was '404 Not Found'
Warning in download.packages(pkgs, destdir = tmpd, available = available, :
download of package ‘BiocGenerics’ failed
trying URL
'http://bioconductor.org/packages/3.1/bioc/src/contrib/Biobase_2.27.1.tgz'
Error in download.file(url, destfile, method, mode = "wb", ...) :
cannot open URL
'http://bioconductor.org/packages/3.1/bioc/src/contrib/Biobase_2.27.1.tgz'
In addition: Warning message:
In download.file(url, destfile, method, mode = "wb", ...) :
cannot open: HTTP status was '404 Not Found'
Warning in download.packages(pkgs, destdir = tmpd, available = available, :
download of package ‘Biobase’ failed
trying URL
'http://bioconductor.org/packages/3.1/bioc/src/contrib/GenomeInfoDb_1.3.12.tgz'
Error in download.file(url, destfile, method, mode = "wb", ...) :
cannot open URL
'http://bioconductor.org/packages/3.1/bioc/src/contrib/GenomeInfoDb_1.3.12.tgz'
In addition: Warning message:
In download.file(url, destfile, method, mode = "wb", ...) :
cannot open: HTTP status was '404 Not Found'
Warning in download.packages(pkgs, destdir = tmpd, available = available, :
download of package ‘GenomeInfoDb’ failed
trying URL 'http://cran.fhcrc.org/src/contrib/DBI_0.3.1.tgz'
Error in download.file(url, destfile, method, mode = "wb", ...) :
cannot open URL 'http://cran.fhcrc.org/src/contrib/DBI_0.3.1.tgz'
In addition: Warning message:
In download.file(url, destfile, method, mode = "wb", ...) :
cannot open: HTTP status was '404 Not Found'
Warning in download.packages(pkgs, destdir = tmpd, available = available, :
download of package ‘DBI’ failed
trying URL 'http://cran.fhcrc.org/src/contrib/RSQLite_1.0.0.tgz'
Error in download.file(url, destfile, method, mode = "wb", ...) :
cannot open URL 'http://cran.fhcrc.org/src/contrib/RSQLite_1.0.0.tgz'
In addition: Warning message:
In download.file(url, destfile, method, mode = "wb", ...) :
cannot open: HTTP status was '404 Not Found'
Warning in download.packages(pkgs, destdir = tmpd, available = available, :
download of package ‘RSQLite’ failed
trying URL
'http://bioconductor.org/packages/3.1/bioc/src/contrib/S4Vectors_0.5.16.tgz'
Error in download.file(url, destfile, method, mode = "wb", ...) :
cannot open URL
'http://bioconductor.org/packages/3.1/bioc/src/contrib/S4Vectors_0.5.16.tgz'
In addition: Warning message:
In download.file(url, destfile, method, mode = "wb", ...) :
cannot open: HTTP status was '404 Not Found'
Warning in download.packages(pkgs, destdir = tmpd, available = available, :
download of package ‘S4Vectors’ failed
trying URL
'http://bioconductor.org/packages/3.1/bioc/src/contrib/AnnotationDbi_1.29.17.tgz'
Error in download.file(url, destfile, method, mode = "wb", ...) :
cannot open URL
'http://bioconductor.org/packages/3.1/bioc/src/contrib/AnnotationDbi_1.29
Re: [Rd] :: and ::: as .Primitives?
On 22/01/2015 4:06 PM, Peter Haverty wrote:
> Hi all,
>
> I use Luke's "::" hoisting trick often. I think it would be fantastic
> if the JIT just did that for you.
>
> The main trouble, for me, is in code I don't own. When common
> Bioconductor packages are loaded many, many base functions are saddled
> with this substantial dispatch and "::" overhead.
>
> While we have the hood up, the parser could help out a bit here too.
> It already has special cases for "::" and ":::". Currently you get the
> symbols "pkg" and "name" and have to go fishing in the calling
> environment for the associated values.
I don't think the parser should do this, but it does seem like a
reasonable optimization for the compiler to do.
It would be nice to have the
> parser or JIT rewrite base::match as doubleColon("base","match") or
> directly provide the symbols "base" and "match" to the subsequent
> code.
Currently the parser provides the expression `::`(base, match), and the
`::` function converts those symbols to character strings "base" and
"match". While the parser could have saved it some work by giving the
expression `::`("base", "match"), I think it's a bad idea to start
messing with things that way. After all, a user could have defined
their own `::` function, and they should get what they typed.
Duncan Murdoch
__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] :: and ::: as .Primitives?
On Fri, Jan 23, 2015 at 10:11 AM, Hervé Pagès wrote:
> Hi,
>
> On 01/23/2015 07:01 AM, luke-tier...@uiowa.edu wrote:
>>
>> On Thu, 22 Jan 2015, Michael Lawrence wrote:
>>
>>> On Thu, Jan 22, 2015 at 11:44 AM, wrote:
For default methods there ought to be a way to create those so the
default method is computed at creation or load time and stored in an
environment.
>>>
>>>
>>> We had considered that, but we thought the definition of the function
>>> would be easier to interpret if it explicitly specified the namespace,
>>> instead of using tricks with environments. The same applies for
>>> memoizing the lookup in front of a loop.
>>
>>
>> interpret in what sense (human reader or R interpreter)? In either
>> case I'm not convinced.
>
>
> From a developer perspective, especially when debugging, when we do
> selectMethod("match", ...) and it turns out that this returns the
> default method, it's good to see:
>
> Method Definition (Class "derivedDefaultMethod"):
>
> function (x, table, nomatch = NA_integer_, incomparables = NULL,
> ...)
> base::match(x, table, nomatch = nomatch, incomparables = incomparables,
> ...)
>
>
> Signatures:
> x table
> target "DataFrame" "ANY"
> defined "ANY" "ANY"
>
> rather than some obscure/uninformative body. I hope we can keep that.
That was the goal of this patch. We want to keep that, and make
match() ~25% faster when falling back to the default method (for small
inputs). Right now, loading BiocGenerics, IRanges, etc, slows many
functions down by roughly that amount.
>
>>
>>> The implementation of these functions is almost simpler in C than it
>>> is in R, so there is relatively little risk to this change. But I
>>> agree the benefits are also somewhat minor.
>>
>>
>> I don't disagree, but it remains that even calling the C version has
>> costs that should not need to be paid. But maybe we can leave that to
>> the compiler/byte code engine. Optimizing references to symbols
>> resolved statically to name spaces and imports is on the to do list,
>> and with a little care that mechanism should work for foo::bar uses as
>> well.
>
>
> That would be great. Thanks!
>
>
> H.
>
>>
>> Best,
>>
>> luke
>>
>>>
For other cases if I want to use foo::bar many times, say
in a loop, I would do
foo_bar <- foo::bar
and use foo_bar, or something along those lines.
When :: and ::: were introduce they were intended primarily for
reflection and debugging, so speed was not an issue. ::: is still
really only reliably usable that way, and making it faster may just
encourage bad practice. :: is different and there are good arguments
for using it in code, but I'm not yet seeing good arguments for use in
ways that would be performance-critical, but I'm happy to be convinced
otherwise. If there is a need for a faster :: then going to a
SPECIALSXP is fine; it would also be good to make the byte code
compiler aware of it, and possibly to work on ways to improve the
performance further e.g. through cacheing.
Best,
luke
On Thu, 22 Jan 2015, Peter Haverty wrote:
> Hi all,
>
> When S4 methods are defined on base function (say, "match"), the
> function becomes a method with the body "base::match(x,y)". A call to
> such a function often spends more time doing "::" than in the function
> itself. I always assumed that "::" was a very low-level thing, but it
> turns out to be a plain old function defined in base/R/namespace.R.
> What would you all think about making "::" and ":::" .Primitives? I
> have submitted some examples, timings, and a patch to the R bug
> tracker (https://bugs.r-project.org/bugzilla3/show_bug.cgi?id=16134).
> I'd be very interested to hear your thoughts on the matter.
>
> Regards,
> Pete
>
>
> Peter M. Haverty, Ph.D.
> Genentech, Inc.
> phave...@gene.com
>
> __
> R-devel@r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-devel
>
--
Luke Tierney
Ralph E. Wareham Professor of Mathematical Sciences
University of Iowa Phone: 319-335-3386
Department of Statistics andFax: 319-335-3017
Actuarial Science
241 Schaeffer Hall email: luke-tier...@uiowa.edu
Iowa City, IA 52242 WWW: http://www.stat.uiowa.edu
__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
>>>
>>>
>>
>
> --
> Hervé Pagès
>
> Program in Computational Biology
> Division of Public Health Sciences
> Fred Hutchinson Cancer Research Center
> 1100 Fairview Ave. N, M1-B514
> P.O. Box 19024
> Seattle, WA 98109-1024
>
> E-mail: hp
