[Rd] R package dependency issues when namespace is not attached
I have always assumed that having a package in the 'Depends' field would automatically also import the namespace. However, it seems that in R 2.15, dependencies do not become available until the package is actually attached to the searchpath. Is this intended behavior? The problem appears as follows: Suppose there is a package 'Child' which Depends, but does not explicitly import a package called 'Parent' and contains a function that calls out to an object in the namespace of 'Parent'. When this function is called without attaching 'Child' to the search path, the function in 'Parent' cannot be found. Here an example from the manual of the bigdata package, but the problem is very widespread: x = matrix(rnorm(50*80),50,80) beta = c(3,2,1.5,rep(0,77)) y = rnorm(50) + x%*%beta z1 = bigdata::lasso.stars(x,y) The example fails because lasso.stars depends on 'glmnet' which is not loaded until bigdata is attached. The only way to be able to call lasso.stars is to actually attach the bigdata package: library(bigdata) z1 = bigdata::lasso.stars(x,y) Now to further complicate things, it seems that this problem is inherited to any 'grandchild' package that imports, in this case, the lasso.stars function. I have a hard time finding a good example but I am sure they are out there. Is this a bug? I know that it can be avoided by asking package authors to use Imports instead of Depends, but in practice the majority of the packages on CRAN still use Depends. It seems like the problem is easily fixed if R would automatically import the namespace of any Depends packages into to the child package namespace? __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] R package dependency issues when namespace is not attached
On 12-05-13 3:15 AM, Jeroen Ooms wrote: I have always assumed that having a package in the 'Depends' field would automatically also import the namespace. However, it seems that in R 2.15, dependencies do not become available until the package is actually attached to the searchpath. Is this intended behavior? The problem appears as follows: Suppose there is a package 'Child' which Depends, but does not explicitly import a package called 'Parent' and contains a function that calls out to an object in the namespace of 'Parent'. When this function is called without attaching 'Child' to the search path, the function in 'Parent' cannot be found. Here an example from the manual of the bigdata package, but the problem is very widespread: x = matrix(rnorm(50*80),50,80) beta = c(3,2,1.5,rep(0,77)) y = rnorm(50) + x%*%beta z1 = bigdata::lasso.stars(x,y) The example fails because lasso.stars depends on 'glmnet' which is not loaded until bigdata is attached. The only way to be able to call lasso.stars is to actually attach the bigdata package: library(bigdata) z1 = bigdata::lasso.stars(x,y) Now to further complicate things, it seems that this problem is inherited to any 'grandchild' package that imports, in this case, the lasso.stars function. I have a hard time finding a good example but I am sure they are out there. Is this a bug? I know that it can be avoided by asking package authors to use Imports instead of Depends, but in practice the majority of the packages on CRAN still use Depends. It seems like the problem is easily fixed if R would automatically import the namespace of any Depends packages into to the child package namespace? Not sure if it's a bug, but the correct solution in bigdata is to import the glmnet function in its NAMESPACE. Then the namespace that gets loaded when you type bigdata::lasso.stars will be able to see the glmnet function. Perhaps Depends in the DESCRIPTION file should do the import automatically, but it will be faster to import just one function than everything from a package that has a lot of exports. So maybe it's a bug because we don't do that, but I think there would be complaints if we did. On the other hand, if bigdata::lasso.stars loaded glmnet onto the search path, I think that would be a bug. The search path belongs to the user, not to R, and the user might have used the :: notation to avoid messing with it. Duncan Murdoch __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] R Installation Manual - ATLAS BLAS guidance that is not in the current version
On 09/05/2012 19:54, Adler, Avraham wrote: Good afternoon. I am trying to compile a version of Rblas.dll based on ATLAS for the Corei7. I had remembered that there was mention of which file to adjust and that "xerbla" needed to be removed from one of the outputs from the last time I tried a few years ago. The most recent version of the R Installation manual does not say anything about this. An older version (2.10 I believe) has the following text: Optionally, you can install a version of ATLAS (`math-atlas.sourceforge.net' (http://math-atlas.sourceforge.net/)) tuned to your system for fast linear algebra routines. Pre-built `Rblas.dll' for various Pentium and AthlonXP chips are available in the `windows/contrib/ATLAS' area on CRAN. If you are building R from source, there are macros `USE_ATLAS' and `ATLAS_PATH' in the file `MkRules'. Set `USE_ATLAS = YES' and `ATLAS_PATH' to where the ATLAS libraries are located. You will need to make the libraries yourself(1): none of the binaries we have seen are compiled for the correct compiler. Since R has its own `xerbla' it is best to delete that in ATLAS by ar d /path/to/libf77blas.a xerbla.o Would it be possible to restore the above information to the next version of the manual for future reference, please? No, as this is no longer supported (and various attempts to make it work have failed with incorrect results/segfaults). We do support the use of the legacy Goto BLAS these days (64-bit only, but anyone who cares about performance will be using 64-bit). As you have found in another thread, avoid over-optimizing code: the risk of incorrect answers is serious and the performance gains small. Thank you, Avraham Adler __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel -- Brian D. Ripley, rip...@stats.ox.ac.uk Professor of Applied Statistics, http://www.stats.ox.ac.uk/~ripley/ University of Oxford, Tel: +44 1865 272861 (self) 1 South Parks Road, +44 1865 272866 (PA) Oxford OX1 3TG, UKFax: +44 1865 272595 __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] R package dependency issues when namespace is not attached
On 13.05.2012 10:59, Duncan Murdoch wrote: On 12-05-13 3:15 AM, Jeroen Ooms wrote: I have always assumed that having a package in the 'Depends' field would automatically also import the namespace. However, it seems that in R 2.15, dependencies do not become available until the package is actually attached to the searchpath. Is this intended behavior? The problem appears as follows: Suppose there is a package 'Child' which Depends, but does not explicitly import a package called 'Parent' and contains a function that calls out to an object in the namespace of 'Parent'. When this function is called without attaching 'Child' to the search path, the function in 'Parent' cannot be found. Here an example from the manual of the bigdata package, but the problem is very widespread: x = matrix(rnorm(50*80),50,80) beta = c(3,2,1.5,rep(0,77)) y = rnorm(50) + x%*%beta z1 = bigdata::lasso.stars(x,y) The example fails because lasso.stars depends on 'glmnet' which is not loaded until bigdata is attached. The only way to be able to call lasso.stars is to actually attach the bigdata package: library(bigdata) z1 = bigdata::lasso.stars(x,y) Now to further complicate things, it seems that this problem is inherited to any 'grandchild' package that imports, in this case, the lasso.stars function. I have a hard time finding a good example but I am sure they are out there. Is this a bug? I know that it can be avoided by asking package authors to use Imports instead of Depends, but in practice the majority of the packages on CRAN still use Depends. It seems like the problem is easily fixed if R would automatically import the namespace of any Depends packages into to the child package namespace? Not sure if it's a bug, but the correct solution in bigdata is to import the glmnet function in its NAMESPACE. Then the namespace that gets loaded when you type bigdata::lasso.stars will be able to see the glmnet function. Perhaps Depends in the DESCRIPTION file should do the import automatically, but it will be faster to import just one function than everything from a package that has a lot of exports. So maybe it's a bug because we don't do that, but I think there would be complaints if we did. On the other hand, if bigdata::lasso.stars loaded glmnet onto the search path, I think that would be a bug. The search path belongs to the user, not to R, and the user might have used the :: notation to avoid messing with it. I do not see any problem in R. If someone is going to import a Namespace, he or she has to do that via import directives in the NAMESPACE file. If someone is going to have a package on the search path, he or she has to require() it. The DESCRIPTION file is used to derive the dependency structures among packages for installation order, check order etc. Best, Uwe Duncan Murdoch __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] R package dependency issues when namespace is not attached
On Sun, May 13, 2012 at 10:14 AM, Uwe Ligges wrote: > I do not see any problem in R. If someone is going to import a Namespace, he > or she has to do that via import directives in the NAMESPACE file. If > someone is going to have a package on the search path, he or she has to > require() it. The DESCRIPTION file is used to derive the dependency > structures among packages for installation order, check order etc. I am not sure everyone is aware of this. Many package authors seem to be assuming that having a package in the Depends field of the DESCRIPTION is a sufficient condition for having the dependency package available at runtime, regardless of how the function is invoked by the user. I think this is the usual meaning of a dependency. There are a lot of packages on CRAN that use Depends and are not explicitly importing anything. Among others, this holds for any package without a NAMESPACE file. Also looking at the definition of the 'Depends' field in the 'writing r extensions' manual there is not a single hint that Depends is not sufficient for having the package available at runtime, and any function that is used should still be manually imported or required() as you suggest. __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] R package dependency issues when namespace is not attached
On Sun, May 13, 2012 at 10:14 AM, Uwe Ligges wrote: > > > On 13.05.2012 10:59, Duncan Murdoch wrote: >> >> On 12-05-13 3:15 AM, Jeroen Ooms wrote: >>> >>> I have always assumed that having a package in the 'Depends' field >>> would automatically also import the namespace. However, it seems that >>> in R 2.15, dependencies do not become available until the package is >>> actually attached to the searchpath. Is this intended behavior? >>> >>> The problem appears as follows: Suppose there is a package 'Child' >>> which Depends, but does not explicitly import a package called >>> 'Parent' and contains a function that calls out to an object in the >>> namespace of 'Parent'. When this function is called without attaching >>> 'Child' to the search path, the function in 'Parent' cannot be found. >>> >>> Here an example from the manual of the bigdata package, but the >>> problem is very widespread: >>> >>> x = matrix(rnorm(50*80),50,80) >>> beta = c(3,2,1.5,rep(0,77)) >>> y = rnorm(50) + x%*%beta >>> z1 = bigdata::lasso.stars(x,y) >>> >>> The example fails because lasso.stars depends on 'glmnet' which is not >>> loaded until bigdata is attached. The only way to be able to call >>> lasso.stars is to actually attach the bigdata package: >>> >>> library(bigdata) >>> z1 = bigdata::lasso.stars(x,y) >>> >>> Now to further complicate things, it seems that this problem is >>> inherited to any 'grandchild' package that imports, in this case, the >>> lasso.stars function. I have a hard time finding a good example but I >>> am sure they are out there. >>> >>> Is this a bug? I know that it can be avoided by asking package authors >>> to use Imports instead of Depends, but in practice the majority of the >>> packages on CRAN still use Depends. It seems like the problem is >>> easily fixed if R would automatically import the namespace of any >>> Depends packages into to the child package namespace? >> >> >> Not sure if it's a bug, but the correct solution in bigdata is to import >> the glmnet function in its NAMESPACE. Then the namespace that gets >> loaded when you type bigdata::lasso.stars will be able to see the glmnet >> function. >> >> Perhaps Depends in the DESCRIPTION file should do the import >> automatically, but it will be faster to import just one function than >> everything from a package that has a lot of exports. So maybe it's a bug >> because we don't do that, but I think there would be complaints if we did. >> >> On the other hand, if bigdata::lasso.stars loaded glmnet onto the search >> path, I think that would be a bug. The search path belongs to the user, >> not to R, and the user might have used the :: notation to avoid messing >> with it. > > > I do not see any problem in R. If someone is going to import a Namespace, he > or she has to do that via import directives in the NAMESPACE file. If > someone is going to have a package on the search path, he or she has to > require() it. The DESCRIPTION file is used to derive the dependency > structures among packages for installation order, check order etc. So should package authors both list a package in the depends of DESCRIPTION and explicitly import what is needed so if someone else uses their code without loading the package, everything needed is available? > > Best, > Uwe > > > > > >> Duncan Murdoch >> >> __ >> R-devel@r-project.org mailing list >> https://stat.ethz.ch/mailman/listinfo/r-devel > > > __ > R-devel@r-project.org mailing list > https://stat.ethz.ch/mailman/listinfo/r-devel -- Joshua Wiley Ph.D. Student, Health Psychology Programmer Analyst II, Statistical Consulting Group University of California, Los Angeles https://joshuawiley.com/ __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] R package dependency issues when namespace is not attached
On 12-05-13 3:14 PM, Jeroen Ooms wrote: On Sun, May 13, 2012 at 10:14 AM, Uwe Ligges wrote: I do not see any problem in R. If someone is going to import a Namespace, he or she has to do that via import directives in the NAMESPACE file. If someone is going to have a package on the search path, he or she has to require() it. The DESCRIPTION file is used to derive the dependency structures among packages for installation order, check order etc. I am not sure everyone is aware of this. Many package authors seem to be assuming that having a package in the Depends field of the DESCRIPTION is a sufficient condition for having the dependency package available at runtime, regardless of how the function is invoked by the user. I think this is the usual meaning of a dependency. There are a lot of packages on CRAN that use Depends and are not explicitly importing anything. Among others, this holds for any package without a NAMESPACE file. Also looking at the definition of the 'Depends' field in the 'writing r extensions' manual there is not a single hint that Depends is not sufficient for having the package available at runtime, and any function that is used should still be manually imported or required() as you suggest. What do you suggest as the solution? Duncan Murdoch __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] R package dependency issues when namespace is not attached
On 05/13/2012 12:14 PM, Jeroen Ooms wrote: On Sun, May 13, 2012 at 10:14 AM, Uwe Ligges wrote: I do not see any problem in R. If someone is going to import a Namespace, he or she has to do that via import directives in the NAMESPACE file. If someone is going to have a package on the search path, he or she has to require() it. The DESCRIPTION file is used to derive the dependency structures among packages for installation order, check order etc. I am not sure everyone is aware of this. Many package authors seem to be assuming that having a package in the Depends field of the DESCRIPTION is a sufficient condition for having the dependency package available at runtime, regardless of how the function is invoked by the user. I think this is the usual meaning of a I think this is because name spaces are relatively new, so authors are yet to realize the consequences of not importing the definitions their package uses. As a package developer, I want to have the code my package sees be exactly what is needed, and no more. There are many good reasons for this, including isolating as much as possible my code from changes in other packages and minimizing the costs of symbol look-up. These issues become increasing important as the hierarchy of package relationships becomes deep. The best practice is for authors to import all necessary symbols, but no more! Martin dependency. There are a lot of packages on CRAN that use Depends and are not explicitly importing anything. Among others, this holds for any package without a NAMESPACE file. Also looking at the definition of the 'Depends' field in the 'writing r extensions' manual there is not a single hint that Depends is not sufficient for having the package available at runtime, and any function that is used should still be manually imported or required() as you suggest. __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel -- Computational Biology Fred Hutchinson Cancer Research Center 1100 Fairview Ave. N. PO Box 19024 Seattle, WA 98109 Location: M1-B861 Telephone: 206 667-2793 __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] R package dependency issues when namespace is not attached
On 12-05-13 4:06 PM, Martin Morgan wrote: On 05/13/2012 12:14 PM, Jeroen Ooms wrote: On Sun, May 13, 2012 at 10:14 AM, Uwe Ligges wrote: I do not see any problem in R. If someone is going to import a Namespace, he or she has to do that via import directives in the NAMESPACE file. If someone is going to have a package on the search path, he or she has to require() it. The DESCRIPTION file is used to derive the dependency structures among packages for installation order, check order etc. I am not sure everyone is aware of this. Many package authors seem to be assuming that having a package in the Depends field of the DESCRIPTION is a sufficient condition for having the dependency package available at runtime, regardless of how the function is invoked by the user. I think this is the usual meaning of a I think this is because name spaces are relatively new, so authors are yet to realize the consequences of not importing the definitions their package uses. They aren't that new, but I think our efforts at back-compatibility have slowed adoption. If we were more demanding of package developers, we wouldn't have this problem; but I think we'd have a lot fewer packages. Even with our current policy of aiming for back-compatibility we get a lot of complaints that we are asking too much. As a package developer, I want to have the code my package sees be exactly what is needed, and no more. There are many good reasons for this, including isolating as much as possible my code from changes in other packages and minimizing the costs of symbol look-up. These issues become increasing important as the hierarchy of package relationships becomes deep. The best practice is for authors to import all necessary symbols, but no more! I agree, but many authors don't want to think about things that way. Duncan Murdoch Martin dependency. There are a lot of packages on CRAN that use Depends and are not explicitly importing anything. Among others, this holds for any package without a NAMESPACE file. Also looking at the definition of the 'Depends' field in the 'writing r extensions' manual there is not a single hint that Depends is not sufficient for having the package available at runtime, and any function that is used should still be manually imported or required() as you suggest. __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] R package dependency issues when namespace is not attached
On Sun, May 13, 2012 at 1:06 PM, Martin Morgan wrote: > On 05/13/2012 12:14 PM, Jeroen Ooms wrote: > As a package developer, I want to have the code my package sees be exactly > what is needed, and no more. Exactly. That is why you probably don't use Depends, but Imports in combination with a NAMESPACE file. Which is great, and we should encourage that practice. But as long as 'Depends' is also supported, this should be working properly as well. Here a quote from http://cran.r-project.org/doc/contrib/Leisch-CreatingPackages.pdf: "A stronger form of dependency can be specified in the optional Depends field listing packages which are necessary to run our code." It think it seems reasonable to assume that when a package author decides to use 'Depends' (for whatever reason), they want the namespace to be available to their package. Hence I think R should import the full namespace of packages in the Depends field. I don't think this will generate too much overhead, because in most circumstances, the package will be loaded and attached anyway. Furthermore this will not slow down or affect packages that use the better practice of specifying 'Imports' instead of 'Depends' and explicitly import only required symbols. __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] R package dependency issues when namespace is not attached
On 05/13/2012 01:39 PM, Duncan Murdoch wrote:
On 12-05-13 4:06 PM, Martin Morgan wrote:
On 05/13/2012 12:14 PM, Jeroen Ooms wrote:
On Sun, May 13, 2012 at 10:14 AM, Uwe Ligges
wrote:
I do not see any problem in R. If someone is going to import a
Namespace, he
or she has to do that via import directives in the NAMESPACE file. If
someone is going to have a package on the search path, he or she has to
require() it. The DESCRIPTION file is used to derive the dependency
structures among packages for installation order, check order etc.
I am not sure everyone is aware of this. Many package authors seem to
be assuming that having a package in the Depends field of the
DESCRIPTION is a sufficient condition for having the dependency
package available at runtime, regardless of how the function is
invoked by the user. I think this is the usual meaning of a
I think this is because name spaces are relatively new, so authors are
yet to realize the consequences of not importing the definitions their
package uses.
They aren't that new, but I think our efforts at back-compatibility have
slowed adoption. If we were more demanding of package developers, we
wouldn't have this problem; but I think we'd have a lot fewer packages.
Even with our current policy of aiming for back-compatibility we get a
lot of complaints that we are asking too much.
perhaps it would be easy to provide a check (I realize this is close on
the heels of the undefined global variables thread), along the lines of
> library(codetools)
> checkUsageEnv(getNamespace("bigdata"), suppressLocal=TRUE)
lasso.stars: no visible global function definition for 'glmnet'
lasso.stars: no visible global function definition for 'glmnet'
or to suggest a NAMESPACE, done imperfectly by
https://hedgehog.fhcrc.org/bioconductor/trunk/madman/Rpacks/codetoolsBioC
(username / password: readonly)
> library(codetoolsBioC)
> library(bigdata)
> deps <- writeNamespaceImports("bigdata", file=stdout())
#Generated by codetoolsBioC version 0.0.16
#Timestamp: Sun May 13 15:01:12 2012
#Imports: glmnet, graphics, Matrix
importMethodsFrom(Matrix, mean, t)
importFrom(glmnet, glmnet)
importFrom(graphics, lines, par, plot)
As a package developer, I want to have the code my package sees be
exactly what is needed, and no more. There are many good reasons for
this, including isolating as much as possible my code from changes in
other packages and minimizing the costs of symbol look-up. These issues
become increasing important as the hierarchy of package relationships
becomes deep.
The best practice is for authors to import all necessary symbols, but no
more!
I agree, but many authors don't want to think about things that way.
Duncan Murdoch
Martin
dependency. There are a lot of packages on CRAN that use Depends and
are not explicitly importing anything. Among others, this holds for
any package without a NAMESPACE file.
Also looking at the definition of the 'Depends' field in the 'writing
r extensions' manual there is not a single hint that Depends is not
sufficient for having the package available at runtime, and any
function that is used should still be manually imported or required()
as you suggest.
__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
--
Computational Biology
Fred Hutchinson Cancer Research Center
1100 Fairview Ave. N. PO Box 19024 Seattle, WA 98109
Location: M1-B861
Telephone: 206 667-2793
__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
