Re: [Rd] legitimate use of :::

[Uwe Ligges]

On 22.08.2013 07:45, Yihui Xie wrote:

Hi,

So now R CMD check starts to warn against :::, but I believe sometimes
it is legitimate to use it when developing R packages. For example, I
have some utils functions that are not exported but I want to share
them across the packages that I maintain. I do not need to coordinate
with other authors about these internal functions since I'm the only
author and I know clearly what I'm doing, and I want to avoid copying
and pasting the code across packages just to avoid the NOTE in R CMD
check. What should I do in this case?


Nothing. The way you describe above seems to be a reasonable usage, iff 
you are the same maintainer who knows what is going on. Other 
maintainers should not use one of your not exported (hence non API) 
functions, of course.


Uwe Ligges




Regards,
Yihui
--
Yihui Xie 
Web: http://yihui.name
Department of Statistics, Iowa State University
102 Snedecor Hall, Ames, IA

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel



__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

[Rd] Confusion about Depends:, Imports:, Enhances:, import(), inportFrom()

[Michael Friendly]

In checking my vcdExtra package, the following NOTE newly appeared 
(R-Forge, using R version 3.0.1 Patched (2013-08-20 r63635))


Package in Depends field not imported from: ‘gnm’
  These packages needs to imported from for the case when
  this namespace is loaded but not attached.

In the DESCRIPTION file, I have

Depends: R (>= 2.10), vcd, gnm (>= 1.0.3)

In NAMESPACE:

# we are a vcd extension
import(vcd)

I've read 1.1.1 of R-Exts, but it is not clear to me whether I should 
also import gnm or change

the DESCRIPTION file to use

Imports: vcd, gnm (>= 1.0.3)

R-Exts says: The ‘Imports’ field lists packages whose namespaces are 
imported from (as specified in the
NAMESPACE file) but which do not need to be attached, but how can I tell 
if gnm needs to be attached?


Also, what is the difference between Imports: in DESCRIPTION and 
imports() in NAMESPACE?


-Michael

--
Michael Friendly Email: friendly AT yorku DOT ca
Professor, Psychology Dept. & Chair, Quantitative Methods
York University  Voice: 416 736-2100 x66249 Fax: 416 736-5814
4700 Keele StreetWeb:   http://www.datavis.ca
Toronto, ONT  M3J 1P3 CANADA

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [Rd] Confusion about Depends:, Imports:, Enhances:, import(), inportFrom()

[R. Michael Weylandt]

On Aug 22, 2013, at 9:09, Michael Friendly  wrote:

> In checking my vcdExtra package, the following NOTE newly appeared (R-Forge, 
> using R version 3.0.1 Patched (2013-08-20 r63635))
> 
> Package in Depends field not imported from: ‘gnm’
>  These packages needs to imported from for the case when
>  this namespace is loaded but not attached.
> 
> In the DESCRIPTION file, I have
> 
> Depends: R (>= 2.10), vcd, gnm (>= 1.0.3)
> 
> In NAMESPACE:
> 
> # we are a vcd extension
> import(vcd)
> 
> I've read 1.1.1 of R-Exts, but it is not clear to me whether I should also 
> import gnm or change
> the DESCRIPTION file to use
> 
> Imports: vcd, gnm (>= 1.0.3)
> 
> R-Exts says: The ‘Imports’ field lists packages whose namespaces are imported 
> from (as specified in the
> NAMESPACE file) but which do not need to be attached, but how can I tell if 
> gnm needs to be attached?

I think the current best practice is to use Imports unless gnm provides 
functions the end user needs in order to use your package. 

Practically, I think this usually comes down to asking whether gnm provides 
relevant generics -- if you provide methods only, the end user would only be 
able to call them directly (ick!) unless gnm is both loaded and attached 
(Depends)

If gnm is tools for your code, but not the user to call directly, Imports is 
cleaner. 

Does that help?

Michael

> 
> Also, what is the difference between Imports: in DESCRIPTION and imports() in 
> NAMESPACE?
> 

One enables the other: Imports in DESCRIPTION allows for both imports() and 
importsFrom(). It could probably be automatic, but the DESC file is much older 
than the NAMESPACE file. 


> -Michael
> 
> -- 
> Michael Friendly Email: friendly AT yorku DOT ca
> Professor, Psychology Dept. & Chair, Quantitative Methods
> York University  Voice: 416 736-2100 x66249 Fax: 416 736-5814
> 4700 Keele StreetWeb:   http://www.datavis.ca
> Toronto, ONT  M3J 1P3 CANADA
> 
> __
> R-devel@r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-devel

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [Rd] legitimate use of :::

[Michael Friendly]

On 8/22/2013 7:45 AM, Uwe Ligges wrote:



On 22.08.2013 07:45, Yihui Xie wrote:

Hi,

So now R CMD check starts to warn against :::, but I believe sometimes
it is legitimate to use it when developing R packages. For example, I
have some utils functions that are not exported but I want to share
them across the packages that I maintain. I do not need to coordinate
with other authors about these internal functions since I'm the only
author and I know clearly what I'm doing, and I want to avoid copying
and pasting the code across packages just to avoid the NOTE in R CMD
check. What should I do in this case?


Nothing. The way you describe above seems to be a reasonable usage, iff
you are the same maintainer who knows what is going on. Other
maintainers should not use one of your not exported (hence non API)
functions, of course.

Uwe Ligges




Related to this is the use of other-package unexported utility functions 
that don't pass Uwe's iff test, but I, as maintainer,

want to use in my package.

Cases in point:  in heplots, I had used stats:::Pillai, stats:::Wilks,
stats:::Roy and stats:::LH for calculation in one of my functions.
Similarly, I had a need to use car:::df.terms, also unexported, but
don't want to ask John Fox to export it just for my use.  Uwe's
reply suggests that I should not be using car:::df.terms, however.

To avoid the NOTEs (which often triggers a 'pls fix' upon submission to
CRAN), I simply copied/pasted these functions to my package, but this 
seems wasteful.


-Michael


--
Michael Friendly Email: friendly AT yorku DOT ca
Professor, Psychology Dept. & Chair, Quantitative Methods
York University  Voice: 416 736-2100 x66249 Fax: 416 736-5814
4700 Keele StreetWeb:   http://www.datavis.ca
Toronto, ONT  M3J 1P3 CANADA

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [Rd] Confusion about Depends:, Imports:, Enhances:, import(), inportFrom()

[Henrik Bengtsson]

On Thu, Aug 22, 2013 at 10:05 AM, R. Michael Weylandt
  wrote:
>
>
> On Aug 22, 2013, at 9:09, Michael Friendly  wrote:
>
>> In checking my vcdExtra package, the following NOTE newly appeared (R-Forge, 
>> using R version 3.0.1 Patched (2013-08-20 r63635))
>>
>> Package in Depends field not imported from: ‘gnm’
>>  These packages needs to imported from for the case when
>>  this namespace is loaded but not attached.
>>
>> In the DESCRIPTION file, I have
>>
>> Depends: R (>= 2.10), vcd, gnm (>= 1.0.3)
>>
>> In NAMESPACE:
>>
>> # we are a vcd extension
>> import(vcd)
>>
>> I've read 1.1.1 of R-Exts, but it is not clear to me whether I should also 
>> import gnm or change
>> the DESCRIPTION file to use
>>
>> Imports: vcd, gnm (>= 1.0.3)
>>
>> R-Exts says: The ‘Imports’ field lists packages whose namespaces are 
>> imported from (as specified in the
>> NAMESPACE file) but which do not need to be attached, but how can I tell if 
>> gnm needs to be attached?
>
> I think the current best practice is to use Imports unless gnm provides 
> functions the end user needs in order to use your package.
>
> Practically, I think this usually comes down to asking whether gnm provides 
> relevant generics -- if you provide methods only, the end user would only be 
> able to call them directly (ick!) unless gnm is both loaded and attached 
> (Depends)
>
> If gnm is tools for your code, but not the user to call directly, Imports is 
> cleaner.
>
> Does that help?
>
> Michael
>
>>
>> Also, what is the difference between Imports: in DESCRIPTION and imports() 
>> in NAMESPACE?
>>
>
> One enables the other: Imports in DESCRIPTION allows for both imports() and 
> importsFrom(). It could probably be automatic, but the DESC file is much 
> older than the NAMESPACE file.

I pretty sure you can also use import()/importFrom() in NAMESPACE for
packages listed under Depends in DESCRIPTION.

/Henrik

>
>
>> -Michael
>>
>> --
>> Michael Friendly Email: friendly AT yorku DOT ca
>> Professor, Psychology Dept. & Chair, Quantitative Methods
>> York University  Voice: 416 736-2100 x66249 Fax: 416 736-5814
>> 4700 Keele StreetWeb:   http://www.datavis.ca
>> Toronto, ONT  M3J 1P3 CANADA
>>
>> __
>> R-devel@r-project.org mailing list
>> https://stat.ethz.ch/mailman/listinfo/r-devel
>
> __
> R-devel@r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-devel

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [Rd] Confusion about Depends:, Imports:, Enhances:, import(), inportFrom()

[R. Michael Weylandt]

On Aug 22, 2013, at 15:33, Henrik Bengtsson  wrote:

> On Thu, Aug 22, 2013 at 10:05 AM, R. Michael Weylandt
>   wrote:
>> 
>> 
>> On Aug 22, 2013, at 9:09, Michael Friendly  wrote:
>> 
>>> In checking my vcdExtra package, the following NOTE newly appeared 
>>> (R-Forge, using R version 3.0.1 Patched (2013-08-20 r63635))
>>> 
>>> Package in Depends field not imported from: ‘gnm’
>>> These packages needs to imported from for the case when
>>> this namespace is loaded but not attached.
>>> 
>>> In the DESCRIPTION file, I have
>>> 
>>> Depends: R (>= 2.10), vcd, gnm (>= 1.0.3)
>>> 
>>> In NAMESPACE:
>>> 
>>> # we are a vcd extension
>>> import(vcd)
>>> 
>>> I've read 1.1.1 of R-Exts, but it is not clear to me whether I should also 
>>> import gnm or change
>>> the DESCRIPTION file to use
>>> 
>>> Imports: vcd, gnm (>= 1.0.3)
>>> 
>>> R-Exts says: The ‘Imports’ field lists packages whose namespaces are 
>>> imported from (as specified in the
>>> NAMESPACE file) but which do not need to be attached, but how can I tell if 
>>> gnm needs to be attached?
>> 
>> I think the current best practice is to use Imports unless gnm provides 
>> functions the end user needs in order to use your package.
>> 
>> Practically, I think this usually comes down to asking whether gnm provides 
>> relevant generics -- if you provide methods only, the end user would only be 
>> able to call them directly (ick!) unless gnm is both loaded and attached 
>> (Depends)
>> 
>> If gnm is tools for your code, but not the user to call directly, Imports is 
>> cleaner.
>> 
>> Does that help?
>> 
>> Michael
>> 
>>> 
>>> Also, what is the difference between Imports: in DESCRIPTION and imports() 
>>> in NAMESPACE?
>> 
>> One enables the other: Imports in DESCRIPTION allows for both imports() and 
>> importsFrom(). It could probably be automatic, but the DESC file is much 
>> older than the NAMESPACE file.
> 
> I pretty sure you can also use import()/importFrom() in NAMESPACE for
> packages listed under Depends in DESCRIPTION.
> 

Agreed, but you couldn't auto-generate Depends from the namespace file. I 
suppose if (per my above) Depends was only used for generics...

Not sure if the devtools world does anything like this, though it seems logical 
enough. 

Michael

> /Henrik
> 
>> 
>> 
>>> -Michael
>>> 
>>> --
>>> Michael Friendly Email: friendly AT yorku DOT ca
>>> Professor, Psychology Dept. & Chair, Quantitative Methods
>>> York University  Voice: 416 736-2100 x66249 Fax: 416 736-5814
>>> 4700 Keele StreetWeb:   http://www.datavis.ca
>>> Toronto, ONT  M3J 1P3 CANADA
>>> 
>>> __
>>> R-devel@r-project.org mailing list
>>> https://stat.ethz.ch/mailman/listinfo/r-devel
>> 
>> __
>> R-devel@r-project.org mailing list
>> https://stat.ethz.ch/mailman/listinfo/r-devel

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [Rd] legitimate use of :::

[peter dalgaard]

On Aug 22, 2013, at 20:57 , Michael Friendly wrote:

> Cases in point:  in heplots, I had used stats:::Pillai, stats:::Wilks,
> stats:::Roy and stats:::LH for calculation in one of my functions.

That particular case has been on what remains of my conscience for some time

-- 
Peter Dalgaard, Professor,
Center for Statistics, Copenhagen Business School
Solbjerg Plads 3, 2000 Frederiksberg, Denmark
Phone: (+45)38153501
Email: pd@cbs.dk  Priv: pda...@gmail.com

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [Rd] legitimate use of :::

[John Fox]

Dear Michael and Uwe,

> -Original Message-
> From: r-devel-boun...@r-project.org [mailto:r-devel-bounces@r-
> project.org] On Behalf Of Michael Friendly
> Sent: Thursday, August 22, 2013 2:57 PM
> To: Uwe Ligges
> Cc: R-devel
> Subject: Re: [Rd] legitimate use of :::
> 
> On 8/22/2013 7:45 AM, Uwe Ligges wrote:
> >
> >
> > On 22.08.2013 07:45, Yihui Xie wrote:
> >> Hi,
> >>
> >> So now R CMD check starts to warn against :::, but I believe
> sometimes
> >> it is legitimate to use it when developing R packages. For example,
> I
> >> have some utils functions that are not exported but I want to share
> >> them across the packages that I maintain. I do not need to
> coordinate
> >> with other authors about these internal functions since I'm the only
> >> author and I know clearly what I'm doing, and I want to avoid
> copying
> >> and pasting the code across packages just to avoid the NOTE in R CMD
> >> check. What should I do in this case?
> >
> > Nothing. The way you describe above seems to be a reasonable usage,
> iff
> > you are the same maintainer who knows what is going on. Other
> > maintainers should not use one of your not exported (hence non API)
> > functions, of course.
> >
> > Uwe Ligges
> >
> >
> 
> Related to this is the use of other-package unexported utility
> functions
> that don't pass Uwe's iff test, but I, as maintainer,
> want to use in my package.
> 
> Cases in point:  in heplots, I had used stats:::Pillai, stats:::Wilks,
> stats:::Roy and stats:::LH for calculation in one of my functions.
> Similarly, I had a need to use car:::df.terms, also unexported, but
> don't want to ask John Fox to export it just for my use.  Uwe's
> reply suggests that I should not be using car:::df.terms, however.
> 
> To avoid the NOTEs (which often triggers a 'pls fix' upon submission to
> CRAN), I simply copied/pasted these functions to my package, but this
> seems wasteful.

I think that the ideal solution is for everyone to export functions that
somewhat else might want, but it's hard to anticipate what these are, and it
would be useful then to differentiate functions that are meant for "end"
users from those meant for developers. Maybe packages could document the
latter in something like a Utilities.Rd file. Probably there's a better,
more formal, solution.

The stats:::Pillai, Wilks, HL, and Roy functions seem reasonable candidates
for export -- I too use these functions, in the car package, and have
resorted to the fix that Michael adopted. I'd be happy to export df.terms,
but would rather segregate it from end-user functions.

It's also clear to me that enforcing namespace conventions more
consistently, which is certainly desirable in the abstract, opens a can of
worms, especially for the CRAN administrators. One hopes that we'll all
survive the process and will have better packages in the end.

My two cents.

John

> 
> -Michael
> 
> 
> --
> Michael Friendly Email: friendly AT yorku DOT ca
> Professor, Psychology Dept. & Chair, Quantitative Methods
> York University  Voice: 416 736-2100 x66249 Fax: 416 736-5814
> 4700 Keele StreetWeb:   http://www.datavis.ca
> Toronto, ONT  M3J 1P3 CANADA
> 
> __
> R-devel@r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-devel

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [Rd] legitimate use of :::

[Hadley Wickham]

> To avoid the NOTEs (which often triggers a 'pls fix' upon submission to
> CRAN), I simply copied/pasted these functions to my package, but this seems
> wasteful.

Wasteful of disk space, but disk space is cheap. It's less wasteful of
your time, if the referenced code breaks in an unexpected time.  Your
time is much more valuable than disk space.

A gigabyte of disk space costs about $0.10, so even if you value your
time at a very conservative rate of $100 / hour, you should only spend
an hour of your time reducing package size if it saves at least 1 TB
of disk space. That's a lot of copies of a function!

Hadley

-- 
Chief Scientist, RStudio
http://had.co.nz/

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [Rd] legitimate use of :::

[Yihui Xie]

r63654 has fixed this particular issue, and R-devel will no longer
warn against the use of ::: on packages of the same maintainer.

Regards,
Yihui
--
Yihui Xie 
Web: http://yihui.name
Department of Statistics, Iowa State University
102 Snedecor Hall, Ames, IA


On Thu, Aug 22, 2013 at 6:45 AM, Uwe Ligges
 wrote:
>
>
> On 22.08.2013 07:45, Yihui Xie wrote:
>>
>> Hi,
>>
>> So now R CMD check starts to warn against :::, but I believe sometimes
>> it is legitimate to use it when developing R packages. For example, I
>> have some utils functions that are not exported but I want to share
>> them across the packages that I maintain. I do not need to coordinate
>> with other authors about these internal functions since I'm the only
>> author and I know clearly what I'm doing, and I want to avoid copying
>> and pasting the code across packages just to avoid the NOTE in R CMD
>> check. What should I do in this case?
>
>
> Nothing. The way you describe above seems to be a reasonable usage, iff you
> are the same maintainer who knows what is going on. Other maintainers should
> not use one of your not exported (hence non API) functions, of course.
>
> Uwe Ligges

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [Rd] legitimate use of :::

[Henrik Bengtsson]

On Thu, Aug 22, 2013 at 2:03 PM, Hadley Wickham  wrote:
>> To avoid the NOTEs (which often triggers a 'pls fix' upon submission to
>> CRAN), I simply copied/pasted these functions to my package, but this seems
>> wasteful.
>
> Wasteful of disk space, but disk space is cheap. It's less wasteful of
> your time, if the referenced code breaks in an unexpected time.  Your
> time is much more valuable than disk space.
>
> A gigabyte of disk space costs about $0.10, so even if you value your
> time at a very conservative rate of $100 / hour, you should only spend
> an hour of your time reducing package size if it saves at least 1 TB
> of disk space. That's a lot of copies of a function!

A bigger issue is source-code license conflicts; you may cut'n'paste
GPL code into a distribution that is under another license.

/Henrik


>
> Hadley
>
> --
> Chief Scientist, RStudio
> http://had.co.nz/
>
> __
> R-devel@r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-devel

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [Rd] legitimate use of :::

[Gabriel Becker]

On Thu, Aug 22, 2013 at 2:03 PM, Hadley Wickham  wrote:
>> To avoid the NOTEs (which often triggers a 'pls fix' upon submission to
>> CRAN), I simply copied/pasted these functions to my package, but this
seems
>> wasteful.

>Wasteful of disk space, but disk space is cheap. It's less wasteful of
>your time, if the referenced code breaks in an unexpected time.  Your
>time is much more valuable than disk space.

On the other hand, it's quite dangerous software design. What if the
original author finds a bug and implements a fix, but you don't hear about
it?

Furthermore, what happens when I come along and need the same
functionality? Sure I could make a copy, but maybe I only know about your
copy and don't know it is a copy of something else, so now we have a copy
of a copy, which is even more problematic. Using ::: prevents this issue.

Using ::: also allows us to get the credit for the idea/functionality
right, which is important and which our field could stand for some
improvement on.

I would argue, though, in response to the original question that if you
have functions that you, yourself are using in multiple packages, or that
you know other people are going to want to use, that is an indication that
the function is broadly useful and really should be exported from somewhere
(either the package it lives in now or a lightweight dependency/utility
package where you collect all such functions you create).

The much muddier issue is what to do when someone has written a function
that *almost* does what you want, but has a hardcoded limitation in it that
prevents you from using it. That situation is harder to deal with imo
unless the maintainer of the package is open to patches/pull requests.

~G



-- 
Gabriel Becker
Graduate Student
Statistics Department
University of California, Davis

[[alternative HTML version deleted]]

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [Rd] legitimate use of :::

[Hadley Wickham]

>>Wasteful of disk space, but disk space is cheap. It's less wasteful of
>>your time, if the referenced code breaks in an unexpected time.  Your
>>time is much more valuable than disk space.
>
> On the other hand, it's quite dangerous software design. What if the
> original author finds a bug and implements a fix, but you don't hear about
> it?
>
> Furthermore, what happens when I come along and need the same functionality?
> Sure I could make a copy, but maybe I only know about your copy and don't
> know it is a copy of something else, so now we have a copy of a copy, which
> is even more problematic. Using ::: prevents this issue.

There are costs and benefits to both approaches. Copy-and-paste also
minimises external dependencies which can be important in some cases.
I'm not arguing for unmitigated duplication, but there are definitely
good reasons to do it.

I have quite a few v. simple functions that live in multiple packages.
Often I want to keep the dependencies of packages as lightweight as
possible (learning from past experiences) and avoid tightly coupling
packages together.

Hadley

-- 
Chief Scientist, RStudio
http://had.co.nz/

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [Rd] legitimate use of :::

[John Fox]

Dear Peter,

> -Original Message-
> From: r-devel-boun...@r-project.org [mailto:r-devel-bounces@r-
> project.org] On Behalf Of peter dalgaard
> Sent: Thursday, August 22, 2013 4:45 PM
> To: Michael Friendly
> Cc: R-devel; Uwe Ligges
> Subject: Re: [Rd] legitimate use of :::
> 
> 
> On Aug 22, 2013, at 20:57 , Michael Friendly wrote:
> 
> > Cases in point:  in heplots, I had used stats:::Pillai,
> stats:::Wilks,
> > stats:::Roy and stats:::LH for calculation in one of my functions.
> 
> That particular case has been on what remains of my conscience for some
> time
> 

Happily, it would be easy to relieve your conscience in this matter.

Best,
 John

> --
> Peter Dalgaard, Professor,
> Center for Statistics, Copenhagen Business School
> Solbjerg Plads 3, 2000 Frederiksberg, Denmark
> Phone: (+45)38153501
> Email: pd@cbs.dk  Priv: pda...@gmail.com
> 
> __
> R-devel@r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-devel

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [Rd] legitimate use of :::

[Brian Rowe]

Another point to consider is that copying someone else's code forces you to 
become a maintainer of the copied code. If there are any bug 
fixes/enhancements/what-have-you in the original you won't get those updates. 
So now you own the copied code and need to consider the cost of the codebase 
diverging (from the original).

On Aug 22, 2013, at 5:03 PM, Hadley Wickham  wrote:

>> To avoid the NOTEs (which often triggers a 'pls fix' upon submission to
>> CRAN), I simply copied/pasted these functions to my package, but this seems
>> wasteful.
> 
> Wasteful of disk space, but disk space is cheap. It's less wasteful of
> your time, if the referenced code breaks in an unexpected time.  Your
> time is much more valuable than disk space.
> 
> A gigabyte of disk space costs about $0.10, so even if you value your
> time at a very conservative rate of $100 / hour, you should only spend
> an hour of your time reducing package size if it saves at least 1 TB
> of disk space. That's a lot of copies of a function!
> 
> Hadley
> 
> -- 
> Chief Scientist, RStudio
> http://had.co.nz/
> 
> __
> R-devel@r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-devel

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [Rd] legitimate use of :::

[Hadley Wickham]

On Thu, Aug 22, 2013 at 4:52 PM, Brian Rowe  wrote:
> Another point to consider is that copying someone else's code forces you to 
> become a maintainer of the copied code. If there are any bug 
> fixes/enhancements/what-have-you in the original you won't get those updates. 
> So now you own the copied code and need to consider the cost of the codebase 
> diverging (from the original).

Sometimes that's a good thing - you're equally insulated from the
original maintainer changing the function to work in a way that you
don't like.  Again, I'm not arguing that copy-and-paste is necessarily
the right solution, but it's not necessarily the wrong solution either
- it depends on the context.

Hadley

-- 
Chief Scientist, RStudio
http://had.co.nz/

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [Rd] legitimate use of :::

[Gabor Grothendieck]

If ::: is disallowed then its likely that package developers will need
to export more functions to satisfy the consumers of those otherwise
hidden functions but if more functions are exported then there
will be a greater likelihood of conflicts among packages.

The problem seems to be that there are potentially three sorts of
functions here:

1. a function is hidden
2. a function is accessible via ::: but is not on the search path
3. a function is on the search path

The problem arises in attempting to force fit these three concepts
into only two
categories either by removing the first category (as was done previously)
or by removing the second category (which seems to be the new approach).

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [Rd] legitimate use of :::

[Brian Lee Yung Rowe]

You raise an interesting point that I've mulled over a bit: namespace 
collisions. How many of these issues would go away if there were a better 
mechanism for managing namespaces? eg in other languages you can control which 
objects/modules you wish to import from a library. Under this regime I think 
package developers would be less concerned about exposing functions that 
otherwise would be private. 

On Aug 22, 2013, at 6:27 PM, Gabor Grothendieck  wrote:

> If ::: is disallowed then its likely that package developers will need
> to export more functions to satisfy the consumers of those otherwise
> hidden functions but if more functions are exported then there
> will be a greater likelihood of conflicts among packages.
> 
> The problem seems to be that there are potentially three sorts of
> functions here:
> 
> 1. a function is hidden
> 2. a function is accessible via ::: but is not on the search path
> 3. a function is on the search path
> 
> The problem arises in attempting to force fit these three concepts
> into only two
> categories either by removing the first category (as was done previously)
> or by removing the second category (which seems to be the new approach).

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [Rd] legitimate use of :::

[Uwe Ligges]

On 23.08.2013 00:36, Brian Lee Yung Rowe wrote:

You raise an interesting point that I've mulled over a bit: namespace 
collisions. How many of these issues would go away if there were a better 
mechanism for managing namespaces? eg in other languages you can control which 
objects/modules you wish to import from a library. Under this regime I think 
package developers would be less concerned about exposing functions that 
otherwise would be private.



Exactly, the corresponding NAMESPACE directive is

importFrom()

and it should be used.



On Aug 22, 2013, at 6:27 PM, Gabor Grothendieck  wrote:


If ::: is disallowed then its likely that package developers will need
to export more functions to satisfy the consumers of those otherwise
hidden functions but if more functions are exported then there
will be a greater likelihood of conflicts among packages.

The problem seems to be that there are potentially three sorts of
functions here:

1. a function is hidden
2. a function is accessible via ::: but is not on the search path
3. a function is on the search path




Not entirely right:

If the package or only parts of it are imported via importFrom by 
another package, the package is not loaded, hence not on the search path.


Best,
Uwe Ligges



The problem arises in attempting to force fit these three concepts
into only two
categories either by removing the first category (as was done previously)
or by removing the second category (which seems to be the new approach).


__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [Rd] legitimate use of :::

[Gabor Grothendieck]

On Thu, Aug 22, 2013 at 6:41 PM, Uwe Ligges
 wrote:
>
>
> On 23.08.2013 00:36, Brian Lee Yung Rowe wrote:
>>
>> You raise an interesting point that I've mulled over a bit: namespace
>> collisions. How many of these issues would go away if there were a better
>> mechanism for managing namespaces? eg in other languages you can control
>> which objects/modules you wish to import from a library. Under this regime I
>> think package developers would be less concerned about exposing functions
>> that otherwise would be private.
>
>
>
> Exactly, the corresponding NAMESPACE directive is
>
> importFrom()
>
> and it should be used.
>
>
>
>> On Aug 22, 2013, at 6:27 PM, Gabor Grothendieck 
>> wrote:
>>
>>> If ::: is disallowed then its likely that package developers will need
>>> to export more functions to satisfy the consumers of those otherwise
>>> hidden functions but if more functions are exported then there
>>> will be a greater likelihood of conflicts among packages.
>>>
>>> The problem seems to be that there are potentially three sorts of
>>> functions here:
>>>
>>> 1. a function is hidden
>>> 2. a function is accessible via ::: but is not on the search path
>>> 3. a function is on the search path
>>
>>
>
> Not entirely right:
>
> If the package or only parts of it are imported via importFrom by another
> package, the package is not loaded, hence not on the search path.

OK but it is still true that under the new rules to use importFrom(B,
f) in package A
that f must be exported by B.  That implies that f could cause a
conflict when B is
placed on the search path via library(B) by some other package
(package C) or by the user.

f is either exported by B or not.  If f is exported by B then f will
be placed on
the search path whenever B is placed on the search path and if f is
not exported then A can't import it.  That is there is no way for B to
declare a function to be importable by another package without having that
function also placed on the search path whenever B is loaded by a library(B)l or
a Depends: B from another package.




on the search path


-- 
Statistics & Software Consulting
GKX Group, GKX Associates Inc.
tel: 1-877-GKX-GROUP
email: ggrothendieck at gmail.com

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [Rd] legitimate use of :::

[Gabriel Becker]

My understanding is that lookup happens in the imports before moving on to
the search path,  so if I understand you correctly I don't think that is an
issue. If A also *exported* f, that would be a problem...

>From writing R extensions (talking about functions in a package finding
variables, sec 1.6):

The namespace controls the search strategy for variables used by functions
in the package. If not found locally, R searches the package namespace
first, then the imports, then the base namespace and then the normal search
path.


On Thu, Aug 22, 2013 at 4:45 PM, Gabor Grothendieck  wrote:

> On Thu, Aug 22, 2013 at 6:41 PM, Uwe Ligges
>  wrote:
> >
> >
> > On 23.08.2013 00:36, Brian Lee Yung Rowe wrote:
> >>
> >> You raise an interesting point that I've mulled over a bit: namespace
> >> collisions. How many of these issues would go away if there were a
> better
> >> mechanism for managing namespaces? eg in other languages you can control
> >> which objects/modules you wish to import from a library. Under this
> regime I
> >> think package developers would be less concerned about exposing
> functions
> >> that otherwise would be private.
> >
> >
> >
> > Exactly, the corresponding NAMESPACE directive is
> >
> > importFrom()
> >
> > and it should be used.
> >
> >
> >
> >> On Aug 22, 2013, at 6:27 PM, Gabor Grothendieck <
> ggrothendi...@gmail.com>
> >> wrote:
> >>
> >>> If ::: is disallowed then its likely that package developers will need
> >>> to export more functions to satisfy the consumers of those otherwise
> >>> hidden functions but if more functions are exported then there
> >>> will be a greater likelihood of conflicts among packages.
> >>>
> >>> The problem seems to be that there are potentially three sorts of
> >>> functions here:
> >>>
> >>> 1. a function is hidden
> >>> 2. a function is accessible via ::: but is not on the search path
> >>> 3. a function is on the search path
> >>
> >>
> >
> > Not entirely right:
> >
> > If the package or only parts of it are imported via importFrom by another
> > package, the package is not loaded, hence not on the search path.
>
> OK but it is still true that under the new rules to use importFrom(B,
> f) in package A
> that f must be exported by B.  That implies that f could cause a
> conflict when B is
> placed on the search path via library(B) by some other package
> (package C) or by the user.
>
> f is either exported by B or not.  If f is exported by B then f will
> be placed on
> the search path whenever B is placed on the search path and if f is
> not exported then A can't import it.  That is there is no way for B to
> declare a function to be importable by another package without having that
> function also placed on the search path whenever B is loaded by a
> library(B)l or
> a Depends: B from another package.
>
>
>
>
> on the search path
>
>
> --
> Statistics & Software Consulting
> GKX Group, GKX Associates Inc.
> tel: 1-877-GKX-GROUP
> email: ggrothendieck at gmail.com
>
> __
> R-devel@r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-devel
>



-- 
Gabriel Becker
Graduate Student
Statistics Department
University of California, Davis

[[alternative HTML version deleted]]

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [Rd] legitimate use of :::

[Peter Meilstrup]

It would be nice if the functionality of importFrom() and import() were
available to user level code, rather than just to people building packages
for distribution. One most often encounters namespace conflicts at the user
level, when loading two packages that have no logical connection other than
both bearing on your problem of the moment.

R conflates "having namespaces" with "having a library distribution
mechanism" and while its library distribution mechanism is top notch, most
modern languages do not require you to learn the distribution procedure in
order to just have namespaces.

For instance, in Python you merely put code in a file called foo.py and
then in any other file in the same directory you type "import functionName
from foo". I.E. using namespaces does not require you to build/install
packages. Python namespaces are also hierarchical so that the question of
this thread would easily be resolved by putting functions into
foo._internal and in other packages typing import * from "foo._internal"

Peter

[[alternative HTML version deleted]]

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [Rd] legitimate use of :::

[Gabor Grothendieck]

On Thu, Aug 22, 2013 at 7:57 PM, Gabriel Becker  wrote:
> My understanding is that lookup happens in the imports before moving on to
> the search path,  so if I understand you correctly I don't think that is an
> issue. If A also *exported* f, that would be a problem...
>

A can only import f from B if f has been exported from B so while its
not a problem for A, whenever anyone issues a library(B) f will be
visible on the
search path -- the problem of potential conflicts with f remains.

B really only exported f so that A could import it but a side effect
of that is that
anyone who puts B on the search path makes f visible.

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [Rd] legitimate use of :::

[Gabriel Becker]

(missed the list the first time)

Perhaps an importHiddenFrom directive should be added to the namespace
vocabulary which can import a non-exported function. The non-exported
functions clearly exist somewhere, as package code can use them, so it
would not be impossible to allow that (though I'm not saying it would be
easy, as I haven't looked at how the namespace stuff is implemented).


On Thu, Aug 22, 2013 at 5:19 PM, Gabor Grothendieck  wrote:

> On Thu, Aug 22, 2013 at 7:57 PM, Gabriel Becker 
> wrote:
> > My understanding is that lookup happens in the imports before moving on
> to
> > the search path,  so if I understand you correctly I don't think that is
> an
> > issue. If A also *exported* f, that would be a problem...
> >
>
> A can only import f from B if f has been exported from B so while its
> not a problem for A, whenever anyone issues a library(B) f will be
> visible on the
> search path -- the problem of potential conflicts with f remains.
>
> B really only exported f so that A could import it but a side effect
> of that is that
> anyone who puts B on the search path makes f visible.
>



-- 
Gabriel Becker
Graduate Student
Statistics Department
University of California, Davis

[[alternative HTML version deleted]]

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [Rd] legitimate use of :::

[Peter Meilstrup]

On Thu, Aug 22, 2013 at 2:26 PM, Gabriel Becker wrote:

> On Thu, Aug 22, 2013 at 2:03 PM, Hadley Wickham 
> wrote:
> >> To avoid the NOTEs (which often triggers a 'pls fix' upon submission to
> >> CRAN), I simply copied/pasted these functions to my package, but this
> seems
> >> wasteful.
>
> >Wasteful of disk space, but disk space is cheap. It's less wasteful of
> >your time, if the referenced code breaks in an unexpected time.  Your
> >time is much more valuable than disk space.
>
> On the other hand, it's quite dangerous software design. What if the
> original author finds a bug and implements a fix, but you don't hear about
> it?
>
> Furthermore, what happens when I come along and need the same
> functionality? Sure I could make a copy, but maybe I only know about your
> copy and don't know it is a copy of something else, so now we have a copy
> of a copy, which is even more problematic. Using ::: prevents this issue.
>

Using ::: on a package you don't control can be more dangerous. For a
package author to choose to export a function to the public interface
represents at least some assurance that that interface will be stable or
slow-moving. But there are no implied assurances about how code in the
private namespace might change behind the scenes. I might completely
refactor the code and change the behavior of any private function between
0.0.1 releases, but I would not do that for exported functions.

Peter

[[alternative HTML version deleted]]

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [Rd] legitimate use of :::

[Brian Lee Yung Rowe]

This is what I was getting at as well. It would be great to have a call like

require(package, c('funtion.1','function.2')) 

or similar that gives users granular control over what gets imported in the 
shell. I would be drunk with joy if the same mechanism could be used to 
automatically populate the package directives.


On Aug 22, 2013, at 8:01 PM, Peter Meilstrup  wrote:

> It would be nice if the functionality of importFrom() and import() were
> available to user level code, rather than just to people building packages
> for distribution. One most often encounters namespace conflicts at the user
> level, when loading two packages that have no logical connection other than
> both bearing on your problem of the moment.
> 
> R conflates "having namespaces" with "having a library distribution
> mechanism" and while its library distribution mechanism is top notch, most
> modern languages do not require you to learn the distribution procedure in
> order to just have namespaces.
> 
> For instance, in Python you merely put code in a file called foo.py and
> then in any other file in the same directory you type "import functionName
> from foo". I.E. using namespaces does not require you to build/install
> packages. Python namespaces are also hierarchical so that the question of
> this thread would easily be resolved by putting functions into
> foo._internal and in other packages typing import * from "foo._internal"
> 
> Peter
> 
>   [[alternative HTML version deleted]]
> 
> __
> R-devel@r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-devel

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [Rd] legitimate use of :::

[Gray]

Peter Meilstrup: (05:01PM on Thu, Aug 22)

One most often encounters namespace conflicts at the user level, when
loading two packages that have no logical connection other than both
bearing on your problem of the moment.


Unless I'm mistaken, you can reassign the hidden functions, ie

fna <- joespackage:::usefulfunction

fnb <- janespackage:::usefulfunction

which is a little bit of a pain, but makes the user's code
unambiguous.  This also works with two colons for explicitly exported
functions.

--
Gray Calhoun, Assistant Professor of Economics at Iowa State 
http://gray.clhn.co (web)


__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [Rd] legitimate use of :::

[John Fox]

Dear Gray,

On Thu, 22 Aug 2013 19:41:58 -0500
 Gray  wrote:
> Peter Meilstrup: (05:01PM on Thu, Aug 22)
> >One most often encounters namespace conflicts at the user level, when
> >loading two packages that have no logical connection other than both
> >bearing on your problem of the moment.
> 
> Unless I'm mistaken, you can reassign the hidden functions, ie
> 
> fna <- joespackage:::usefulfunction
> 
> fnb <- janespackage:::usefulfunction
> 

This will now generate a note from R CMD check and an objection from the CRAN 
administrators.

Best,
 John

> which is a little bit of a pain, but makes the user's code
> unambiguous.  This also works with two colons for explicitly exported
> functions.
> 
> -- 
> Gray Calhoun, Assistant Professor of Economics at Iowa State 
> http://gray.clhn.co (web)
> 
> __
> R-devel@r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-devel

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [Rd] legitimate use of :::

[Gabriel Becker]

Hey guys,

Because I was curious and had nothing else that I should have been doing
(that second part is a lie), I looked into the namespace code.

I have a working patch that implements importHiddenFrom. It doesn't
currently check whether you then export that symbol (which should not be
allowed) but that would be easy to implement.

Is there any desire from R-core to have add the importHiddenFrom
functionality? If so I can add the export check and submit the patch either
here or on bugzilla. I figure its a long shot but hey, at least I now know
how the namespace stuff works.

I do agree with Peter Meilstrup that poking around at the internals of
someone else's code is often not a good idea, but as others have pointed
out it is done in practice in some fairly high-profile packages, and if its
going to happen it seems like it would be nice to indicate as much in the
NAMESPACE file.

~G


On Thu, Aug 22, 2013 at 5:41 PM, Gray  wrote:

> Peter Meilstrup: (05:01PM on Thu, Aug 22)
>
>  One most often encounters namespace conflicts at the user level, when
>> loading two packages that have no logical connection other than both
>> bearing on your problem of the moment.
>>
>
> Unless I'm mistaken, you can reassign the hidden functions, ie
>
> fna <- joespackage:::usefulfunction
>
> fnb <- janespackage:::usefulfunction
>
> which is a little bit of a pain, but makes the user's code
> unambiguous.  This also works with two colons for explicitly exported
> functions.
>
> --
> Gray Calhoun, Assistant Professor of Economics at Iowa State
> http://gray.clhn.co (web)
>
>
> __**
> R-devel@r-project.org mailing list
> https://stat.ethz.ch/mailman/**listinfo/r-devel
>



-- 
Gabriel Becker
Graduate Student
Statistics Department
University of California, Davis

[[alternative HTML version deleted]]

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [Rd] legitimate use of :::

[Yihui Xie]

Maybe it is not a good idea for R CMD check to check ::: at all, and a
warning in R-exts and ?':::' may well be enough. On the other hand, it
is just so easy to get around :::, because everybody can see its
source code:

> `:::`
function (pkg, name)
{
pkg <- as.character(substitute(pkg))
name <- as.character(substitute(name))
get(name, envir = asNamespace(pkg), inherits = FALSE)
}

Then the package authors who really want to take the risk may start
another "hide and seek" game, e.g.

`%:::%` = function(pkg, fun) get(fun, envir = asNamespace(pkg),
inherits = FALSE)
'stats' %:::% 'Pillai'

Non-exported functions do not necessarily imply instability. Maybe it
is just because the author is too lazy to document them, or he/she did
not realize these functions happen to be useful for another author.

Although R-devel does not warn against ::: on the packages of the same
maintainer now, these maintainers may change in the future, or one
maintainer can be an author but not maintainer of another package,
from which he/she imports an unexported function, or package authors
have coordinated well with each other about the unexported functions.
I believe there are other legitimate reasons for :::, which might make
it difficult for R to cover all these cases, and also bring additional
communications between package authors and CRAN.

In conclusion, R CMD check cannot really stop :::, and ::: can be
there for good reasons, so how about just let it go?

Regards,
Yihui
--
Yihui Xie 
Web: http://yihui.name
Department of Statistics, Iowa State University
102 Snedecor Hall, Ames, IA


On Thu, Aug 22, 2013 at 9:02 PM, Gabriel Becker  wrote:
> Hey guys,
>
> Because I was curious and had nothing else that I should have been doing
> (that second part is a lie), I looked into the namespace code.
>
> I have a working patch that implements importHiddenFrom. It doesn't
> currently check whether you then export that symbol (which should not be
> allowed) but that would be easy to implement.
>
> Is there any desire from R-core to have add the importHiddenFrom
> functionality? If so I can add the export check and submit the patch either
> here or on bugzilla. I figure its a long shot but hey, at least I now know
> how the namespace stuff works.
>
> I do agree with Peter Meilstrup that poking around at the internals of
> someone else's code is often not a good idea, but as others have pointed
> out it is done in practice in some fairly high-profile packages, and if its
> going to happen it seems like it would be nice to indicate as much in the
> NAMESPACE file.
>
> ~G
>
>
> On Thu, Aug 22, 2013 at 5:41 PM, Gray  wrote:
>
>> Peter Meilstrup: (05:01PM on Thu, Aug 22)
>>
>>  One most often encounters namespace conflicts at the user level, when
>>> loading two packages that have no logical connection other than both
>>> bearing on your problem of the moment.
>>>
>>
>> Unless I'm mistaken, you can reassign the hidden functions, ie
>>
>> fna <- joespackage:::usefulfunction
>>
>> fnb <- janespackage:::usefulfunction
>>
>> which is a little bit of a pain, but makes the user's code
>> unambiguous.  This also works with two colons for explicitly exported
>> functions.
>>
>> --
>> Gray Calhoun, Assistant Professor of Economics at Iowa State
>> http://gray.clhn.co (web)

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel