Re: [Rd] Fortune! (Re: declaring package dependencies)
Thanks, added to the devel-version on R-Forge. Z On Fri, 13 Sep 2013, Dirk Eddelbuettel wrote: On 13 September 2013 at 08:31, Michael Friendly wrote: | Thanks for this explanation, but I'm still confused about how to avoid | the wrath of the CRAN-devel daemon, whose appetite for new morsels of | developer flesh seems ever increasing and makes keeping even a stable | package up-to-date a moving target. _Obviously_ a fortunes candidate. CCing Achim. [ And I informed my first-born that her time with us will surely be limited as I expect CRAN to demand every submitters first child any day now ... ] | Perhaps such changes in R devel should be announced on this list for public | comment before they are enforced in R CMD check. Many of us have suggested that many times before, but CRAN has the longer stick and simply refuses to play along. "Eventually" the tides will turn and some of us will be fed up enough and stop uploading. But so far we still have an (uneasy?) truce, and everybody is playing (fighting?) along. And don't even get me started on the state of r-forge build services... Dirk -- Dirk Eddelbuettel | e...@debian.org | http://dirk.eddelbuettel.com __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] declaring package dependencies
On 13-09-13 12:00 PM, Dirk Eddelbuettel wrote: On 13 September 2013 at 11:42, Paul Gilbert wrote: | On 13-09-13 11:02 AM, Dirk Eddelbuettel wrote: | > It's not so much Rcpp itself or my 20-ish packages but the fact that we (as | > in the Rcpp authors) now stand behind an API that also has to accomodate | > changes in R CMD check. Case in point is current (unannounced) change that | > makes all Depends: Rcpp become Imports: Rcpp because of the NAMESPACE checks. | | I am a bit confused by this Dirk, so maybe I am missing something. I | think this is still a "Note" in R-devel so you do have some time to make | the change, at least several months, maybe more. It is not quite what I | think of as an "announcement", more like a shot across the bow, but it | is also not "unannounced". One package author [as in user of Rcpp and not an author of it] was told by CRAN this week to change his package and came to me for help -- so in that small way the CRAN "non-communication policy" is already creating more work for me, and makes me look silly as "I don't document what Rcpp-using packages need" as I sadly still lack the time machine or psychic powers to infer what may get changed this weekend. | More importantly, I don't think that the requirement is necessarily to | change Depends: Rcpp to Imports: Rcpp, the requirement is to put | imports(Rcpp) in the NAMESPACE file. I think this is so that the package | continues to work even if the user does something with the search path. | The decision to change Depends: Rcpp to Imports: Rcpp really depends on | whether the package author wants Rcpp functions to be available directly Rcpp is a bit of an odd-ball as you mostly need it at compile-time, and you require very few R-level functions (but there is package initialization etc pp). We also only about two handful of functions, and those are for functionality not all 135 packages use (eg Modules etc). But the focus here should not be on my hobby package. The focus needs to be on how four CRAN maintainers (who do a boatload of amazing work which is _truly_ appreciated in its thoroughness and reach) could make the life of authors of 4800+ packages easier by communicating and planning a tad more. Let me paraphrase that: "The CRAN maintainers do a lot of work, and it helps me a lot, but if they only did a little bit more work it would help me even more." I suspect they'd be more receptive to suggestions that had them doing less work, not more. Duncan Murdoch | by users without them needing to specifically attach Rcpp. They are | available with Depends but with Imports they are just used internally in | the package. | | So, one of us is confused. Usually it is me. No, no, I usually keep you company. Dirk __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] declaring package dependencies
On 14 September 2013 at 09:04, Duncan Murdoch wrote: | On 13-09-13 12:00 PM, Dirk Eddelbuettel wrote: | > But the focus here should not be on my hobby package. The focus needs to be | > on how four CRAN maintainers (who do a boatload of amazing work which is | > _truly_ appreciated in its thoroughness and reach) could make the life of | > authors of 4800+ packages easier by communicating and planning a tad more. | | Let me paraphrase that: "The CRAN maintainers do a lot of work, and it | helps me a lot, but if they only did a little bit more work it would | help me even more." | | I suspect they'd be more receptive to suggestions that had them doing | less work, not more. Presumably that also holds for the authors of the 4800+ packages. But my point is not the about "less", it is about "more predictable". Currently, the system appears to be anything but. We can do better. Dirk -- Dirk Eddelbuettel | e...@debian.org | http://dirk.eddelbuettel.com __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] declaring package dependencies
On 09/14/2013 08:04 AM, Duncan Murdoch wrote: >> On 13-09-13 12:00 PM, Dirk Eddelbuettel wrote: But the focus here should not be on my hobby package. The focus needs to be on how four CRAN maintainers (who do a boatload of amazing work which is _truly_ appreciated in its thoroughness and reach) could make the life of authors of 4800+ packages easier by communicating and planning a tad more. Let me paraphrase that: "The CRAN maintainers do a lot of work, and it helps me a lot, but if they only did a little bit more work it would help me even more." I suspect they'd be more receptive to suggestions that had them doing less work, not more. I think you're both right. If the CRAN team would communicate more with this list about pending changes, package authors could make those changes before submitting. Those submissions that won't pass unannounced R-devel checks waste everybody's time, including the time of the CRAN team. Not to mention, most open source projects seem to have concluded that more eyes on pending or proposed changes make everyone's life better in the long term. Regards, Brian -- Brian G. Peterson http://braverock.com/brian/ Ph: 773-459-4973 IM: bgpbraverock __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] declaring package dependencies
On 13-09-14 09:04 AM, Duncan Murdoch wrote: On 13-09-13 12:00 PM, Dirk Eddelbuettel wrote: On 13 September 2013 at 11:42, Paul Gilbert wrote: | On 13-09-13 11:02 AM, Dirk Eddelbuettel wrote: | > It's not so much Rcpp itself or my 20-ish packages but the fact that we (as | > in the Rcpp authors) now stand behind an API that also has to accomodate | > changes in R CMD check. Case in point is current (unannounced) change that | > makes all Depends: Rcpp become Imports: Rcpp because of the NAMESPACE checks. | | I am a bit confused by this Dirk, so maybe I am missing something. I | think this is still a "Note" in R-devel so you do have some time to make | the change, at least several months, maybe more. It is not quite what I | think of as an "announcement", more like a shot across the bow, but it | is also not "unannounced". One package author [as in user of Rcpp and not an author of it] was told by CRAN this week to change his package and came to me for help -- so in that small way the CRAN "non-communication policy" is already creating more work for me, and makes me look silly as "I don't document what Rcpp-using packages need" as I sadly still lack the time machine or psychic powers to infer what may get changed this weekend. | More importantly, I don't think that the requirement is necessarily to | change Depends: Rcpp to Imports: Rcpp, the requirement is to put | imports(Rcpp) in the NAMESPACE file. I think this is so that the package | continues to work even if the user does something with the search path. | The decision to change Depends: Rcpp to Imports: Rcpp really depends on | whether the package author wants Rcpp functions to be available directly Rcpp is a bit of an odd-ball as you mostly need it at compile-time, and you require very few R-level functions (but there is package initialization etc pp). We also only about two handful of functions, and those are for functionality not all 135 packages use (eg Modules etc). But the focus here should not be on my hobby package. The focus needs to be on how four CRAN maintainers (who do a boatload of amazing work which is _truly_ appreciated in its thoroughness and reach) could make the life of authors of 4800+ packages easier by communicating and planning a tad more. Let me paraphrase that: "The CRAN maintainers do a lot of work, and it helps me a lot, but if they only did a little bit more work it would help me even more." I suspect they'd be more receptive to suggestions that had them doing less work, not more. Actually, this is one of the parts that I do not understand. It seems to me that it would be a lot less work for CRAN maintainers if the implications and necessary changes to packages were explained a bit more clearly in a forum like R-devel that many package developers actually read regularly. I many not fully understand how much of the response to package submission gets done automatically, but I do get the sense that there is a fairly large amount of actual human time spent dealing with just my submissions alone. If that is representative of all developers, then CRAN maintainers don't have time to do much else. (The fact that they do much more suggests I may not be representative.) Two specific points have already been mentioned implicitly. CRAN submission testing is often done at a higher/newer level using the latest devel version. This results in lots of rejections for things that I would fix before submission, if I knew about them. If the tests were rolled out with R, and only later incorporated into CRAN submission testing, I think there would be a lot less work for the CRAN maintainers. (This is ignoring the possibility that CRAN submission is really the testing ground for the tests, and to prove the tests requires a fair amount of manual involvement. I'm happy to continue contributing to this -- I've often felt my many contribution is an endless supply of bugs for the checkers to catch.) The second point is that a facility like R-forge that runs the latest checks, on many platforms, is really useful in order to reduce work for both package developers and CRAN maintainers. With R-forge broken, the implication for additional work for CRAN maintainers seems enormous. But even with it working, not all packages are kept on R-forge, and with package version dependencies R-forge does not really work. (i.e. I have to get new versions of some packages onto CRAN before the new versions of other packages will build on R-forge.) Perhaps the package checking part of R-forge should be separated into a pre-submission clearing house to which packages are submitted. If they pass checks there then the package developer could click on a submit button to do the actual submission to CRAN. (Of course there needs to be a mechanism to plead for the fact that the test systems do not have needed resources.) Something like the daily, but with new pre-release versions of packages might actually be better than t
Re: [Rd] declaring package dependencies
I'm a little surprised by this thread. I subscribe to the RSS feeds of changes to NEWS (as Dirk mentioned) and that's been pretty informative in the past : http://developer.r-project.org/RSSfeeds.html Mainly though, I submit to winbuilder before submitting to CRAN, as the CRAN policies advise. winbuilder's R-devel seems to be built daily, saving me the time. Since I don't have Windows it kills two birds with one stone. It has caught many problems for me before submitting to CRAN and I can't remember it ever not responding in a reasonable time. http://win-builder.r-project.org/upload.aspx I've suggested before that winbuilder could be the mechanism to submit to CRAN rather than an ftp upload to incoming. Only if winbuilder passed "OK" on R-devel could it then go to a human. But iirc there was a technical difficulty preventing this. Matthew [[alternative HTML version deleted]] __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] declaring package dependencies
On 13-09-14 9:14 AM, Dirk Eddelbuettel wrote: On 14 September 2013 at 09:04, Duncan Murdoch wrote: | On 13-09-13 12:00 PM, Dirk Eddelbuettel wrote: | > But the focus here should not be on my hobby package. The focus needs to be | > on how four CRAN maintainers (who do a boatload of amazing work which is | > _truly_ appreciated in its thoroughness and reach) could make the life of | > authors of 4800+ packages easier by communicating and planning a tad more. | | Let me paraphrase that: "The CRAN maintainers do a lot of work, and it | helps me a lot, but if they only did a little bit more work it would | help me even more." | | I suspect they'd be more receptive to suggestions that had them doing | less work, not more. Presumably that also holds for the authors of the 4800+ packages. No, I don't think it does. What benefit do the CRAN maintainers get from them? But my point is not the about "less", it is about "more predictable". Currently, the system appears to be anything but. We can do better. Then go ahead. Do better. Duncan Murdoch __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] declaring package dependencies
On 13-09-14 9:16 AM, Brian G. Peterson wrote: On 09/14/2013 08:04 AM, Duncan Murdoch wrote: >> On 13-09-13 12:00 PM, Dirk Eddelbuettel wrote: But the focus here should not be on my hobby package. The focus needs to be on how four CRAN maintainers (who do a boatload of amazing work which is _truly_ appreciated in its thoroughness and reach) could make the life of authors of 4800+ packages easier by communicating and planning a tad more. Let me paraphrase that: "The CRAN maintainers do a lot of work, and it helps me a lot, but if they only did a little bit more work it would help me even more." I suspect they'd be more receptive to suggestions that had them doing less work, not more. I think you're both right. If the CRAN team would communicate more with this list about pending changes, package authors could make those changes before submitting. Those submissions that won't pass unannounced R-devel checks waste everybody's time, including the time of the CRAN team. Paraphrasing again: "If the CRAN team would do this, then they would waste less of everyone's time." I think you should assume that the CRAN team doesn't choose to waste it's own time, so in their judgment, doing more on this list would waste more of its time. Not to mention, most open source projects seem to have concluded that more eyes on pending or proposed changes make everyone's life better in the long term. There are no secrets in R. You are perfectly free to run your package against exactly the same version the CRAN team is using. If you choose not to do so, you're the one wasting everyone's time. Duncan Murdoch __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] declaring package dependencies
On 13-09-14 12:19 PM, Paul Gilbert wrote: On 13-09-14 09:04 AM, Duncan Murdoch wrote: On 13-09-13 12:00 PM, Dirk Eddelbuettel wrote: On 13 September 2013 at 11:42, Paul Gilbert wrote: | On 13-09-13 11:02 AM, Dirk Eddelbuettel wrote: | > It's not so much Rcpp itself or my 20-ish packages but the fact that we (as | > in the Rcpp authors) now stand behind an API that also has to accomodate | > changes in R CMD check. Case in point is current (unannounced) change that | > makes all Depends: Rcpp become Imports: Rcpp because of the NAMESPACE checks. | | I am a bit confused by this Dirk, so maybe I am missing something. I | think this is still a "Note" in R-devel so you do have some time to make | the change, at least several months, maybe more. It is not quite what I | think of as an "announcement", more like a shot across the bow, but it | is also not "unannounced". One package author [as in user of Rcpp and not an author of it] was told by CRAN this week to change his package and came to me for help -- so in that small way the CRAN "non-communication policy" is already creating more work for me, and makes me look silly as "I don't document what Rcpp-using packages need" as I sadly still lack the time machine or psychic powers to infer what may get changed this weekend. | More importantly, I don't think that the requirement is necessarily to | change Depends: Rcpp to Imports: Rcpp, the requirement is to put | imports(Rcpp) in the NAMESPACE file. I think this is so that the package | continues to work even if the user does something with the search path. | The decision to change Depends: Rcpp to Imports: Rcpp really depends on | whether the package author wants Rcpp functions to be available directly Rcpp is a bit of an odd-ball as you mostly need it at compile-time, and you require very few R-level functions (but there is package initialization etc pp). We also only about two handful of functions, and those are for functionality not all 135 packages use (eg Modules etc). But the focus here should not be on my hobby package. The focus needs to be on how four CRAN maintainers (who do a boatload of amazing work which is _truly_ appreciated in its thoroughness and reach) could make the life of authors of 4800+ packages easier by communicating and planning a tad more. Let me paraphrase that: "The CRAN maintainers do a lot of work, and it helps me a lot, but if they only did a little bit more work it would help me even more." I suspect they'd be more receptive to suggestions that had them doing less work, not more. Actually, this is one of the parts that I do not understand. It seems to me that it would be a lot less work for CRAN maintainers if the implications and necessary changes to packages were explained a bit more clearly in a forum like R-devel that many package developers actually read regularly. Then why don't you explain them? They aren't secret. I many not fully understand how much of the response to package submission gets done automatically, but I do get the sense that there is a fairly large amount of actual human time spent dealing with just my submissions alone. If that is representative of all developers, then CRAN maintainers don't have time to do much else. (The fact that they do much more suggests I may not be representative.) Two specific points have already been mentioned implicitly. CRAN submission testing is often done at a higher/newer level using the latest devel version. This results in lots of rejections for things that I would fix before submission, if I knew about them. Then why don't you test against R-devel before submitting? If the tests were rolled out with R, and only later incorporated into CRAN submission testing, I think there would be a lot less work for the CRAN maintainers. I am an R core member, not a member of CRAN. If I make a change to R, I'd like to know the effect it has on the corpus of CRAN and Bioconductor packages, but I don't have the computing resources to run it against all of those in a reasonable amount of time. We have heard from various corporate users of R (Revolution Computing and Google are two) that they would like to contribute to the project, but we don't actually see useful contributions from them. I don't know if you personally have influence with anyone who has such resources, but surely some on this list do. So why don't they write to me and tell me how to use their computing resources to run proposed changes against all published packages? I think it's because it's easier to do nothing than to do something. It's certainly easier to say that it's the CRAN maintainers' fault for poor communication than it is to identify and address the problems. Duncan Murdoch (This is ignoring the possibility that CRAN submission is really the testing ground for the tests, and to prove the tests requires a fair amount of manual involvement. I'm happy to continue contributing to this -- I've often felt my many
Re: [Rd] declaring package dependencies
On 14 September 2013 at 19:01, Duncan Murdoch wrote: | On 13-09-14 9:14 AM, Dirk Eddelbuettel wrote: | > | > On 14 September 2013 at 09:04, Duncan Murdoch wrote: | > | On 13-09-13 12:00 PM, Dirk Eddelbuettel wrote: | > | > But the focus here should not be on my hobby package. The focus needs to be | > | > on how four CRAN maintainers (who do a boatload of amazing work which is | > | > _truly_ appreciated in its thoroughness and reach) could make the life of | > | > authors of 4800+ packages easier by communicating and planning a tad more. | > | | > | Let me paraphrase that: "The CRAN maintainers do a lot of work, and it | > | helps me a lot, but if they only did a little bit more work it would | > | help me even more." | > | | > | I suspect they'd be more receptive to suggestions that had them doing | > | less work, not more. | > | > Presumably that also holds for the authors of the 4800+ packages. | | No, I don't think it does. What benefit do the CRAN maintainers get | from them? As explained by others in the thread, if forthcoming were more clearly communicated, package authors would be prepared and the CRAN maintainers would not have to explain it one-by-one as changes are implemented with no forewarning. | > But my point is not the about "less", it is about "more predictable". | > Currently, the system appears to be anything but. We can do better. | | Then go ahead. Do better. I am trying, eg via this discussion. But you have now managed to sap all energy I may have had. I am done here. Dirk -- Dirk Eddelbuettel | e...@debian.org | http://dirk.eddelbuettel.com __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
