Re: [Rd] small issue with over-zealous clean.
> --- On Sun, 9/12/12, Dirk Eddelbuettel wrote: > On 9 December 2012 at 18:17, Hin-Tak Leung wrote: > | Noticed a problem for a while - tests/testit.Rd, > tests/ver20.Rd are removed on "make clean" unintentionally. > | > | This seems to come from a change in tests/Makefile.in, which adds the line: > | -@rm -f *.tar.gz *.Rd back in May 2012. > | > | --- > | commit c4d70254e7b7f9d7ed17faecfb3097195d852ddc > | Author: ripley > | Date: Sun May 27 09:04:41 2012 + > | > | fix some issues seen by examining no-segfaults.Rout > | [. dropped . embarrassing / amusing distraction on svn/git . . started by the OP, not by Dirk ! ] > That said ..a small bug was introduced back in May . Yes, you are right. BTW: The reason that nobody (from R core, probably not many people otherwise) has found/mentioned this bug before is not the use of svn, but the fact that it is much more convenient (and hence somewhat recommended) to build R outside of its source directory, and in that case the two *.Rd files that belong to ./tests/ are not removed (from the *build* directory's ./tests/) Martin __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] small issue with over-zealous clean.
--- On Mon, 10/12/12, Martin Maechler wrote:
> > --- On Sun, 9/12/12, Dirk
> Eddelbuettel
> wrote:
>
> > On 9 December 2012 at 18:17, Hin-Tak Leung wrote:
> > | Noticed a problem for a while - tests/testit.Rd,
> > tests/ver20.Rd are removed on "make clean"
> unintentionally.
> > |
> > | This seems to come from a change in
> tests/Makefile.in, which adds the line:
> > | -@rm -f *.tar.gz *.Rd back in May 2012.
> > |
> > | ---
> > | commit c4d70254e7b7f9d7ed17faecfb3097195d852ddc
> > | Author: ripley
>
> > | Date: Sun May 27 09:04:41 2012 +
> > |
> > | fix some issues seen by examining
> no-segfaults.Rout
> > |
>
> [. dropped
> . embarrassing / amusing distraction on svn/git .
> . started by the OP, not by Dirk !
> ]
That comment is a bit childish, don't you think?
> > That said ..a small bug was introduced back in
> May .
>
> Yes, you are right.
> BTW: The reason that nobody (from R core, probably not many
> people otherwise) has found/mentioned this bug before is not
> the
> use of svn, but the fact that it is much more convenient
> (and
> hence somewhat recommended) to build R outside of its
> source
> directory, and in that case the two *.Rd files that belong
> to
> ./tests/ are not removed (from the *build* directory's
> ./tests/)
"more convenient" is a subjective matter.
As I mentioned in my original post, I have a few local modifications which are
continually re-applied ("rebase"d, but I shall not be drawn into arguing about
matters of personal preference again) - therefore it is more convenient to
build on top.
Since we are on the topic of locally-continually applied modifications, I
reported another issue about 40 days ago, that reccent R trunk now treat
revision as numeric, so 'unknown' in the topic level Makefile.in should be
changed accordingly to 0 or some number. Here is the diff - one of the
"locally-continually applied modifications" I am talking about:
--- a/Makefile.in
+++ b/Makefile.in
@@ -94,7 +94,7 @@ svnonly:
@if test ! -f "$(srcdir)/doc/FAQ" || test -f non-tarball ; then \
(cd doc/manual && $(MAKE) front-matter html-non-svn) ; \
touch non-tarball ; \
- (cd $(srcdir); LC_ALL=C TZ=GMT svn info || $(ECHO) "Revision:
unknown") 2> /dev/null \
+ (cd $(srcdir); LC_ALL=C TZ=GMT svn info || $(ECHO) "Revision: 0") 2>
/dev/null \
| sed -n -e '/^Revision/p' -e '/^Last Changed Date/'p \
| cut -d' ' -f1,2,3,4 > SVN-REVISION-tmp ; \
$(SHELL) $(top_srcdir)/tools/move-if-change SVN-REVISION-tmp
SVN-REVISION ; \
__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] small issue with over-zealous clean.
> Hin-Tak Leung
> on Mon, 10 Dec 2012 09:23:14 + writes:
> --- On Mon, 10/12/12, Martin Maechler wrote:
[.]
>> > That said ..a small bug was introduced back in
>> May .
>>
>> Yes, you are right.
>> BTW: The reason that nobody (from R core, probably not many
>> people otherwise) has found/mentioned this bug before is not
>> the
>> use of svn, but the fact that it is much more convenient
>> (and
>> hence somewhat recommended) to build R outside of its
>> source
>> directory, and in that case the two *.Rd files that belong
>> to
>> ./tests/ are not removed (from the *build* directory's
>> ./tests/)
> "more convenient" is a subjective matter.
> As I mentioned in my original post, I have a few local modifications
which are continually re-applied ("rebase"d, but I shall not be drawn into
arguing about matters of personal preference again) - therefore it is more
convenient to build on top.
> Since we are on the topic of locally-continually applied modifications, I
reported another issue about 40 days ago, that reccent R trunk now treat
revision as numeric, so 'unknown' in the topic level Makefile.in should be
changed accordingly to 0 or some number. Here is the diff - one of the
"locally-continually applied modifications" I am talking about:
> --- a/Makefile.in
> +++ b/Makefile.in
> @@ -94,7 +94,7 @@ svnonly:
> @if test ! -f "$(srcdir)/doc/FAQ" || test -f non-tarball ; then \
> (cd doc/manual && $(MAKE) front-matter html-non-svn) ; \
> touch non-tarball ; \
> - (cd $(srcdir); LC_ALL=C TZ=GMT svn info || $(ECHO) "Revision:
unknown") 2> /dev/null \
> + (cd $(srcdir); LC_ALL=C TZ=GMT svn info || $(ECHO) "Revision:
0") 2> /dev/null \
> | sed -n -e '/^Revision/p' -e '/^Last Changed Date/'p \
> | cut -d' ' -f1,2,3,4 > SVN-REVISION-tmp ; \
> $(SHELL) $(top_srcdir)/tools/move-if-change SVN-REVISION-tmp SVN-REVISION
; \
That change needs another important change in src/main/version.c
where the string "unknown" has been explicitly looked for.
I have now committed a patch to both ---
using '-99' : something clearly "artificial", rather than '0'
which looks too innocuous.
Martin
__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] How to locate a file quickly
> Wincent > on Mon, 10 Dec 2012 15:15:34 +0800 writes: > Dear all, > During the development of R packages, more and more functions are written > and some of them are placed in a file. After a while, I want to revised a > function, but forget which file the function is in. Is there a recommended > way of locate a function quickly? why do you think this belongs to R-devel ? Please use R-help for such questions. Best regards, Martin __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
[Rd] CMD check and .Rbuildignore
I often run R CMD check on my package source. I've noted of late that this process gives warning messages about files listed in my .Rbuildignore file, where is once ignored these. Terry T. __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] small issue with over-zealous clean.
--- On Mon, 10/12/12, Martin Maechler wrote:
> > Hin-Tak Leung
>
> > on Mon, 10 Dec
> 2012 09:23:14 + writes:
>
> > --- On Mon, 10/12/12, Martin Maechler
>
> wrote:
> [.]
>
> >> > That said ..a small bug was
> introduced back in
> >> May .
> >>
> >> Yes, you are right.
> >> BTW: The reason that nobody (from R
> core, probably not many
> >> people otherwise) has found/mentioned
> this bug before is not
> >> the
> >> use of svn, but the fact that it is
> much more convenient
> >> (and
> >> hence somewhat recommended) to build
> R outside of its
> >> source
> >> directory, and in that case the two
> *.Rd files that belong
> >> to
> >> ./tests/ are not removed (from the
> *build* directory's
> >> ./tests/)
>
> > "more convenient" is a subjective
> matter.
>
> > As I mentioned in my original post, I
> have a few local modifications which are continually
> re-applied ("rebase"d, but I shall not be drawn into arguing
> about matters of personal preference again) - therefore it
> is more convenient to build on top.
>
> > Since we are on the topic of
> locally-continually applied modifications, I reported
> another issue about 40 days ago, that reccent R trunk now
> treat revision as numeric, so 'unknown' in the topic level
> Makefile.in should be changed accordingly to 0 or some
> number. Here is the diff - one of the "locally-continually
> applied modifications" I am talking about:
>
> > --- a/Makefile.in
> > +++ b/Makefile.in
> > @@ -94,7 +94,7 @@ svnonly:
> > @if test ! -f "$(srcdir)/doc/FAQ" || test
> -f non-tarball ; then \
> > (cd doc/manual && $(MAKE)
> front-matter html-non-svn) ; \
> > touch non-tarball ; \
> > -
> (cd $(srcdir); LC_ALL=C TZ=GMT svn info ||
> $(ECHO) "Revision: unknown") 2> /dev/null \
> > +
> (cd $(srcdir); LC_ALL=C TZ=GMT svn info ||
> $(ECHO) "Revision: 0") 2> /dev/null \
> > | sed -n -e '/^Revision/p' -e '/^Last
> Changed Date/'p \
> > | cut -d' ' -f1,2,3,4 >
> SVN-REVISION-tmp ; \
> > $(SHELL)
> $(top_srcdir)/tools/move-if-change SVN-REVISION-tmp
> SVN-REVISION ; \
>
> That change needs another important change in
> src/main/version.c
> where the string "unknown" has been explicitly looked for.
>
> I have now committed a patch to both ---
> using '-99' : something clearly "artificial", rather than
> '0'
> which looks too innocuous.
Thanks.
Genuine revision number starts from 1, so 0 does the minimal job of indicating
it is not genuine. I can't remember what was the precise problem/symptom,
possibly R showing a very ugly error message when it starts, trying to display
some basic version info.
I have 7 such "continually locally applied modifications" at the moment - the
oldest two dated back 14 months ago. The other 6 are something which are
temporarily useful, or at least harmless, but not obviously correct way (or
"one of the obviously correct alternatives") of doing things.
I certainly find that being able to carry "continually locally applied
modifications", somewhat indefinitely and privately, from 40 days old to 14
months old, while tracking and incorporating upstream changes, and continue to
make and track local changes, a useful feature of distributed version control
systems in general, versus centralized version control systems. The 7th mod
would have been either automatically dropped (if the up-stream applies the
exact same change, which is 40 days later), or *offered* to be dropped/modified
in this case when upstream applies a different change to the same code area.
BTW, the current version of the recommended library Matrix has not been able to
build since early September. (over 3 months ago). I assume that it will be
fixed eventually, or before R 16, in the next few months.
__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] Changing arguments inside .Call. Wise to encourage "const" on all arguments?
On Dec 10, 2012, at 1:51 AM, Paul Johnson wrote:
> I'm continuing my work on finding speedups in generalized inverse
> calculations in some simulations. It leads me back to .C and .Call,
> and some questions I've never been able to answer for myself. It may
> be I can push some calculations to LAPACK in or C BLAS, that's why I
> realized again I don't understand the call by reference or value
> semantics of .Call
>
> Why aren't users of .Call encouraged to "const" their arguments, and
> why doesn't .Call do this for them (if we really believe in return by
> value)?
>
Because there is a difference between the *data* part of the SEXP and the
object itself. Internal structure of the object may need to be modified (e.g.
the NAMED ref counting increased when you assign it) in a call to R API. You
can't flag the data part as const separately, so you have to use non-const SEXP.
> R Gentleman's R Programming for Bioinformatics is the most
> understandable treatment I've found on .Call. It appears to me .Call
> leaves "wiggle room" where there should be none. Here's Gentleman on
> p. 185. "For .Call and .External, the return value is an R object (the
> C functions must return a SEXP), and for these functions the values
> that were passed are typically not modified. If they must be
> modified, then making a copy in R, prior to invoking the C code, is
> necessary."
>
> I *think* that means:
>
> .Call allows return by reference, BUT we really wish users would not
> use it. Users can damage R data structures that are pointed to unless
> they really truly know what they are doing on the C side. ??
>
> This seems dangerous. Why allow return by reference at all?
>
Because it is completely legal to do things like
SEXP last(SEXP bar) {
if (TYPEOF(bar) = VECSXP && LENGTH(bar) > 0)
return VECTOR_ELT(bar, LENGTH(bar) - 1);
Rf_error("sorry, I only work on lists");
}
There is no point in duplicating the element.
> On p. 197, there's a similar comment "Any function that has been
> invoked by either .External or .Call will have all of its arguments
> protected already. You do not need to protect them. [T]hey were
> not duplicated and should be treated as read-only values."
>
> "should be ... read-only" concerns me. They are "protected" in the
> garbage collector sense,
Yes
> but they are not protected from "return by
> reference" damage. Right?
>
There is no "return by reference damage".
The only problem is if you modify input arguments while someone else holds a
reference, but there is no way in C to prevent that while still allowing them
to be useful. Note that it is legal to modify input arguments if there are no
references to it.
Cheers,
Simon
> Why doesn't the documentation recommend function writers to mark
> arguments to C functions as const? Isn't that what the return by
> value policy would suggest?
>
> Here's a troublesome example in R src/main/array.c:
>
> /* DropDims strips away redundant dimensioning information. */
> /* If there is an appropriate dimnames attribute the correct */
> /* element is extracted and attached to the vector as a names */
> /* attribute. Note that this function mutates x. */
> /* Duplication should occur before this is called. */
>
> SEXP DropDims(SEXP x)
> {
>SEXP dims, dimnames, newnames = R_NilValue;
>int i, n, ndims;
>
> PROTECT(x);
> dims = getAttrib(x, R_DimSymbol);
> [... SNIP ]
>setAttrib(x, R_DimNamesSymbol, R_NilValue);
>setAttrib(x, R_DimSymbol, R_NilValue);
>setAttrib(x, R_NamesSymbol, newnames);
> [... SNIP ]
>
> return x;
> }
>
> Well, at least there's a warning comment with that one. But it does
> show .Call allows call by reference.
>
> Why allow it, though? DropDims could copy x, modify the copy, and return it.
>
> I wondered why DropDims bothers to return x at all. We seem to be
> using modify and return by reference there.
>
> I also wondered why x is PROTECTED, actually. Its an argument, wasn't
> it automatically protected? Is it no longer protected after the
> function starts modifying it?
>
> Here's an example with similar usage in Writing R Extensions, section
> 5.10.1 "Calling .Call". It protects the arguments a and b (needed
> ??), then changes them.
>
> #include
> #include
>
> SEXP convolve2(SEXP a, SEXP b)
> {
> R_len_t i, j, na, nb, nab;
> double *xa, *xb, *xab;
> SEXP ab;
>
> PROTECT(a = AS_NUMERIC(a)); /* PJ wonders, doesn't this alter
> "a" in calling code*/
> PROTECT(b = AS_NUMERIC(b));
> na = LENGTH(a); nb = LENGTH(b); nab = na + nb - 1;
> PROTECT(ab = NEW_NUMERIC(nab));
> xa = NUMERIC_POINTER(a); xb = NUMERIC_POINTER(b);
> xab = NUMERIC_POINTER(ab);
> for(i = 0; i < nab; i++) xab[i] = 0.0;
> for(i = 0; i < na; i++)
> for(j = 0; j < nb; j++) xab[i + j] += xa[i] * xb[j];
> UNPROTECT(3);
> return(ab);
> }
>
>
> Doesn't
>
>
Re: [Rd] Changing arguments inside .Call. Wise to encourage "const" on all arguments?
On Dec 10, 2012, at 2:05 PM, Simon Urbanek wrote:
>
> On Dec 10, 2012, at 1:51 AM, Paul Johnson wrote:
>
>> I'm continuing my work on finding speedups in generalized inverse
>> calculations in some simulations. It leads me back to .C and .Call,
>> and some questions I've never been able to answer for myself. It may
>> be I can push some calculations to LAPACK in or C BLAS, that's why I
>> realized again I don't understand the call by reference or value
>> semantics of .Call
>>
>> Why aren't users of .Call encouraged to "const" their arguments, and
>> why doesn't .Call do this for them (if we really believe in return by
>> value)?
>>
>
> Because there is a difference between the *data* part of the SEXP and the
> object itself. Internal structure of the object may need to be modified (e.g.
> the NAMED ref counting increased when you assign it) in a call to R API. You
> can't flag the data part as const separately, so you have to use non-const
> SEXP.
>
>
>> R Gentleman's R Programming for Bioinformatics is the most
>> understandable treatment I've found on .Call. It appears to me .Call
>> leaves "wiggle room" where there should be none. Here's Gentleman on
>> p. 185. "For .Call and .External, the return value is an R object (the
>> C functions must return a SEXP), and for these functions the values
>> that were passed are typically not modified. If they must be
>> modified, then making a copy in R, prior to invoking the C code, is
>> necessary."
>>
>> I *think* that means:
>>
>> .Call allows return by reference, BUT we really wish users would not
>> use it. Users can damage R data structures that are pointed to unless
>> they really truly know what they are doing on the C side. ??
>>
>> This seems dangerous. Why allow return by reference at all?
>>
>
> Because it is completely legal to do things like
>
> SEXP last(SEXP bar) {
> if (TYPEOF(bar) = VECSXP && LENGTH(bar) > 0)
> return VECTOR_ELT(bar, LENGTH(bar) - 1);
> Rf_error("sorry, I only work on lists");
> }
>
Martin Morgan pointed out that this example is a bad one -- which is true. The
common idiom that is safe is
SEXP foo(SEXP bar) {
...
return bar;
}
However, the last() example above is bad, because returning the element
directly is a bad idea -- the conservative approach would be to use
duplicate(), the more efficient one would be to bump up NAMED. Sorry, my bad. I
guess I was rather strengthening Paul's point to duplicate() when in doubt even
if it's less efficient :).
Cheers,
Simon
> There is no point in duplicating the element.
>
>
>
>> On p. 197, there's a similar comment "Any function that has been
>> invoked by either .External or .Call will have all of its arguments
>> protected already. You do not need to protect them. [T]hey were
>> not duplicated and should be treated as read-only values."
>>
>> "should be ... read-only" concerns me. They are "protected" in the
>> garbage collector sense,
>
> Yes
>
>
>> but they are not protected from "return by
>> reference" damage. Right?
>>
>
> There is no "return by reference damage".
>
> The only problem is if you modify input arguments while someone else holds a
> reference, but there is no way in C to prevent that while still allowing them
> to be useful. Note that it is legal to modify input arguments if there are no
> references to it.
>
> Cheers,
> Simon
>
>
>> Why doesn't the documentation recommend function writers to mark
>> arguments to C functions as const? Isn't that what the return by
>> value policy would suggest?
>>
>> Here's a troublesome example in R src/main/array.c:
>>
>> /* DropDims strips away redundant dimensioning information. */
>> /* If there is an appropriate dimnames attribute the correct */
>> /* element is extracted and attached to the vector as a names */
>> /* attribute. Note that this function mutates x. */
>> /* Duplication should occur before this is called. */
>>
>> SEXP DropDims(SEXP x)
>> {
>> SEXP dims, dimnames, newnames = R_NilValue;
>> int i, n, ndims;
>>
>> PROTECT(x);
>> dims = getAttrib(x, R_DimSymbol);
>> [... SNIP ]
>> setAttrib(x, R_DimNamesSymbol, R_NilValue);
>> setAttrib(x, R_DimSymbol, R_NilValue);
>> setAttrib(x, R_NamesSymbol, newnames);
>> [... SNIP ]
>>
>> return x;
>> }
>>
>> Well, at least there's a warning comment with that one. But it does
>> show .Call allows call by reference.
>>
>> Why allow it, though? DropDims could copy x, modify the copy, and return it.
>>
>> I wondered why DropDims bothers to return x at all. We seem to be
>> using modify and return by reference there.
>>
>> I also wondered why x is PROTECTED, actually. Its an argument, wasn't
>> it automatically protected? Is it no longer protected after the
>> function starts modifying it?
>>
>> Here's an example with similar usage in Writing R Extensions, section
>> 5.10.1 "Calling .Call". It protects the arguments a and b (needed
>> ??), then changes them.
>>
>> #include
>> #include
[Rd] Bug in mclapply?
I've been using mclapply and have encountered situations where it gives errors or returns incorrect results. Here's a minimal example, which gives the error on R 2.15.2 on Mac and Linux: library(parallel) f <- function(x) NULL mclapply(1, f, mc.preschedule = FALSE, mc.cores = 1) # Error in sum(sapply(res, inherits, "try-error")) : # invalid 'type' (list) of argument I believe it happens when the following are true: - The function returns NULL - mc.preschedule = FALSE - mc.cores >= length of the input data Here are some examples I used to trace down the problem. library(parallel) f <- function(x) NULL # Error when mc.preschedule=FALSE and mc.cores >= length(x) mclapply(1, f, mc.preschedule = FALSE, mc.cores = 1)# Error mclapply(1, f, mc.preschedule = FALSE, mc.cores = 2)# Error mclapply(1:2, f, mc.preschedule = FALSE, mc.cores = 1) # OK # In the following 2 cases, I get an error about 10-20% of the time. # The other times, the result is worse: it returns a list with only one # element, not two! mclapply(1:2, f, mc.preschedule = FALSE, mc.cores = 2) # Error mclapply(1:2, f, mc.preschedule = FALSE, mc.cores = 3) # Error # When mc.preschedule=TRUE, always works mclapply(1, f, mc.preschedule = TRUE, mc.cores = 1)# OK mclapply(1:2, f, mc.preschedule = TRUE, mc.cores = 1) # OK mclapply(1:2, f, mc.preschedule = TRUE, mc.cores = 2) # OK # lapply() always works lapply(1, f)# OK lapply(1:2, f) # OK lapply(1:2, f) # OK # If function returns non-null, it works g <- function(x) 0 mclapply(1, g, mc.preschedule = FALSE, mc.cores = 1)# OK mclapply(1:2, g, mc.preschedule = FALSE, mc.cores = 1) # OK mclapply(1:2, g, mc.preschedule = FALSE, mc.cores = 2) # OK Digging around in mclapply(), I think it happens because mccollect(jobs) is returning an empty list. But when I use options(error=recover) and debug the function, I find that when I call mccollect(jobs) again, it returns a list with values -- it's as though mccollect() is returning too early. This will illustrate: > mclapply(1, f, mc.preschedule = FALSE, mc.cores = 1) Error in sum(sapply(res, inherits, "try-error")) : invalid 'type' (list) of argument Enter a frame number, or 0 to exit 1: mclapply(1, f, mc.preschedule = FALSE, mc.cores = 1) Selection: 1 Called from: top level Browse[1]> res named list() Browse[1]> res <- mccollect(jobs) Browse[1]> res $`12348` NULL The error happens on line 63 of mclapply.r, which is after `res <- mccollect(jobs)` is called, on line 61. At this point, res should be a named list with values filled in, but it's empty. When I run `res <- mccollect(jobs)` again, it gives the correct values. Is there a good way to work around this issue for now? -Winston [[alternative HTML version deleted]] __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
