[Rd] Deep copy of factor levels?
Hi It seems that selecting an element of a factor will copy its levels (Ubuntu 13.04, R 3.0.2). Below is the output of a script that creates a factor with 1 elements and then calls as.list() on it. The new object seems to use more than 700 MB, and inspection of the levels of the individual elements of the list suggest that they are distinct objects. Perhaps some performance gain could be achieved by copying the levels "by reference", but I don't know R internals well enough to see if it's possible. Is there a particular reason for creating a full copy of the factor levels? This has come up when looking at the performance of rbind.fill (in the plyr package) with factors: https://github.com/hadley/plyr/issues/206 . Best regards Kirill > gc() used (Mb) gc trigger (Mb) max used (Mb) Ncells 325977 17.51074393 57.4 10049951 536.8 Vcells 4617168 35.3 87439742 667.2 204862160 1563.0 > system.time(x <- factor(seq_len(1e4))) user system elapsed 0.008 0.000 0.007 > system.time(xx <- as.list(x)) user system elapsed 4.263 0.000 4.322 > gc() used (Mb) gc trigger (Mb) max used (Mb) Ncells385991 20.71074393 57.4 10049951 536.8 Vcells 104672187 798.6 112367694 857.3 204862160 1563.0 > .Internal(inspect(levels(xx[[1]]))) @387f620 16 STRSXP g1c7 [MARK,NAM(2)] (len=1, tl=0) @144da4e8 09 CHARSXP g1c1 [MARK,gp=0x60] [ASCII] [cached] "1" @144da518 09 CHARSXP g1c1 [MARK,gp=0x60] [ASCII] [cached] "2" @27d1298 09 CHARSXP g1c1 [MARK,gp=0x60] [ASCII] [cached] "3" @144da548 09 CHARSXP g1c1 [MARK,gp=0x60] [ASCII] [cached] "4" @144da578 09 CHARSXP g1c1 [MARK,gp=0x60] [ASCII] [cached] "5" ... > .Internal(inspect(levels(xx[[2]]))) @1b38cb90 16 STRSXP g1c7 [MARK,NAM(2)] (len=1, tl=0) @144da4e8 09 CHARSXP g1c1 [MARK,gp=0x60] [ASCII] [cached] "1" @144da518 09 CHARSXP g1c1 [MARK,gp=0x60] [ASCII] [cached] "2" @27d1298 09 CHARSXP g1c1 [MARK,gp=0x60] [ASCII] [cached] "3" @144da548 09 CHARSXP g1c1 [MARK,gp=0x60] [ASCII] [cached] "4" @144da578 09 CHARSXP g1c1 [MARK,gp=0x60] [ASCII] [cached] "5" ... __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] Deep copy of factor levels?
Do use current R (3.1.0 alpha at present: 3.0.2 is obsolete) before reporting such things. I think you will see that this has changed On 17/03/2014 09:13, Kirill Müller wrote: Hi It seems that selecting an element of a factor will copy its levels (Ubuntu 13.04, R 3.0.2). Below is the output of a script that creates a factor with 1 elements and then calls as.list() on it. The new object seems to use more than 700 MB, and inspection of the levels of the individual elements of the list suggest that they are distinct objects. Perhaps some performance gain could be achieved by copying the levels "by reference", but I don't know R internals well enough to see if it's possible. Is there a particular reason for creating a full copy of the factor levels? This has come up when looking at the performance of rbind.fill (in the plyr package) with factors: https://github.com/hadley/plyr/issues/206 . Best regards Kirill > gc() used (Mb) gc trigger (Mb) max used (Mb) Ncells 325977 17.51074393 57.4 10049951 536.8 Vcells 4617168 35.3 87439742 667.2 204862160 1563.0 > system.time(x <- factor(seq_len(1e4))) user system elapsed 0.008 0.000 0.007 > system.time(xx <- as.list(x)) user system elapsed 4.263 0.000 4.322 > gc() used (Mb) gc trigger (Mb) max used (Mb) Ncells385991 20.71074393 57.4 10049951 536.8 Vcells 104672187 798.6 112367694 857.3 204862160 1563.0 > .Internal(inspect(levels(xx[[1]]))) @387f620 16 STRSXP g1c7 [MARK,NAM(2)] (len=1, tl=0) @144da4e8 09 CHARSXP g1c1 [MARK,gp=0x60] [ASCII] [cached] "1" @144da518 09 CHARSXP g1c1 [MARK,gp=0x60] [ASCII] [cached] "2" @27d1298 09 CHARSXP g1c1 [MARK,gp=0x60] [ASCII] [cached] "3" @144da548 09 CHARSXP g1c1 [MARK,gp=0x60] [ASCII] [cached] "4" @144da578 09 CHARSXP g1c1 [MARK,gp=0x60] [ASCII] [cached] "5" ... > .Internal(inspect(levels(xx[[2]]))) @1b38cb90 16 STRSXP g1c7 [MARK,NAM(2)] (len=1, tl=0) @144da4e8 09 CHARSXP g1c1 [MARK,gp=0x60] [ASCII] [cached] "1" @144da518 09 CHARSXP g1c1 [MARK,gp=0x60] [ASCII] [cached] "2" @27d1298 09 CHARSXP g1c1 [MARK,gp=0x60] [ASCII] [cached] "3" @144da548 09 CHARSXP g1c1 [MARK,gp=0x60] [ASCII] [cached] "4" @144da578 09 CHARSXP g1c1 [MARK,gp=0x60] [ASCII] [cached] "5" ... __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel -- Brian D. Ripley, rip...@stats.ox.ac.uk Professor of Applied Statistics, http://www.stats.ox.ac.uk/~ripley/ University of Oxford, Tel: +44 1865 272861 (self) 1 South Parks Road, +44 1865 272866 (PA) Oxford OX1 3TG, UKFax: +44 1865 272595 __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] How to convert time_t to R date object
On 16 March 2014 at 22:24, Sandip Nandi wrote:
| Hi Bill ,
|
| The following C code may help you , time_t is typedef to long int
|
| SEXP getTime() {
|
| time_t current_time;
| char* c_time_string;
| current_time = time(NULL);
| c_time_string = ctime(¤t_time);
|
| return mkString(c_time_String); // or if you want to return as int vector
| return scalarInt(current_time);
| }
|
| If you feel anything wrong , I will be very happy to know .
That is a) time and not date, and b) formatted as a string.
Dirk
| Thanks,
| Sandip
|
|
| On Sun, Mar 16, 2014 at 9:54 PM, Bill Wang wrote:
|
| Hi Dirk,
|
| Thanks for your reply, I neede convert time_t to R type in C code, can not
| use Rcpp. Maybe Rcpp source code could help me.
|
| Cheers,
| Bill
|
|
| 2014-03-16 22:55 GMT+08:00 Dirk Eddelbuettel :
|
| >
| > On 16 March 2014 at 18:36, Bill Wang wrote:
| > | I am writing a R extensions, and I need pass time_t to R in C, but I
| > don't
| > | know how to do.
| > | Can you give me some help? do not use double directly.
| >
| > Just treat it as an int:
| >
| > R> library(Rcpp)
| > R> cppFunction("Date time_t2date(time_t what) { return((int) what); }
| ")
| > R> time_t2date(0)
| > [1] "1970-01-01"
| > R> time_t2date( Sys.Date() )
| > [1] "2014-03-16"
| > R>
| >
| >
| > Here I use Rcpp to define the 'time_t2date' function on the fly.
| >
| > It takes the time_t and returns a Date type (which here is a C++ Date
| type
| > mapping to the R Date -- you can ignore that, but will have to write the
| > legwork yourself if you don't use Rcpp).
| >
| > As 'time_t' is 'long int' on my system, so I cast it to int. The rest is
| > automagic (thanks to RcpP).
| >
| > Notice that I also get today's date down and up correctly.
| >
| > See 'Writing R Extensions' for the details at the C level.
| >
| > See the Rcpp documentation (and, if I may, my book) for details on Rcpp
| if
| > that interests you.
| >
| > Dirk
| >
| > --
| > Dirk Eddelbuettel | e...@debian.org | http://dirk.eddelbuettel.com
| >
|
|
|
| --
|
| *Travel | Programming*
| *http://freecnpro.net*
|
| [[alternative HTML version deleted]]
|
| __
| R-devel@r-project.org mailing list
| https://stat.ethz.ch/mailman/listinfo/r-devel
|
|
--
Dirk Eddelbuettel | e...@debian.org | http://dirk.eddelbuettel.com
__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] How to convert time_t to R date object
Bill, On 17 March 2014 at 12:54, Bill Wang wrote: | Thanks for your reply, I neede convert time_t to R type in C code, can not use | Rcpp. Maybe Rcpp source code could help me. Start by reading 'Writing R Extensions' and figure out how to send an int back and forth. Then cast between int and time_t. Then set the class of the int variable to Date type. Dirk -- Dirk Eddelbuettel | e...@debian.org | http://dirk.eddelbuettel.com __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] How to convert time_t to R date object
Hi Sandip,
I think that you know on the 64-bit Unix/Linux/OS X platforms, long is
64-bit whereas int and INTEGER are 32-bit, so return directly Integer
unexpected problems may occur.
Cheers,
Bill
2014-03-17 13:24 GMT+08:00 Sandip Nandi :
> Hi Bill ,
>
> The following C code may help you , time_t is typedef to long int
>
> SEXP getTime() {
>
> time_t current_time;
> char* c_time_string;
> current_time = time(NULL);
> c_time_string = ctime(¤t_time);
>
>return mkString(c_time_String); // or if you want to return as int
> vector return scalarInt(current_time);
> }
>
> If you feel anything wrong , I will be very happy to know .
>
> Thanks,
> Sandip
>
>
> On Sun, Mar 16, 2014 at 9:54 PM, Bill Wang wrote:
>
>> Hi Dirk,
>>
>> Thanks for your reply, I neede convert time_t to R type in C code, can not
>> use Rcpp. Maybe Rcpp source code could help me.
>>
>> Cheers,
>> Bill
>>
>>
>> 2014-03-16 22:55 GMT+08:00 Dirk Eddelbuettel :
>>
>> >
>> > On 16 March 2014 at 18:36, Bill Wang wrote:
>> > | I am writing a R extensions, and I need pass time_t to R in C, but I
>> > don't
>> > | know how to do.
>> > | Can you give me some help? do not use double directly.
>> >
>> > Just treat it as an int:
>> >
>> > R> library(Rcpp)
>> > R> cppFunction("Date time_t2date(time_t what) { return((int) what);
>> }")
>> > R> time_t2date(0)
>> > [1] "1970-01-01"
>> > R> time_t2date( Sys.Date() )
>> > [1] "2014-03-16"
>> > R>
>> >
>> >
>> > Here I use Rcpp to define the 'time_t2date' function on the fly.
>> >
>> > It takes the time_t and returns a Date type (which here is a C++ Date
>> type
>> > mapping to the R Date -- you can ignore that, but will have to write the
>> > legwork yourself if you don't use Rcpp).
>> >
>> > As 'time_t' is 'long int' on my system, so I cast it to int. The rest is
>> > automagic (thanks to RcpP).
>> >
>> > Notice that I also get today's date down and up correctly.
>> >
>> > See 'Writing R Extensions' for the details at the C level.
>> >
>> > See the Rcpp documentation (and, if I may, my book) for details on Rcpp
>> if
>> > that interests you.
>> >
>> > Dirk
>> >
>> > --
>> > Dirk Eddelbuettel | e...@debian.org | http://dirk.eddelbuettel.com
>> >
>>
>>
>>
>> --
>>
>> *Travel | Programming*
>> *http://freecnpro.net*
>>
>> [[alternative HTML version deleted]]
>>
>> __
>> R-devel@r-project.org mailing list
>> https://stat.ethz.ch/mailman/listinfo/r-devel
>>
>
>
--
*Travel | Programming*
*http://freecnpro.net*
[[alternative HTML version deleted]]
__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] How to convert time_t to R date object
On 17/03/2014 11:39, Dirk Eddelbuettel wrote: Bill, On 17 March 2014 at 12:54, Bill Wang wrote: | Thanks for your reply, I neede convert time_t to R type in C code, can not use | Rcpp. Maybe Rcpp source code could help me. Start by reading 'Writing R Extensions' and figure out how to send an int back and forth. Then cast between int and time_t. Then set the class of the int variable to Date type. But note that time_t is not 'int' on many modern systems: almost all 64-bit ones and on some 32-bit ones. I guess this is actually meant to be a date-time object, hence class POSIXct. Class POSIXct is based on doubles, so return a double and add the classes in the R wrapper. If you want class Date, divide by 86400 and return an integer. That does assume that the system is POSIX-compliant and so ignores leap seconds. I have not encountered one that is not for many years, but allegedly there are locales which do count leap seconds on some Unix-alikes. C code to adjust that is in src/main/datetime.c in the R sources. -- Brian D. Ripley, rip...@stats.ox.ac.uk Professor of Applied Statistics, http://www.stats.ox.ac.uk/~ripley/ University of Oxford, Tel: +44 1865 272861 (self) 1 South Parks Road, +44 1865 272866 (PA) Oxford OX1 3TG, UKFax: +44 1865 272595 __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
[Rd] R-devel 3.2.0 MAC OS
Hello, I'm trying to build a package to submit to CRAN using R-devel 3.2.0 MAC OS from http://r.research.att.com/. As my package has dependencies, where should I get the contributed packages for this R-devel version? I found the package sources here http://r.research.att.com/src/contrib/3.2.0/, but I'm looking for the binaries? Thanks, Axel. [[alternative HTML version deleted]] __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
[Rd] valgrind and C++
Hi, I am sorry if this is perceived as a C++ question rather than an R question. After uploading an R library to CRAN (MCMCglmm) the C++ code failed to pass the memory checks. The errors come in pairs like: Mismatched free() / delete / delete [] at 0x4A077E6: free (vg_replace_malloc.c:446) by 0x144FA28E: MCMCglmm (MCMCglmm.cc:2184) Address 0x129850c0 is 0 bytes inside a block of size 4 alloc'd at 0x4A07CE4: operator new[](unsigned long) (vg_replace_malloc.c:363) by 0x144F12B7: MCMCglmm (MCMCglmm.cc:99) which is associated with lines allocating and freeing memory (nG is an integer): int *keep = new int [nG]; and delete [] keep; To me this looks fine, and on my machine (Scientific Linux 6.4) using gcc 4.4.7-3 and valgrind 1:3.8.1-3.2 I get no such errors. Its not clear to me which flavour of Linux or compiler the CRAN team used, although from MCMCglmm-Ex.Rout I can see the same version of valgrind was used. Any insight would be very welcome. Kind Regards, Jarrod -- The University of Edinburgh is a charitable body, registered in Scotland, with registration number SC005336. __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] R-devel 3.2.0 MAC OS
On Mar 17, 2014, at 1:17 PM, Axel Urbiz wrote: > Hello, > > I'm trying to build a package to submit to CRAN using R-devel 3.2.0 MAC OS > from http://r.research.att.com/. > > As my package has dependencies, where should I get the contributed packages > for this R-devel version? I found the package sources here > http://r.research.att.com/src/contrib/3.2.0/, but I'm looking for the > binaries? > Please use R-SIG-Mac, and there are currently no R-devel binaries in the light of the 3.1.0 release since they have just been branched and resources are scarce to build the full set. We typically don't provide package binaries for R-devel until later when it starts diverging. Cheers, Simon __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] CRAN rejects package because of write statement in Fortran
So the best solution I've had just yet for this problem is to convert numeric
to character in C and bypass the Fortran "Write" in that way. Unfortunately
my C is a bit rusty and I'm having trouble getting this to work. I image in
the C it should look something like:
void F77_SUB(converti)(char *ch_Result, int
i_Int){*ch_Result=sprintf("%d",*a); }
and in the Fortran something like
call converti(ch_Result,i_Int)
but this is clearly not right any help would be greatly appreciated.
--
View this message in context:
http://r.789695.n4.nabble.com/CRAN-rejects-package-because-of-write-statement-in-Fortran-tp4683287p4686997.html
Sent from the R devel mailing list archive at Nabble.com.
__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
