[Rd] package has a loading problem
Hello, I have difficulties in finding why I cannot load a custom R package in Linux (Suse 10). When I run R CMD check for a package source folder I get the following message: ". Error in dyn.load(file, ...) : unable to load shared library '/home/yarz/projects/Riman/Riman.Rcheck/Riman/libs/Rimanlib.so': /home/yarz/projects/Riman/Riman.Rcheck/Riman/libs/Rimanlib.so: undefined symbol: immagnify Error in library(Riman) : .First.lib failed for 'Riman' Execution halted It looks like this package has a loading problem: see the messages for details. " The same message I get when I call library(Riman) inside R after installation of the package. The installation via R CMD INSTALL caused no error messages. However, Riman.so had been built before R CMD INSTALL was called, and put into the source subfolder Riman/inst/libs. It was built via a usual gcc call and linked with all the necessary libraries (so it is weird that references seemed to be undefined). The strange thing is that when I borrow Riman.so built by some other person in my group and replace my copy with it, loading problem disappears! We both use the same server with Linux Suse 10, the same R installation, the same version of gcc and the same Makefiles for building Riman.so . We went through variables I might have omitted to set correctly, but they seemed OK. If you could give me any hint on what might be a problem, where else to look, I would be very happy. Many thanks in advance. Regards, Yulia __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] package has a loading problem
Have you create a zzz.R file in the R folder? I think your issue is
related to this...
Example of the zzz.R file:
".First.lib" <-
function(lib, pkg)
{
library.dynam("your_pkd_name", package = pkg, lib.loc = lib)
return(invisible(0))
}
This is required when you have C/Fortran code. See the writing R
extension manual.
Best,
Mathieu
Yulia Arzhaeva a écrit :
Hello,
I have difficulties in finding why I cannot load a custom R package in
Linux (Suse 10). When I run R CMD check for a package source folder I
get the following message:
".
Error in dyn.load(file, ...) :
unable to load shared library
'/home/yarz/projects/Riman/Riman.Rcheck/Riman/libs/Rimanlib.so':
/home/yarz/projects/Riman/Riman.Rcheck/Riman/libs/Rimanlib.so:
undefined symbol: immagnify
Error in library(Riman) : .First.lib failed for 'Riman'
Execution halted
It looks like this package has a loading problem: see the messages for
details. "
The same message I get when I call library(Riman) inside R after
installation of the package.
The installation via R CMD INSTALL caused no error messages. However,
Riman.so had been built before R CMD INSTALL was called, and put into
the source subfolder Riman/inst/libs. It was built via a usual gcc
call and linked with all the necessary libraries (so it is weird that
references seemed to be undefined).
The strange thing is that when I borrow Riman.so built by some other
person in my group and replace my copy with it, loading problem
disappears! We both use the same server with Linux Suse 10, the same
R installation, the same version of gcc and the same Makefiles for
building Riman.so . We went through variables I might have omitted to
set correctly, but they seemed OK.
If you could give me any hint on what might be a problem, where else
to look, I would be very happy. Many thanks in advance.
Regards,
Yulia
__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
--
Institute of Mathematics
Ecole Polytechnique Fédérale de Lausanne
STAT-IMA-FSB-EPFL, Station 8
CH-1015 Lausanne Switzerland
http://stat.epfl.ch/
Tel: + 41 (0)21 693 7907
__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
[Rd] RODBC Close Memory Leak Question
Hi everyone,
In relation to the RODBC odbcClose bug which was fixed back in the
changelog below:
Version: 1.2-3 (2008-01-24, released)
* Plug a memory leak in inRODBCClose (closing a connection),
reported by Stephan Henne.
* Use translateChar() on character data sent in.
Background:
I am running some data from a SQL Server database, through unixODBC
(freetds) into R via the RODBC package. If I make many calls to
odbcConnect, perform a query and then close the connection over a period
of time the memory attached appears never to be freed and the process
eventually - say 4 hours into the run - dies due to lack of memory.
Research:
I have created a dummy piece of code which shows the growing memory
allocation shown below. Each call brings back between 20-100 pieces of
data which fits into a one column vector - so not a lot of data. The
varible x is normally returns from a function and used subsequently but
for the purposes of this test I have kept it simple.
library("RODBC")
for( i in 1:100 ) {
conn <- odbcConnect()
<-- anonymised
x <- sqlQuery(conn, "SELECT * FROM table_name"); <--
anonymised
odbcClose(conn);
}
quit()
I am not sure whether the leak is related to this previous bug or if it is
related to the way the garbage collection works, but either way if I run
this piece of code the memory attached to the database handle is never
freed so as I bring back more and more data over a large number of runs my
memory slowly fills up and the application crashes, unable to allocate any
more memory.
Just to make sure I was running the latest and greatest RODBC I installed
it again from source, but the initial installation was only done last
month so I presume .install.packages would have got this latest version
anyway.
Any advice on where to start searching to patch this as it is a real
pain. I have included below version information and the valgrind
output. If this is a new bug I will add it the bug tracker but I wanted
to make sure I was not missing anything blindingly obvious to someone in
the know.
Thanks for your help,
Tom
version
_
platform i686-pc-linux-gnu
arch i686
os linux-gnu
system i686, linux-gnu
status
major 2
minor 7.0
year 2008
month 04
day22
svn rev45424
language R
version.string R version 2.7.0 (2008-04-22)
==8682== Memcheck, a memory error detector.
==8682== Copyright (C) 2002-2005, and GNU GPL'd, by Julian Seward et al.
==8682== Using LibVEX rev 1471, a library for dynamic binary translation.
==8682== Copyright (C) 2004-2005, and GNU GPL'd, by OpenWorks LLP.
==8682== Using valgrind-3.1.0, a dynamic binary instrumentation framework.
==8682== Copyright (C) 2000-2005, and GNU GPL'd, by Julian Seward et al.
==8682== For more details, rerun with: -v
==8682==
R version 2.7.0 (2008-04-22)
Copyright (C) 2008 The R Foundation for Statistical Computing
ISBN 3-900051-07-0
R is free software and comes with ABSOLUTELY NO WARRANTY.
You are welcome to redistribute it under certain conditions.
Type 'license()' or 'licence()' for distribution details.
Natural language support but running in an English locale
R is a collaborative project with many contributors.
Type 'contributors()' for more information and
'citation()' on how to cite R or R packages in publications.
Type 'demo()' for some demos, 'help()' for on-line help, or
'help.start()' for an HTML browser interface to help.
Type 'q()' to quit R.
library("RODBC")
--8682-- WARNING: unhandled syscall: 311
--8682-- You may be able to write your own handler.
--8682-- Read the file README_MISSING_SYSCALL_OR_IOCTL.
for( i in 1:100 ) {
+ conn <- odbcConnect() <--
anonymised
+ x <- sqlQuery(conn, "SELECT * FROM table_name"); <--
anonymised
+ odbcClose(conn);
+ }
quit()
==8682==
==8682== ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 255 from 1)
==8682== malloc/free: in use at exit: 13,470,816 bytes in 6,739 blocks.
==8682== malloc/free: 121,121 allocs, 114,382 frees, 134,084,145 bytes
allocated.
==8682== For counts of detected errors, rerun with: -v
==8682== searching for pointers to 6,739 not-freed blocks.
==8682== checked 13,332,140 bytes.
==8682==
==8682== 56 bytes in 1 blocks are possibly lost in loss record 15 of 55
==8682==at 0x40045EB: calloc (vg_replace_malloc.c:279)
==8682==by 0x805AA60: R_chk_calloc (memory.c:2368)
==8682==by 0x57C89D6: RODBCDriverConnect (RODBC.c:244)
==8682==by 0x8160A68: do_dotcall (dotcode.c:863)
==8682==by 0x81857CE: Rf_eval (eval.c:489)
==8682==by 0x81895B7: do_set (eval.c:1420)
==8682==by 0x81855B0: Rf_eval (eval.c:463)
==8682==by 0x818631E: do_begin (eval.c:1172)
==8682==by 0x81855B0: Rf_eval (eval.c:463)
==8682==
Re: [Rd] package has a loading problem
Hi,
Yes I have this file already.
Yulia
On Thu, Jul 10, 2008 at 6:47 PM, Mathieu Ribatet
<[EMAIL PROTECTED]> wrote:
> Have you create a zzz.R file in the R folder? I think your issue is related
> to this...
>
> Example of the zzz.R file:
>
> ".First.lib" <-
> function(lib, pkg)
> {
> library.dynam("your_pkd_name", package = pkg, lib.loc = lib)
> return(invisible(0))
> }
>
> This is required when you have C/Fortran code. See the writing R extension
> manual.
> Best,
> Mathieu
>
> Yulia Arzhaeva a écrit :
>>
>> Hello,
>>
>> I have difficulties in finding why I cannot load a custom R package in
>> Linux (Suse 10). When I run R CMD check for a package source folder I
>> get the following message:
>> ".
>> Error in dyn.load(file, ...) :
>> unable to load shared library
>> '/home/yarz/projects/Riman/Riman.Rcheck/Riman/libs/Rimanlib.so':
>> /home/yarz/projects/Riman/Riman.Rcheck/Riman/libs/Rimanlib.so:
>> undefined symbol: immagnify
>> Error in library(Riman) : .First.lib failed for 'Riman'
>> Execution halted
>>
>> It looks like this package has a loading problem: see the messages for
>> details. "
>>
>> The same message I get when I call library(Riman) inside R after
>> installation of the package.
>> The installation via R CMD INSTALL caused no error messages. However,
>> Riman.so had been built before R CMD INSTALL was called, and put into
>> the source subfolder Riman/inst/libs. It was built via a usual gcc
>> call and linked with all the necessary libraries (so it is weird that
>> references seemed to be undefined).
>>
>> The strange thing is that when I borrow Riman.so built by some other
>> person in my group and replace my copy with it, loading problem
>> disappears! We both use the same server with Linux Suse 10, the same
>> R installation, the same version of gcc and the same Makefiles for
>> building Riman.so . We went through variables I might have omitted to
>> set correctly, but they seemed OK.
>>
>> If you could give me any hint on what might be a problem, where else
>> to look, I would be very happy. Many thanks in advance.
>>
>> Regards,
>> Yulia
>>
>> __
>> R-devel@r-project.org mailing list
>> https://stat.ethz.ch/mailman/listinfo/r-devel
>>
>
> --
> Institute of Mathematics
> Ecole Polytechnique Fédérale de Lausanne
> STAT-IMA-FSB-EPFL, Station 8
> CH-1015 Lausanne Switzerland
> http://stat.epfl.ch/
> Tel: + 41 (0)21 693 7907
>
>
>
__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
[Rd] RNGkind() state
This looks like a bug to me, and is a bit hard to describe, but easy to
reproduce.
Basically, if RNGkind is saved as something other than the default, and if the
first
operation in a session is a set.seed(), the default is reverted to. Reproduce
by:
cafe-rozo> R --vanilla
R version 2.7.1 (2008-06-23)
Copyright (C) 2008 The R Foundation for Statistical Computing
ISBN 3-900051-07-0
R is free software and comes with ABSOLUTELY NO WARRANTY.
You are welcome to redistribute it under certain conditions.
Type 'license()' or 'licence()' for distribution details.
R is a collaborative project with many contributors.
Type 'contributors()' for more information and
'citation()' on how to cite R or R packages in publications.
Type 'demo()' for some demos, 'help()' for on-line help, or
'help.start()' for an HTML browser interface to help.
Type 'q()' to quit R.
> sessionInfo()
R version 2.7.1 (2008-06-23)
i386--netbsdelf
locale:
C
attached base packages:
[1] stats graphics grDevices utils datasets methods base
> set.seed(1); runif(1) # check the default
[1] 0.2655087
> RNGkind("Mar")
> RNGkind()
[1] "Marsaglia-Multicarry" "Inversion"
> set.seed(1); runif(1) # now check again
[1] 0.006153224
> set.seed(1) # save seed and RNGkind
> q("yes")
cafe-rozo> R
R version 2.7.1 (2008-06-23)
: etc
[Previously saved workspace restored]
> runif(1) # with a saved seed, Marsaglia-Multicarry is used
[1] 0.006153224
> set.seed(1); runif(1)
[1] 0.006153224
> q()
Save workspace image? [y/n/c]: n
cafe-rozo>
R version 2.7.1 (2008-06-23)
: etc
[Previously saved workspace restored]
> set.seed(1); runif(1) # saved RNGkind is ignored ...
[1] 0.2655087
> RNGkind() # and overridden ...
[1] "Mersenne-Twister" "Inversion"
> q("no")
cafe-rozo> R
R version 2.7.1 (2008-06-23)
: etc
[Previously saved workspace restored]
> RNGkind() # and wrongly reported!
[1] "Mersenne-Twister" "Inversion"
> runif(1) # Note this is actually using
> Marsaglia-Multicarry
[1] 0.006153224
>
Ray Brownrigg
Mathematics, Statistics and Computer Science
Victoria University of Wellington
Wellington, NZ
__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
[Rd] memory leak in readline code
Several folks have previously written that valgrind notices
a memory leak in R's readline code. It looks like it leaks
a copy of every input line.
% ~/R-svn/r-devel/R/bin/R --debugger=valgrind --debugger-args=--leak-check=full
--vanilla
==10725== Memcheck, a memory error detector.
==10725== Copyright (C) 2002-2006, and GNU GPL'd, by Julian Seward et al.
==10725== Using LibVEX rev 1658, a library for dynamic binary translation.
==10725== Copyright (C) 2004-2006, and GNU GPL'd, by OpenWorks LLP.
==10725== Using valgrind-3.2.1, a dynamic binary instrumentation framework.
==10725== Copyright (C) 2000-2006, and GNU GPL'd, by Julian Seward et al.
==10725== For more details, rerun with: -v
==10725==
R version 2.8.0 Under development (unstable) (2008-07-07 r46046)
Type 'q()' to quit R.
> invisible("hello")
> invisible("hello")
> invisible("hello")
> invisible("hello")
> invisible("hello")
> invisible("hello")
> invisible("hello")
> invisible("hello")
> invisible("hello")
> invisible("hello")
> q()
==10743==
==10743== ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 40 from 2)
==10743== malloc/free: in use at exit: 12,591,699 bytes in 5,927 blocks.
==10743== malloc/free: 21,015 allocs, 15,088 frees, 27,744,803 bytes allocated.
==10743== For counts of detected errors, rerun with: -v
==10743== searching for pointers to 5,927 not-freed blocks.
==10743== checked 12,612,676 bytes.
==10743==
==10743== 234 bytes in 13 blocks are definitely lost in loss record 20 of 42
==10743==at 0x40046EE: malloc (vg_replace_malloc.c:149)
==10743==by 0x68BFF9: xmalloc (in /usr/lib/libreadline.so.4.3)
==10743==by 0x6770D5: readline_internal_teardown (in
/usr/lib/libreadline.so.4.3)
==10743==by 0x688992: rl_callback_read_char (in /usr/lib/libreadline.so.4.3)
==10743==by 0x80E739C: Rstd_ReadConsole (sys-std.c:905)
==10743==by 0x8057F61: Rf_ReplIteration (main.c:205)
==10743==by 0x805827E: R_ReplConsole (main.c:306)
==10743==by 0x8058514: run_Rmainloop (main.c:966)
==10743==by 0x805676D: main (Rmain.c:33)
==10743==
==10743== LEAK SUMMARY:
==10743==definitely lost: 234 bytes in 13 blocks.
==10743== possibly lost: 0 bytes in 0 blocks.
==10743==still reachable: 12,591,465 bytes in 5,914 blocks.
==10743== suppressed: 0 bytes in 0 blocks.
==10743== Reachable blocks (those to which a pointer was found) are not shown.
==10743== To see them, rerun with: --show-reachable=yes
Some experiments show that the number of blocks leaked from
readline_internal_teardown is the number of input lines plus 2 (11+2=13
in this case) and the number of leaked bytes is the total number of
bytes in those input lines (including the trailing nulls) plus 40
(10*19+4+40=234 in this case).
I think the readline callback function is expected to free
its 'char *line' argument. The readline manual says the
input line must be freed when using the simple readline()
interface but is silent about memory management when using
the callback interface. I have not looked in the readline
source code.
See
http://www.mail-archive.com/[EMAIL PROTECTED]/msg04863.html
for a same problem in other software using readline:
> Are you sure about this? Does readline really expect callback to free
> the line? (and if it does, can it be bug in the specific readline
> version you are using?)
> - ML
A cursory look suggests Elmo is right.
That you should free the returned line is explicitly mentioned when
using the readline() call:
http://www.delorie.com/gnu/docs/readline/rlman_24.html#IDX174
No further mention at the reference for the alternative interface:
http://www.delorie.com/gnu/docs/readline/rlman_41.html#IDX288
When I added 'free(line)' to src/unix/sys-std.c:readline_handler()
valgrind stopped complaining about the leaks and did not
complain about using freed memory.
It does start complaining a new leak in the readline code,
but it looks like that may be a constant 2 block, 40 byte
leak, not something that grows as the session goes on:
==11246== ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 40 from 2)
==11246== malloc/free: in use at exit: 12,591,505 bytes in 5,916 blocks.
==11246== malloc/free: 21,015 allocs, 15,099 frees, 27,744,803 bytes allocated.
==11246== For counts of detected errors, rerun with: -v
==11246== searching for pointers to 5,916 not-freed blocks.
==11246== checked 12,612,676 bytes.
==11246==
==11246== 40 bytes in 2 blocks are definitely lost in loss record 10 of 42
==11246==at 0x40046EE: malloc (vg_replace_malloc.c:149)
==11246==by 0x68BFF9: xmalloc (in /usr/lib/libreadline.so.4.3)
==11246==by 0x68F65B: sh_set_lines_and_columns (in
/usr/lib/libreadline.so..3)
==11246==by 0x688AEE: _rl_get_screen_size (in /usr/lib/libreadline.so.4.3)
==11246==by 0x68918C: _rl_init_terminal_io (in /usr/lib/libreadline.so.4.3)
==11246==by 0x677A77: rl_initialize (in /usr/lib/libreadline.so.4.3)
==11246==by 0x6888C6: (within /usr/lib/libreadline.so.4
[Rd] Suggestion: 20% speed up of which() with two-character mod
Hi,
by replacing 'll' with 'wh' in the source code for base::which() one
gets ~20% speed up for *named logical vectors*.
CURRENT CODE:
which <- function(x, arr.ind = FALSE)
{
if(!is.logical(x))
stop("argument to 'which' is not logical")
wh <- seq_along(x)[ll <- x & !is.na(x)]
m <- length(wh)
dl <- dim(x)
if (is.null(dl) || !arr.ind) {
names(wh) <- names(x)[ll]
}
...
wh;
}
SUGGESTED CODE: (Remove 'll' and use 'wh')
which2 <- function(x, arr.ind = FALSE)
{
if(!is.logical(x))
stop("argument to 'which' is not logical")
wh <- seq_along(x)[x & !is.na(x)]
m <- length(wh)
dl <- dim(x)
if (is.null(dl) || !arr.ind) {
names(wh) <- names(x)[wh]
}
...
wh;
}
That's all.
BENCHMARKING:
# To measure both in same environment
which1 <- base::which;
environment(which1) <- globalenv(); # Needed?
N <- 1e6;
set.seed(0xbeef);
x <- sample(c(TRUE, FALSE), size=N, replace=TRUE);
names(x) <- seq_along(x);
B <- 10;
t1 <- system.time({ for (bb in 1:B) idxs1 <- which1(x); });
t2 <- system.time({ for (bb in 1:B) idxs2 <- which2(x); });
stopifnot(identical(idxs1, idxs2));
print(t1/t2);
# Fair benchmarking
t2 <- system.time({ for (bb in 1:B) idxs2 <- which2(x); });
t1 <- system.time({ for (bb in 1:B) idxs1 <- which1(x); });
print(t1/t2);
## usersystem elapsed
## 1.283186 1.052632 1.25
You get similar results if you put for loop outside the system.time()
call (and sum up the timings).
Cheers
Henrik
__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
