Hi,
I've been looking at the "libltdl and RTLD_GLOBAL" issue with embedding
perl in a dlopen'd plugin.
An instance of this with freeradius is #416266 (recently reassigned to
perl), and I see #327585 against openldap is another one.
To recap, the problem is that lt_dlopen() from the Debian system libltdl
has called dlopen(3) with RTLD_LOCAL instead of RTLD_GLOBAL ever since
#195821 was fixed. As the compiled XS modules aren't linked against
libperl, its symbols aren't exposed to them, resulting in errors like
'/usr/lib/perl/5.10/auto/Data/Dumper/Dumper.so: undefined symbol: Perl_sv_cmp'.
Observations:
- this problem isn't specific to perl and can easily be triggered with the
freeradius rlm_python module too [1]
- it's clearly possible to dlopen() compiled Perl modules from a dlopen'd
module if you don't use libltdl, see apache2+libapache2-mod-perl2 for
an example
- the XS modules are actually plugins in a private directory, not generic
shared libraries. Having unresolved symbols in a plugin without
a corresponding NEEDED entry seems to be very common, see for
example /usr/lib/apache2/modules, /usr/lib/python2.5/lib-dynload/,
/usr/lib/cdebconf etc.
- as noted in #327585, linking the XS shared objects against libperl
is potentially a problem on *i386, where /usr/bin/perl is statically
linked with libperl.a for performance reasons. (I don't have any data
about these performance reasons myself, I'm relying on hearsay and
/usr/share/doc/perl/README.Debian.gz here.)
While this does seem to work in a quick and limited test of mine,
it would bring in both libperl.a and libperl.so for all uses of
/usr/bin/perl that need XS modules, and I'm not sure which version of
the functions would get used later. If the PIC versions win, we'd be
giving away the performance benefit we got from static linking in the
first place.
At the very least, it would add 1.5M to the size of the perl-base
package on i386 AFAICS. I'm not sure how much the memory footprint of
the /usr/bin/perl invocations would increase.
Also note that we currently ship /usr/lib/libperl.a on all the
architectures, so everything that applies to the i386 /usr/bin/perl
case applies to anybody using the static library on the other archs too.
Given that i386 is still our most popular architecture, the other
proposed options don't seem very appealing either:
* only link the modules against libperl.so on the other architectures
(no fix for i386)
* link /usr/bin/perl dynamically on i386 too
(reduced performance in the very common case
for the benefit of a very uncommon case)
- it turns out libltdl nowadays does have an interface where you can
specify RTLD_GLOBAL. From the libtool Changelog.2007:
2007-05-08 Gary V. Vaughan <g...@gnu.org>
Without this patch, lt_dlopen always opens modules with symbol
visibility set according to the underlying implementation.
Here, we add lt_dlopenadvise() to allow callers to request,
among other things, local or global symbol visibility from the
underlying dlloader:
Indeed, the attached proof of concept makes the freeradius problem go away
for me, and I expect openldap could work with something similar. (FWIW,
note that the trivial my_dlopenextglobal() function was adapted from the
libtool documentation, so it might be considered to be under the GFDL.)
Josip: based on the above, I think #416266 should be fixed in freeradius
and not in perl. If you agree, please reassign back yourself.
[1]: add python to the instantiate{} block in radiusd.conf
and something like
# cat /etc/freeradius/modules/python
python {
mod_instantiate = radiusd_test
func_instantiate = instantiate
}
# cat /usr/local/lib/python2.5/site-packages/radiusd_test.py
import sys
import socket
def instantiate(test):
sys.stderr.write("hello, world!")
and you get
rlm_python:EXCEPT:<type 'exceptions.ImportError'>:
/usr/lib/python2.5/lib-dynload/_socket.so: undefined symbol: PyExc_ValueError
rlm_python:python_load_function: failed to import python function
'radiusd_test.instantiate'
--
Niko Tyni nt...@debian.org
>From 260797d463e6d94254520160236af864c54c5589 Mon Sep 17 00:00:00 2001
From: Niko Tyni <nt...@debian.org>
Date: Sun, 29 Nov 2009 22:10:30 +0200
Subject: [PATCH] Switch to lt_dlopenadvise() to get RTLD_GLOBAL set.
Proof of concept for fixing http://bugs.debian.org/416266
---
src/main/modules.c | 17 ++++++++++++++++-
1 files changed, 16 insertions(+), 1 deletions(-)
diff --git a/src/main/modules.c b/src/main/modules.c
index ea1c256..7a93b0e 100644
--- a/src/main/modules.c
+++ b/src/main/modules.c
@@ -391,6 +391,21 @@ int detach_modules(void)
}
+static lt_dlhandle my_dlopenextglobal (const char *filename)
+{
+ lt_dlhandle handle = 0;
+ lt_dladvise advise;
+
+ if (!lt_dladvise_init (&advise) && !lt_dladvise_ext (&advise)
+ && !lt_dladvise_global (&advise))
+ handle = lt_dlopenadvise (filename, advise);
+
+ lt_dladvise_destroy (&advise);
+
+ return handle;
+}
+
+
/*
* Find a module on disk or in memory, and link to it.
*/
@@ -411,7 +426,7 @@ static module_entry_t *linkto_module(const char *module_name,
/*
* Keep the handle around so we can dlclose() it.
*/
- handle = lt_dlopenext(module_name);
+ handle = my_dlopenextglobal(module_name);
if (handle == NULL) {
cf_log_err(cf_sectiontoitem(cs),
"Failed to link to module '%s': %s\n",
--
1.6.5.2
