[Rd] status of the FreeBSD port/package and some questions
Hello,
Over time, the FreeBSD R port/package [1] has accumulated some cruft. We've
begun an overhaul and the process has generally been positive since most of the
local patches and workarounds are no longer necessary.
However, we still require a few workarounds.
1. Libjpeg
The autotools check for libjpeg >=6b fails even when libjpeg is present
(e.g. /usr/local/lib/libjpeg.so.8 exists). To work around this, when the user
selects the JPEG option from a list of configuration choices,
graphics/jpeg-turbo (currently version 1.4.2) is installed as a dependency and
the variable r_cv_header_jpeglib_h is set to yes.
2. ARMv6 Build Issues
We used to have a local patch similar to the change applied for
https://stat.ethz.ch/pipermail/r-devel/2015-October/071902.html. Now that these
checks are included in src/extra/tre/tre-internal.h, we only need, I believe, an
#include in tre-internal.h for the definitions of UINT32_MAX and
INT32_MAX. This fixes one build error, but the build still fails on ARMv6
(http://pkg.awarnach.mathstat.dal.ca/data/10armv6-default/2016-07-23_13h52m17s/logs/errors/R-3.3.1_2.log).
3. BLAS
Users can choose from four options for BLAS: ATLAS, OpenBLAS, Netlib, and RBLAS
(bundled BLAS). I've noticed some segmentation faults with OpenBLAS. They
occur when quitting R, but for some reason, only through Emacs/ESS. A core file
is dumped, but I haven't looked into it yet. I also haven't tested ATLAS or
NetLib's BLAS extensively. Based on what I've read in the "R Installation and
Administration" manual, I will likely switch the default to R's bundled BLAS.
4. PDF Manuals and TeX Dependencies
Since the TeX dependencies to build the PDF manuals is quite heavy, we give
users the option to turn this off. I don't see an autoconf option to do this,
so here is our workaround when the user chooses not to install the PDF
manuals:
.for i in KPSEWHICH MAKEINDEX PDFLATEX PDFTEX TEX TEXI2DVI
CONFIGURE_ENV+=ac_cv_path_${i}=""
.endfor
5. Miscellaneous
We changed a few default locations.
@${REINPLACE_CMD} -e 's|(libdir)/pkgconfig|(prefix)/libdata/pkgconfig|g'\
${WRKSRC}/src/nmath/standalone/Makefile.in \
${WRKSRC}/src/unix/Makefile.in
@${REINPLACE_CMD} -e "s|/usr/local|${LOCALBASE}|g" \
${WRKSRC}/configure
We created versions for shared libraries.
.for L in R Rblas Rlapack
@if [ -d ${STAGEDIR}${PREFIX}/lib/R/lib ] ; then \
cd ${STAGEDIR}${PREFIX}/lib/R/lib; if [ -f lib${L}.so ] ; then \
${LN} -sf lib${L}.so lib${L}.so.${${L:tu}_SOVERSION} ; fi; fi
.endfor
We've also created a separate, simple port/package for libRMath. It basically
turns off all configuration options
--without-cairo --without-ICU --disable-java --without-jpeglib
--without-libintl-prefix --without-libpng --disable-nls --without-readline
--without-tcltk --without-libtiff --without-x
and installs
include/Rmath.h lib/libRmath.a lib/libRmath.so
lib/libRmath.so.%%RMATH_SOVERSION%%
Here are my proposed Makefiles for math/R and math/libRmath
http://ftfl.ca/misc/Makefile_R
http://ftfl.ca/misc/Makefile_libRMath
They are built for the FreeBSD ports system, but I guess most of what's going on
will be clear. If you have any comments or suggestions for improvements, please
share.
Regards,
Joseph
[1] A port is what we call the "blueprint" system for package building. Most
users will probably just install the pre-built binary package, but if they want
a package with customized options, they can use the port to build a package
themselves.
signature.asc
Description: PGP signature
__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
[Rd] shared libraries: missing soname
Hello, R's shared libraries are linked without setting the soname. This is causing problems for some consumers. Error: /usr/local/lib/R/library/tseries/libs/tseries.so is linked to /usr/local/lib/R/lib/libRblas.so which does not have a SONAME. math/R needs to be fixed. What's the correct way to add the necessary linker flags? I believe it should be something like this. cc -shared -Wl,-soname,... Regards, Joseph signature.asc Description: PGP signature __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] shared libraries: missing soname
Hello Dirk, Dirk Eddelbuettel writes: > On 20 November 2016 at 19:28, Joseph Mingrone wrote: > | Hello, > | > | R's shared libraries are linked without setting the soname. This is > causing problems for some consumers. > | > | Error: /usr/local/lib/R/library/tseries/libs/tseries.so is linked to > | /usr/local/lib/R/lib/libRblas.so which does not have a SONAME. > math/R needs > | to be fixed. > | > | What's the correct way to add the necessary linker flags? I believe it > should be something like this. > | > |cc -shared -Wl,-soname,... > I think that may be true "in theory" (for libraries loaded by ldd(8) via > `/etc/ld.conf`) but not "in practice" for R which loads these shared > libraries itself (via dlopen(3) etc). R may use dlopen() but other customers may not. > For what it is worth, I have been providing the Debian packages "as is" for > now 15+ years and nobody has complained. > What system are you on to get that complaint? This is on FreeBSD. Our apt-get, pkg, will not register shared library dependencies if the shared library does not have a soname. If the library gets changed and is no longer compatible with the previous version, and there is no soname to check, pkg will not know that all its dependencies need to be reinstalled. Regards, Joseph signature.asc Description: PGP signature __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] shared libraries: missing soname
Dirk, Dirk Eddelbuettel writes: > On 20 November 2016 at 21:49, Joseph Mingrone wrote: > | Hello Dirk, > | > | Dirk Eddelbuettel writes: > | > | > On 20 November 2016 at 19:28, Joseph Mingrone wrote: > | > | Hello, > | > | > | > | R's shared libraries are linked without setting the soname. This is > causing problems for some consumers. > | > | > | > | Error: /usr/local/lib/R/library/tseries/libs/tseries.so is > linked to > | > | /usr/local/lib/R/lib/libRblas.so which does not have a SONAME. > math/R needs > | > | to be fixed. > | > | > | > | What's the correct way to add the necessary linker flags? I believe it > should be something like this. > | > | > | > |cc -shared -Wl,-soname,... > | > | > I think that may be true "in theory" (for libraries loaded by ldd(8) via > | > `/etc/ld.conf`) but not "in practice" for R which loads these shared > | > libraries itself (via dlopen(3) etc). > | > | R may use dlopen() but other customers may not. > Yes, well, but are there other customers? Yes. Here is one example. https://rkward.kde.org/ > | > For what it is worth, I have been providing the Debian packages "as is" > for > | > now 15+ years and nobody has complained. > | > | > What system are you on to get that complaint? > | > | This is on FreeBSD. Our apt-get, pkg, will not register shared library > dependencies if the shared library does not have a soname. If the library > | gets changed and is no longer compatible with the previous version, and > there is no soname to check, pkg will not know that all its dependencies > | need to be reinstalled. > Hmpf. No mechanism for fallback / default soname? In that case you (or > whoever acts as FreeBSD maintainer) may have to supply one. Using a fallback or default soname would defeat the purpose, which is to detect when the library's interface has changed, so that the proper action can be taken. I could provide a local patch for R's autotools, but as a package maintainer yourself, I expect you also prefer when upstream get's it right. Are there any constructive reasons not to include a proper soname? Regards, Joseph signature.asc Description: PGP signature __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] shared libraries: missing soname
Dirk, Dirk Eddelbuettel writes: > On 21 November 2016 at 23:24, Joseph Mingrone wrote: > | Dirk Eddelbuettel writes: > | > On 20 November 2016 at 21:49, Joseph Mingrone wrote: > | > | Hello Dirk, > | > | > | > | Dirk Eddelbuettel writes: > | > | > | > | > On 20 November 2016 at 19:28, Joseph Mingrone wrote: > | > | > | Hello, > | > | > | > | > | > | R's shared libraries are linked without setting the soname. This > is causing problems for some consumers. > | > | > | > | > | > | Error: /usr/local/lib/R/library/tseries/libs/tseries.so is > linked to > | > | > | /usr/local/lib/R/lib/libRblas.so which does not have a > SONAME. math/R needs > | > | > | to be fixed. > | > | > | > | > | > | What's the correct way to add the necessary linker flags? I > believe it should be something like this. > | > | > | > | > | > |cc -shared -Wl,-soname,... > | > | > | > | > I think that may be true "in theory" (for libraries loaded by ldd(8) > via > | > | > `/etc/ld.conf`) but not "in practice" for R which loads these shared > | > | > libraries itself (via dlopen(3) etc). > | > | > | > | R may use dlopen() but other customers may not. > | > | > Yes, well, but are there other customers? > | > | Yes. Here is one example. https://rkward.kde.org/ > Really? We had that eg in Debian too for a decade plus and it works just > fine _as is_ and finds its libraries. Without requiring a change. These are also not fatal errors on FreeBSD, where everything, for now, also just works. ...until a library's interface changes. You seem to be arguing that sonmaes are pointless. We disagree. > It (AFAIK) just embeds R "as is" (as does my much smaller RInside). > edd@bud:~$ ldd /usr/bin/rkward | grep R # no R libs known to ldd > edd@bud:~$ ldd /usr/bin/rkward | wc -l # lots other shared libraries > 40 > edd@bud:~$ I can't say for certain (I'm not an rkward user), but looking at the build log, it seems to. Do you have R built as a shared library? Here are select lines from rkward's build log: ... -- Checking for existence of R shared library -- Exists at /usr/local/lib/R/lib/libR.so -- Checking whether we should link against Rlapack library -- Yes, /usr/local/lib/R/lib/libRlapack.so exists -- Checking whether we should link against Rblas library ... ... /usr/bin/c++ -O2 -pipe -I/usr/local/include -fstack-protector -fno-strict-aliasing -Wnon-virtual-dtor -Wno-long-long -Wundef -Wcast-align -Wchar-subscripts -Wall -W -Wpointer-arith -Wformat-security -Woverloaded-virtual -fno-common -fvisibility=hidden -Werror=return-type -fvisibility-inlines-hidden -Wno-return-type-c-linkage -O2 -DNDEBUG -DQT_NO_DEBUG -Wl,-rpath=/usr/local/lib/gcc48 -L/usr/local/lib/gcc48 -B/usr/local/bin -fstack-protector CMakeFiles/rkward.rbackend.dir/rkrbackend.o CMakeFiles/rkward.rbackend.dir/rksignalsupport.o CMakeFiles/rkward.rbackend.dir/rklocalesupport.o CMakeFiles/rkward.rbackend.dir/rkrsupport.o CMakeFiles/rkward.rbackend.dir/rkstructuregetter.o CMakeFiles/rkward.rbackend.dir/rkrbackendprotocol_backend.o CMakeFiles/rkward.rbackend.dir/rkreventloop.o CMakeFiles/rkward.rbackend.dir/rkrbackendprotocol_shared.o CMakeFiles/rkward.rbackend.dir/rdata.o CMakeFiles/rkward.rbackend.dir/rkbackendtransmitter.o CMakeFiles/rkward.rbackend.dir/rktransmitter.o -o rkward.rbackend -L/usr/local/lib -L/usr/local/lib/R/lib -L/wrkdirs/usr/ports/math/rkward-kde4/work/rkward-0.6.5/lib -L/usr/local/lib/qt4 ../../lib/librkgraphicsdevice.backend.a -lR -lRlapack -lRblas -pthread /usr/local/lib/qt4/libQtNetwork.so /usr/local/lib/qt4/libQtCore.so -pthread /usr/local/lib/libintl.so -Wl,-rpath,/usr/local/lib:/usr/local/lib/R/lib:/usr/local/lib/qt4: ... > | > | > For what it is worth, I have been providing the Debian packages "as > is" for > | > | > now 15+ years and nobody has complained. > | > | > | > | > What system are you on to get that complaint? > | > | > | > | This is on FreeBSD. Our apt-get, pkg, will not register shared library > dependencies if the shared library does not have a soname. If the library > | > | gets changed and is no longer compatible with the previous version, and > there is no soname to check, pkg will not know that all its dependencies > | > | need to be reinstalled. > | > | > Hmpf. No mechanism for fallback / default soname? In that case you (or > | > whoever acts as FreeBSD maintainer) may have to supply one. > | > | Using a fallback or default soname would defeat the purpose, which is to > detect > | when the library's interface ha
Re: [Rd] shared libraries: missing soname
Dirk Eddelbuettel writes: > On 22 November 2016 at 00:02, Joseph Mingrone wrote: > | These are also not fatal errors on FreeBSD, where everything, for now, also > just > | works. ...until a library's interface changes. You seem to be arguing that > | sonmaes are pointless. We disagree. > You are putting words in my mouth. In my very first reply to you, I pointed > out that (for non-BSD systems at least) the sonames do not matter as R loads > the libraries itself, rather than via ldd. No more, no less. Let me restate. You seem to be arguing that, because R itself doesn't consume it's shared libraries via ldd(), sonames serve no purpose, in this case. Please correct me if I'm putting words in your mouth. > | I can't say for certain (I'm not an rkward user), but looking at the build > Why did _you_ then bring up rkward as an example? That was your suggestion. Because you asked, "Yes, well, but are there other customers?" Also, I'm trying to put myself in the perspective of package users. Is this a more appropriate example? # ldd /usr/local/lib/R/library/tseries/libs/tseries.so | grep libR libRblas.so => /usr/local/lib/R/lib/libRblas.so (0x80120c000) libR.so => /usr/local/lib/R/lib/libR.so (0x801c0) signature.asc Description: PGP signature __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] shared libraries: missing soname
Martin Maechler writes: > To the issue: I also don't see what your point is. > R works with these so libraries as intended in all cases as > far as we know, and so I don't understand why anything needs to > be changed. > All these libraries "belong to R" and are tied to a specific > version of R and are not be used outside of R, so I also don't see > your point. "are not to be used outside of R". You are distributing shared libraries that aren't meant to be shared (outside of R)? But they are. And I gave an example. This is the first sentence of section 3.1.1 [1] of the Linux Documentation Project's -Program Library HOWTO-, "Every shared library has a special name called the 'soname'". I'll ask again, since this was ignored the two previous times I asked it (but I don't expect an answer). Are there any constructive reasons not to include a proper soname in the shared libraries? I will just patch the FreeBSD port. Joseph [1] http://www.tldp.org/HOWTO/Program-Library-HOWTO/shared-libraries.html#AEN46 signature.asc Description: PGP signature __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] shared libraries: missing soname
Martin Maechler writes: > Well, Dirk has said to have given his last reply on this thread. > I (as a member of R-core) am glad about people like Dirk who > take some of our load and helpfully answer such > questions/reports on R-devel. I am glad too. Thank you. My ultimate goal is to ensure that R works as well on FreeBSD as it does elsewhere. It mostly just works (again, appreciative), but there are a few challenges because some R conventions and some FreeBSD conventions may collide. Dirk Eddelbuettel writes: > Consider eg this: > edd@max:~$ locate Matrix.so > /usr/lib/R/library/Matrix/libs/Matrix.so > /usr/local/lib/R/site-library/Matrix/libs/Matrix.so > /usr/local/lib/R-devel/lib/R/library/Matrix/libs/Matrix.so > edd@max:~$ > I have three versions of Matrix.so. And still not problem. The third one > only enters for R-devel, not R. The first two are distinguished by R itself > _by virtue of different paths_ just like Martin and I told you. > And _no_ other program on my system knows about Matrix.so: > edd@max:~$ ldconfig -p | grep Matrix.so > edd@max:~$ ldconfig -p | wc -l > 2814 > edd@max:~$ > ldd / ldconfig do NOT know Matrix.so -- as I told you before -- despite the > fact that they know thousands of other things on this (development) machine. For a frequently-updated Debian package repository, I assume you _manually_ bump all the Debian R packages whenever the main R package is upgraded. Is that correct? That is, when a Debian user upgrades the r-base package from, say, version 3.2.5 to 3.3.1, then r-cran-tseries must/should be rebuilt/reinstalled against the new R package? If so, maybe there is something useful we could submit. I will have to study how R uses autotools, because that also seems a bit different than I am used to. I also notice that on Debian you make a soft link of /usr/lib/R/lib/libR.so to /usr/lib/libR.so. Given all that has been discussed, I am unclear why. Thank you for sticking with the thread, Joseph signature.asc Description: PGP signature __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] shared libraries: missing soname
Dirk,
Dirk Eddelbuettel writes:
> You assume that change == breakage. Yet that assumption is baseless.
> Which is what someone like Martin (R Core, and "at it" since the 80s pre-R
> and 90s with the almost very beginning of R) and myself (around R since the
> late 90s, somewhat involved since the early 00s) keep telling you.
> At some point it might appear to be approproiiate for you to actually take
> our word for it.
It is not that I am not taking your word. There are some unique (to me)
approaches in R and some of them are subtle. I am also taking criticism from
the other side, because, as I said, some conventions collide. When Martin said,
> All these libraries "belong to R" and are tied to a specific version of R...
I understood that "tied to a specific version of R" meant that (Debian/FreeBSD)
R packages should be updated in lock step with R. So, on Debian, changes in the
r-core package never necessitate a bump of r-cran-*? In other words, the
libR.so interface is guaranteed to be stable across releases?
> | I also notice that on Debian you make a soft link of /usr/lib/R/lib/libR.so
> to
> | /usr/lib/libR.so. Given all that has been discussed, I am unclear why.
> Well noticed -- yet a stricly personaly reason via two projects I have been
> (co-)authoring: littler and RInside. They both "embed" R via libR and we do
> both rpath ("somewhat" verboten by Debian Policy as it hard codes a path,
> hence the alternate of placing it where ldd / ldconfig find it).
> But please note that that is _me_ doing this, and the R Core gospel we have
> been trying for you to understand still stand: __what you insist is needed
> actually is not__.
Understood.
> | Thank you for sticking with the thread,
> Sorry for coming through as pedantic but you (and we're now at what, six
> posts in and counting?) still start from the wrong (at least outside of the
> gilded confines of FreeBSD) premise. It. Just. Works.
> Just give us and the unknown-but-sometimes-estimated-to-be-in-the-millions of
> R users some credit. What is there __works__. Seriously. Debian folks are
> pendantic for technical excellence __and even they have no issue with
> per-package and local shared libraries__. Which is what this is. Really.
Pedantic is fine. I give credit! I would not be investing my (comparatively
minuscule) time if I did not appreciate R. I would also not be doing anyone a
service if I released a flawed FreeBSD package that did not do the project
justice.
Joseph
signature.asc
Description: PGP signature
__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
[Rd] reproducible segmentation fault installing packages on FreeBSD 11.1
Hi,
This happens when attempting to install any package. There were no such
problems on 11.0.
Some other ways to trigger the problem:
curlGetHeaders("http://bugs.r-project.org";)
There are no problems with the first two calls, but then the crash
always happens on the third call.
tf <- tempfile()
download.file("http://cran.r-project.org/",tf,method="libcurl";)
This one is ok for the first five calls, but always crashes on the
sixth call.
download.file("http://cran.r-project.org/",tf,method="internal";)
does not trigger the issue.
Is there any further information I can provide or anything else I can
test?
Joseph
jrm@phe ~ % R --debugger=gdb
GNU gdb (GDB) 8.0 [GDB v8.0 for FreeBSD]
Copyright (C) 2017 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law. Type "show copying"
and "show warranty" for details.
This GDB was configured as "x86_64-portbld-freebsd11.0".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
Re: [Rd] reproducible segmentation fault installing packages on FreeBSD 11.1
Dirk Eddelbuettel writes: > On 31 July 2017 at 19:38, Joseph Mingrone wrote: > | This happens when attempting to install any package. There were no such > | problems on 11.0. > | > | Some other ways to trigger the problem: > [...] > | trying URL 'https://cloud.r-project.org/src/contrib/Rcpp_0.12.12.tar.gz' > | [New LWP 100854 of process 56011] > | > | Thread 7 received signal SIGSEGV, Segmentation fault. > | [Switching to LWP 100854 of process 56011] > | uw_frame_state_for (context=context@entry=0x7fffdfffde20, > fs=fs@entry=0x7fffdfffdb70) > | at /usr/ports/lang/gcc5/work/gcc-5.4.0/libgcc/unwind-dw2.c:1249 > | 1249 return MD_FALLBACK_FRAME_STATE_FOR (context, fs); > Looks like a gcc error to me. Rcpp itself is pretty widely used and tested > and I am not aware of it having issue per se on any of the *BSDs. The problem isn't specific to any package or packages in general, but seems related to libcurl and/or gfortran. The third call to CURLcode ret = curl_easy_perform(hnd); in in_do_curlGetHeaders() from src/modules/internet/libcurl.c triggers the problem _unless_ R is built with flang instead of gfortran. For FreeBSD 11.1 R users, there are two workarounds [1]. 1) Build the FreeBSD R package with the FLANG option instead of GFORTRAN. FLANG may be the default soon. 2) Add options(download.file.method="wget") to ~/.Rprofile with a newline after it. Ensure ftp/wget is installed, because it is not pulled in by math/R. Joseph [1] Both these workarounds prevent a predictable crash, but have only been lightly tested. signature.asc Description: PGP signature __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] reproducible segmentation fault installing packages on FreeBSD 11.1
For anyone interested, here is the FreeBSD bug report. https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=221127 signature.asc Description: PGP signature __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
