Re: Coinstallability of Fortran libraries built with different compilers
Le samedi 22 octobre 2011 à 16:24 -0700, Steve Langasek a écrit : > > One point to think of is how this works with multiarch, which is > > being introduced > > in Debian. Instead of 'ifort' should we use the architecture triplet, eg. > > i386-linux-intel instead ? > > Then the libraries go in i386-linux-intel rather than i386-linux-gnu > > for gfortran; > > ditto for the .mod files in /usr/include/i386-linux-intel > > I'm not familiar with this i386-linux-intel triplet. Is this a triplet > targeted by the toolchain? Does software built for this target not use GNU > libc? (I guess I can't presume that it uses any libc at all, since we're > speaking specifically of fortran here.) It uses the same libc, but it uses a different Fortran library. Because of that library, you can link Fortran libraries and programs only against other Fortran libraries built with the same compiler. The architecture triplet would be the same, but it would need an addition for the Fortran library. IIRC the problem used to be the same with C++ until the libstdc++ ABI was standardized. -- .''`. Josselin Mouette : :' : `. `' `- signature.asc Description: This is a digitally signed message part
Re: Coinstallability of Fortran libraries built with different compilers
On Sat, Oct 22, 2011 at 04:24:23PM -0700, Steve Langasek wrote: > > One point to think of is how this works with multiarch, which is > > being introduced in Debian. Instead of 'ifort' should we use the > > architecture triplet, eg. i386-linux-intel instead ? > > Then the libraries go in i386-linux-intel rather than i386-linux-gnu > > for gfortran; ditto for the .mod files in > > /usr/include/i386-linux-intel > > I'm not familiar with this i386-linux-intel triplet. Is this a triplet > targeted by the toolchain? Does software built for this target not use GNU > libc? (I guess I can't presume that it uses any libc at all, since we're > speaking specifically of fortran here.) I'm not sure about libc dependencies of fortran binaries, I'll leave Alastair to answer that bit. My understanding on library use and ABI compatibilities is that the critical point are .mod files in /usr/include, whereas .a or .so files are perfectly reusable across compilers. That means that fortran binaries compiled with any compiler are free to depend on C libraries built with any compiler. For example, /usr/lib/libnetcdff.so links with curl, libm, libc, libhdf5, and plenty others according to ldd. Ideally one would want to have parallel, per-compiler versions of fortran libraries, because of the different .mod file formats, and then share all the chain of C dependencies. Ciao, Enrico -- GPG key: 4096R/E7AD5568 2009-05-08 Enrico Zini signature.asc Description: Digital signature
Re: Coinstallability of Fortran libraries built with different compilers
On Sun, Oct 23, 2011 at 09:45:06AM +0200, Enrico Zini wrote: > Alastair to answer that bit. My understanding on library use and ABI > compatibilities is that the critical point are .mod files in > /usr/include, whereas .a or .so files are perfectly reusable across > compilers. I stand corrected: when .a or .so binary compatibility happen, it is by luck: http://en.wikipedia.org/wiki/Name_mangling#Name_mangling_in_Fortran although, according to that explanation, call name mangling seems to be standardising somehow. Ciao, Enrico -- GPG key: 4096R/E7AD5568 2009-05-08 Enrico Zini signature.asc Description: Digital signature
Re: Coinstallability of Fortran libraries built with different compilers
On 2011-10-23 08:45, Enrico Zini wrote: On Sat, Oct 22, 2011 at 04:24:23PM -0700, Steve Langasek wrote: One point to think of is how this works with multiarch, which is being introduced in Debian. Instead of 'ifort' should we use the architecture triplet, eg. i386-linux-intel instead ? Then the libraries go in i386-linux-intel rather than i386-linux-gnu for gfortran; ditto for the .mod files in /usr/include/i386-linux-intel i386-linux-intel does not yet officially exist. Where are the triplets specified? I was using this as an example. Similar issues exist with other (proprietary) compilers. I'm not familiar with this i386-linux-intel triplet. Is this a triplet targeted by the toolchain? Does software built for this target not use GNU libc? (I guess I can't presume that it uses any libc at all, since we're speaking specifically of fortran here.) I'm not sure about libc dependencies of fortran binaries, I'll leave Alastair to answer that bit. My understanding on library use and ABI compatibilities is that the critical point are .mod files in /usr/include, whereas .a or .so files are perfectly reusable across compilers. Yes, the problem is that .mod files are architecture and compiler dependent. Hence it is important that the discovery path for them is not arch. and compiler dependent too: e.g. pkg-config needs to be used accordingly to discover the appropriate path (use pkg-config --variable=fflags rather than --cflags, which returns -I or -M for the appropriate compiler, for the appropriate path). I believe that, as of current versions, intel compilers icc and ifort generate .a and .so files that are perfectly reusable. This has not been guaranteed with compiler and version, though: some have (had) incompatible extensions so that while they link with GNU libc, the reverse was is not true. This is similar in concept to i386 / amd64 only being backward compatible, etc. Hence the usefulness of the multiarch concept in this case. That means that fortran binaries compiled with any compiler are free to depend on C libraries built with any compiler. For example, /usr/lib/libnetcdff.so links with curl, libm, libc, libhdf5, and plenty others according to ldd. Ideally one would want to have parallel, per-compiler versions of fortran libraries, because of the different .mod file formats, and then share all the chain of C dependencies. Ciao, Enrico -- Alastair McKinstry , , http://blog.sceal.ie Anyone who believes exponential growth can go on forever in a finite world is either a madman or an economist - Kenneth Boulter, Economist. -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/4ea3dce2.8020...@sceal.ie
Re: Coinstallability of Fortran libraries built with different compilers
On Sun, 23 Oct 2011 at 10:22:42 +0100, Alastair McKinstry wrote: > i386-linux-intel does not yet officially exist. Where are the > triplets specified? GNU configuration types are specified by GNU config.sub and config.guess (distributed with autoconf, but they have their own upstream git repository). The canonical form is CPU-MANUFACTURER-OS or CPU-MANUFACTURER-KERNEL-OS (arm-unknown-linux-gnu, powerpc-apple-darwin1.3, sparc-sun-solaris2) but the manufacturer part is hardly ever relevant to the ABI, so cross-compiler prefixes, toolchain directories and multiarch tuples omit it. On Linux the manufacturer part is typically "unknown" anyway. The Intel vs. GNU Fortran compiler is a smaller difference than the GNU triplets normally encode: when a GNU tuple says "i386-linux-gnu" or "i386-kfreebsd-gnu", the "gnu" part refers to the GNU operating system (mainly the use of glibc as the C library, but to some extent also things like coreutils). The C++ ABI transitions we've had in the last few years (e.g. libarts1 to libarts1c2 to libarts1c2a) didn't affect the GNU tuple either. S -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/20111023123526.ga16...@reptile.pseudorandom.co.uk
Re: Coinstallability of Fortran libraries built with different compilers
On Sun, Oct 23, 2011 at 09:40:47AM +0200, Josselin Mouette wrote: > Le samedi 22 octobre 2011 à 16:24 -0700, Steve Langasek a écrit : > > > One point to think of is how this works with multiarch, which is > > > being introduced > > > in Debian. Instead of 'ifort' should we use the architecture triplet, eg. > > > i386-linux-intel instead ? > > > Then the libraries go in i386-linux-intel rather than i386-linux-gnu > > > for gfortran; > > > ditto for the .mod files in /usr/include/i386-linux-intel > > I'm not familiar with this i386-linux-intel triplet. Is this a triplet > > targeted by the toolchain? Does software built for this target not use GNU > > libc? (I guess I can't presume that it uses any libc at all, since we're > > speaking specifically of fortran here.) > It uses the same libc, but it uses a different Fortran library. > Because of that library, you can link Fortran libraries and programs > only against other Fortran libraries built with the same compiler. The > architecture triplet would be the same, but it would need an addition > for the Fortran library. Ok; it doesn't sound like multiarch is a good fit for this then, unfortunately. -- Steve Langasek Give me a lever long enough and a Free OS Debian Developer to set it on, and I can move the world. Ubuntu Developerhttp://www.debian.org/ slanga...@ubuntu.com vor...@debian.org signature.asc Description: Digital signature
Dealing with embedded javascript libraries
Hello,
I would like to discuss our handling of embedded javascript libraries.
In theory, like any other embedded library, they are to be avoided and
we have a lintian warning catching many of the common cases:
http://lintian.debian.org/tags/embedded-javascript-library.html
Unfortunately, blindly replacing the file with a symlink can
create problems of its own. Upstream tested their application
with the version of the library that they ship. Using another
version might break things horribly, an example here:
https://bugs.launchpad.net/ubuntu/+source/wordpress/+bug/816962
Even if the Debian maintainer takes care to ensure the correct
version is used (which is difficult to do, you never know what the next
upstream version of the javascript will do), it's quite possible that he
will have to switch regularly between the embedded version and the
packaged one just because of differing schedules between the two projects.
If the package uses symlinks to directories, those switchs
quickly become painful (and forgetting about them even more, see
for example #639733).
And with javascript libraries, there's no failure at build time,
you only discover much later when something is not working...
(BTW, a large part of what I'm saying also applies to PHP libraries but
the resulting runtimes failures are usually easier to catch and diagnose)
So I'm wondering if we're not giving ourselves bad advice by recommending
to replace those files with symlinks. Most of the time the Debian
maintainer is not knowledgeable enough (or doesn't have the time required
to make the proper assessment) to safely decide to diverge from upstream.
It's good to reduce the workload of the security team but only if we
don't break stuff while doing it.
My current suggestion is to use what I would call opportunistic
replacement of those embedded copies, if the packaged library matches the
embedded one, then provide a symlink instead of a copy and ensure the
dependencies hardcode the current upstream version of the packaged
library (via ${misc:Depends} most likely).
And instead of creating symlinks to directories, we duplicate the
directory hierarchy and we only symlink files so that it's easy to
switch back and forth between the embedded copy and the packaged one.
I have some early prototype code in the wordpress package, see
debian/dh_linktree. It can replace identical files with symlinks
if you pass --replace-only --same-only. It doesn't implement the
dependency generation though. In any case it needs further thoughts
and cleanups.
http://anonscm.debian.org/gitweb/?p=collab-maint/wordpress.git;a=blob;f=debian/dh_linktree;hb=master
What are your thoughts on this topic?
One downside of this approach is that a package like wordpress
would have to be rebuilt when a new upstream version of one of its
javascript libraries is released. (And we don't have bin-nmu for
arch: all currently).
Cheers,
--
Raphaël Hertzog ◈ Debian Developer
Pre-order a copy of the Debian Administrator's Handbook and help
liberate it: http://debian-handbook.info/go/ulule-rh/
--
To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org
Archive: http://lists.debian.org/20111023151317.ga26...@rivendell.home.ouaza.com
Re: Dealing with embedded javascript libraries
Raphael Hertzog, 2011-10-23 17:13:17 +0200 : [...] > Unfortunately, blindly replacing the file with a symlink can > create problems of its own. Upstream tested their application > with the version of the library that they ship. Using another > version might break things horribly, an example here: > https://bugs.launchpad.net/ubuntu/+source/wordpress/+bug/816962 I don't do much library packaging myself, but it was my understanding that versions of libraries that break API/ABI are meant to go in different binary packages, usually with a version number in the package name. Javascript doesn't have an ABI, but libraries written in that language are still libraries, aren't they? Roland. -- Roland Mas That's one of the good fings about not existin'; they leave you alone most of the time. -- in My Hero (Tom Holt) -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/87y5wbvi9s@mirexpress.internal.placard.fr.eu.org
Re: Dealing with embedded javascript libraries
On Sun, Oct 23, 2011 at 11:13 PM, Raphael Hertzog wrote: > And with javascript libraries, there's no failure at build time, > you only discover much later when something is not working... This is the crux of the issue and it is not specific to JavaScript libraries, anything that is interpreted has this issue. I'm pretty sure I've seen API breakage in libraries implemented in Python for example. More automated and manual testing can help here I guess. Also hopefully maintainers are using the packages they maintain and will therefore notice when they are broken by newer JavaScript libraries. > What are your thoughts on this topic? One of the other problems with embedded JavaScript libraries is that often only the pre-compiled/obfuscated/minified version is distributed, which would be a violation of DFSG item 2. -- bye, pabs http://wiki.debian.org/PaulWise -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/CAKTje6FGAQUVeSjpVHwUOfgtuSo1Ta5GzpUaCFoxK=thvq9...@mail.gmail.com
Re: Bug#646069: Debian and AutoMake
Hi there. This email isn't a criticism of rxtx - thanks Scott for accepting my patch. It's more of a starting point for issues with development on Debian and other distributions - see the last comment. On 23/10/11 15:08, Scott Howard wrote: clone 646069 -1 retitle -1 rxtx: make -dbg package for rxtx retitle 646069 rxtx: fails when "java.ext.dirs" system property contains more than one directory rxtx doesn't fail, it just fails to read any gnu.io.rxtx.properties file that may exist in one of these directories because the filename is constructed from the directory names joined with ":". I created a gnu.io.rxtx.properties file in one of the directories but I don't know what would happen (Java-wise) if there were more than one of these files - I haven't tested it. It will work without finding any gnu.io.rxtx.properties file. severity -1 wishlist thanks Thanks for the patch, great work. Separated this into two bugs 1) I'll apply the patch and forward it upstream. RXTX really isn't maintained much anymore upstream, but I'll at least share it on the mailing list. 2) We'll build with make -DDEBUG_VERBOSE and make a -dbg package for gdb. A "foo-dbg" package is one with debugging symbols for package "foo". Maybe "rxtx-verbose" would be a better name, with rxtx <- conflicts -> rxtx-verbose as an install option. As for the make uninstall, autotools does a poor job with uninstall targets [1] and upstream has a custom install target that just puts the RXTXcomm.jar into the same directory as the jvm and libtool install the native libraries. The Debian source package really isn't intended to be used to install outside of the debian packaging system. That's a shame if it's the case in general. I certainly wouldn't have considered using dpkg-buildpackage given the turnaround I wanted with the source code changes. fakeroot debian/rules binary didn't work either. I've got some packages in SourceForge. You can get them via http://sourceforge.net/users/philipashmore You can build Debian/Ubuntu packages with make debian You can install/uninstall them with make install make uninstall using the makefile in the package root that wraps automake. You will need to use "sudo" if you want to install them in a system directory, like /usr or /usr/local. If you've git-cloned a package repository you can also do things like make git branch=a.b.c distcheck make git tag=a.b.c release debian The "tag" variant is for branches you tag locally. If you want to build a package in a sandbox along with all its dependent v3c packages, you git clone the v3c packages into some directory writeable by you, then you can use v3c's "v3c-tryout" script v3c-tryout , as in ./v3c-tryout v3c-dcom-0.3.2-01 tryout-v3c-dcom and it will 1. git checkout the respective versions of the dependent packages as required by the version of the package you specified, into the sandbox dir. 2. build and install them to the sandbox dir in order. 3. create shell scripts to set up a (b/d)ash session inside the sandbox, to test the packages or run "make check", mess with the sources, uninstall... to uninstall, just delete the .jar and glibtool --mode=uninstall $(RM) list_of_targetlib Cheers, Scott [1] http://sourceware.org/autobook/autobook/autobook_109.html I'd really like to see something like this adapted by Debian and other distributions to save developers from reinventing the wheel or discovering that the package doesn't adhere to published documentation. What do you think? Philip -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/4ea43e15.6050...@philipashmore.com
Re: Dealing with embedded javascript libraries
On Sun, 23 Oct 2011, Paul Wise wrote: > > And with javascript libraries, there's no failure at build time, > > you only discover much later when something is not working... > This is the crux of the issue and it is not specific to JavaScript > libraries, anything that is interpreted has this issue. I'm pretty > sure I've seen API breakage in libraries implemented in Python for > example. > More automated and manual testing can help here I guess. yeap -- in python world the situation is generally better because unittesting is easy and popular, so for nearly all python packages I maintain I introduce package build-time unittesting against all supported versions of Python. That helps A LOT to assure correct performance especially having heavy backporting in mind (we usually build nearly for all recent releases of debian and ubuntu within neuro.debian.net project). Sure thing not all projects have 100% unittest coverage, also problems some times go unnoticed if it is arch 'all' package while has some arch specific operations (I/O, byteswapping) which do not get picked up unless they get excercised by unittests of dependent modules with packages of arch 'any'... but altogether there testing is indispensable to verify compatibility of API and just general QA. So, if only we could provide relevant automated basic testing to those JS-dependent packages... -- =--= Keep in touch www.onerussian.com Yaroslav Halchenko www.ohloh.net/accounts/yarikoptic -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/20111023165103.gc10...@onerussian.com
Re: Dealing with embedded javascript libraries
On Sun, Oct 23, 2011 at 5:29 PM, Paul Wise wrote: > On Sun, Oct 23, 2011 at 11:13 PM, Raphael Hertzog wrote: > > > And with javascript libraries, there's no failure at build time, > > you only discover much later when something is not working... > > This is the crux of the issue and it is not specific to JavaScript > libraries, anything that is interpreted has this issue. I'm pretty > sure I've seen API breakage in libraries implemented in Python for > example. > > I agree. So far it seems I have been pretty lucky with my package witty, which depends on jquery but upstream is not really happy Debian and Ubuntu replace the jquery version they have tested with. > What are your thoughts on this topic? One of the other problems with embedded JavaScript libraries is that > often only the pre-compiled/obfuscated/minified version is > distributed, which would be a violation of DFSG item 2. > > IMHO that should not be a problem provided that: - The JavaScript library is open source with the proper license, etc - Upstream is using the Javascript library "as is" (i. e. with no local modifications) Maybe README.Debian should mention "this package embeds the JavaScript library XXX which is available independently in package libjs-XXX (source package: libjs-XXX) :-? -- Pau Garcia i Quiles http://www.elpauer.org (Due to my workload, I may need 10 days to answer)
Re: Dealing with embedded javascript libraries
Roland Mas writes: > I don't do much library packaging myself, but it was my understanding > that versions of libraries that break API/ABI are meant to go in > different binary packages, usually with a version number in the package > name. Javascript doesn't have an ABI, but libraries written in that > language are still libraries, aren't they? Yes, I think so. The problem arises IMO because the conventions are different. Generally, programmers using Javascript libraries expect and encourage the direct inclusion of the library in the application, and there are very few cultural norms to discourage this code duplication. I would very much like that to change – that programmers should expect a single instance of a Javascript library to be useable across the OS, and that a Javascript library without a dependable ABI should be shunned by most application writers, and for applications to declare the library versions they expect in a standardised, automated way. But I don't know what to do to get there. -- \ “A ‘No’ uttered from deepest conviction is better and greater | `\ than a ‘Yes’ merely uttered to please, or what is worse, to | _o__) avoid trouble.” —Mohandas K. Gandhi | Ben Finney -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/87bot7l2hb@benfinney.id.au
