Hi Stuart and Marc,
Thanks a lot for responding.
Debugging the problem brought me to realize that GCC (both bundled
gcc/g++ and the egcc/eg++ port) adds a "-L/usr/lib" argument *in the
leading position* to LD.
Example:
$ echo "int main(){}">t.c; gcc -c -o t.o t.c; gcc -o t t.o -LMYDIRTEST -Wl,-v
collect2 version 4.2.1 20070719 (OpenBSD/x86-64 ELF)
/usr/bin/ld --eh-frame-hdr -e __start -Bdynamic -dynamic-linker
/usr/libexec/ld.so -L/usr/lib -o t /usr/lib/crt0.o /usr/lib/crtbegin.o
-LMYDIRTEST -L/usr/lib/gcc-lib/amd64-unknown-openbsd6.7/4.2.1 t.o -v -lgcc -lc
-lgcc /usr/lib/crtend.o
LLD 8.0.1 (compatible with GNU linkers)
As you see here, my "-LMYDIRTEST" argument ends up AFTER the -L/usr/lib
argument which gcc/egcc adds by itself.
This then has the effect that "-L/path/to/my/zlib/build/lib -lz" will
NOT attempt to pick my custom build EVER, but instead ALWAYS pick the
/usr/lib/libz.so .
This is a great headache for any attempt to make a local build of
software where zlib is a component!
And makes me curious, is this deliberate?
I tested clang on OpenBSD and on Linux and it does not have this
behavior. Also tested GCC on Linux, and it does not have this behavior.
I could imagine that the LEADING instead of TRAILING -L/usr/lib could
be some kind of security measure, however if that was the case, I'm
sure OpenBSD clang would have had the same, and it does not.
What is the motivation?
The likely follow-up question will then be, how do I disable this
behavior (and get -L/usr/lib as last LD argument as is the case on
clang everywhere and GCC on other platforms).
Below here for completeness response to Stuart's and Marc's previously
shared thoughts on the topic.
Overall after having given this question a couple of days of work, I
feel there is a certain asymmetry to this -L/usr/lib prefix behavior
and it could need a bit of discussion or at the very least
clarification.
Either OpenBSD's port changes this, or I need to build a local
patched port myself.
Also, and this is a bit beyond my GCC skills - the OpenBSD GCC port
contains patches that introduces this behavior in the first place,
does it not - if anyone has in depth understanding of GCC, please
feel free to point out which OpenBSD port patch file(s) it is that
causes this behavior, if so I could just "rm" that one in my local
ports repo and voillas my eg++ and egcc will work the way I want
out of the box.
Before ending this email I should also mention a partial mitigation
of the -L/usr/lib prefix problem: Namely, link in the zlib .so file
by mentioning it by .so filename on the GCC (LD) command line, instead
of via the "-lz" argument.
This works, however it will only work within a fully home-built custom
codebase, as any other code project that depends on zlib out there,
guaranteedly is hardcoded to link to zlib by "-lz". One would need to
patch all such projects' makefiles to link to it by "libz.so" instead
of "-lz", and that would be tedious and error-prone, good only as a
hack.
Thanks!
Bob
(I wrote:)
> > > I'll start with the detail problem, and discuss the reproduction at the
> > > bottom:
> > > This has brought me to the bizarre issue that the libpng build plainly
> > > refuses to link to my custom libz file /home/myuser/lib/libz.so.1 .
> > > Instead, it insists with linking to the OS' global
> > > /usr/lib/libz.so.5.0 .
(Stuart wrote:)
> > This is as expected really, it looks for the higher library version
> > number.
I evaluated the version number hypothesis, for instance by bumping
my zlib build's SONAME version number to 9999999.0 , and my build
would not get preference. I also talked to a GCC guy and he said
there is no such thing as version preference in respect of gcc -lNAME .
Have you actually seen this "pick highest version number" behavior on
any platform?
(Marc wrote:)
> Not really, it's more that you have to make sure to put your own directory
> before the system directory, which requires a bit more magic than just -L.
>
> (remember that linking will stop at the first directory with a satisfying
> library, and link with the highest version number found in there, which
> is fortunate for stuff like libtool!)
Marc, what you suggest here is correct and not correct - because
GCC on OpenBSD *always* adds -L/usr/lib in the leading position to LD's
arguments, then it does not matter in what order you as user add any
-L yourself.
(Re. libtool I don't know what you had on your mind.)
(I wrote:)
> > Hi,
> > I am trying to make a custom build of libpng in my home directory,
> > using a libz build that I made in my home directory also.
> > Both are latest version, libpng 1.6.37 same as OpenBSD's port
> > https://cvsweb.openbsd.org/ports/graphics/png/Makefile?rev=1.125&content-type=text/x-cvsweb-markup
> > and libz the latest one they published. For zlib it's 1.2.11
> > from http://zlib.net/ from 2017, OpenBSD does not have a port but
> > base bundles a 2009 version. Since 2009, significantly an export
> > "inflateReset2" has been added.
>
> As you have seen it is difficult to have library functions in one
> version in base and in another version built elsewhere (whether that's
> in $HOME or in ports). Ports only does this for libraries where there's
> really no other choice and where that has been done they're used very
> rarely (the port can then not depend on any libraries which pull in the
> library from base). Currently that is libbind (used only by asdig and
> zeek) and openssl (used as a static library by sslscan, and dynamic
> for nrpe and nsca-ng).
In my case I am making a local build of an own piece of software. The
purpose could as well be crosscompilation. Here, the fact that "-lz"
always means the OS' version is very much not desired.
(Stuart wrote:)
> You might get somewhere further with something like "-nostdlib
> -L/home/myuser/lib -L/usr/lib". You might need either rpath or change
> to using static libs.
>
> (not relevant to libpng which is in C, but if you try to build C++ software
> with GCC rather than Clang you will often run into problems with incompatible
> standard libraries too).
"-nostdlib" indeed removes the -L/usr/lib .
That is, only on the GCC port on -current, which incorporates this
patch:
https://cvsweb.openbsd.org/cgi-bin/cvsweb/ports/lang/gcc/8/patches/patch-gcc_config_i386_openbsdelf_h.diff?r1=1.3&r2=1.4&f=h
I did however grep the GCC codebase for "nostdlib" and I see a bunch
of matches. I can't really assess what -nostdlib does really, this
would be a question for the GCC emailing list, however it appears to
me to be a deep setting that you don't want to use.
(Stuart wrote:)
> > What appears is that GCC ignores the "-L" search path given to it.
>
> -L adds, it doesn't overwrite.
Right so what I discovered was that -L adds too late.
(Stuart wrote:)
> Hopefully you're just doing this to use pngfix; if you try and replace
> the installed libpng with one built like this you're likely to end up
> with all sorts of problems.
(Right indeed, I am not trying to replace libpng for OpenBSD binary
packages - instead what I'm doing is build my own C-C++ software
project in my home directory as a local build, and for uniformity to
the project I like it to use my own local build of libz.)
