Re: coreutils-6.2: various runtime problems on Darwin-8.7.0 HFS+ (including attachment this time)

[sci-fi]

[let me see if gmane lets me reply to posts]

On 2006-09-25 08:10:03 -0500, Jim Meyering <[EMAIL PROTECTED]> said:


<[EMAIL PROTECTED]> wrote:

I am new to this list, but not at all new to building and using
open source projects.  (Please CC me if needed as I haven't yet
joined this list.  Or I think I can access and post via
news.gmane.org with regular newsreaders.)

I built coreutils-6.2 on Darwin-8.7.0 ("Tiger" 10.4.7 and the
latest XCode-2.4).  I actually have installed coreutils-5.97 to
compare with.

5.97 seems to work fine, but 6.2 seems to have many things wrong.

Most notably the new 'ls' in 6.2 is acting very very strange.
We get double listings in 'long' mode of every file, with
exactly the first set saying "No such file or directory",
including the dot-dirs.  The second set lists each file/dir as
expected.

I am attaching a sample to show you what I mean here.  Please
find the archive named 'coreutils-6.2-bugs.tar.bz2'.  It should
expand to create these files:

-rw-r--r-- 1 root wheel 1957313 Sep 25 00:43 config.log
-rw-r--r-- 1 root wheel   31244 Sep 25 00:43 config_consolelog.txt
-rw-r--r-- 1 root admin   29725 Sep 25 00:51 ls_whoops_consolelog.txt
-rw-r--r-- 1 root wheel   67706 Sep 25 00:45 make_consolelog.txt
-rw-r--r-- 1 root wheel   55908 Sep 25 00:49 makecheck_consolelog.txt


Thank you for the complete bug report!


I'm obviously quite worried about releasing this code to Darwin
users, so I thought I better do a bit of hollarin' here.  ;)


You're right to be cautious.
The problems involving ls, cp, and install are all due
to the fact that Darwin's acl_get_file function (used
in coreutils' lib/acl.c) is not behaving as advertised:
it's returning NULL and setting errno to ENOENT for a file
that *does* exist.  However, the man page says it does that only
for files that do not exist.  When a function like that reports
a failure, coreutils programs report it and exit nonzero, and
that's why you're seeing so many diagnostics and test failures.


This might explain other ACL problems I've read about at other sites.
I am attempting to sniff around Apple's maillists to see if this is a
known bug.  We are getting browser "disconnect" errors ATM while
waiting for their archives to load-up.  ~sigh~

I am also trying to pull the xnu project from 10.4.7.ppc and take a
peek that way.  I don't know if xnu can be fetch via CVS there.

Unfortunately there is no way us mere mortals (free ADC accounts) are
allowed to search their Bug Submissions database.  One can only keep
track of bugs one has filed under one's id there.


Using the very latest version from coreutils and gnulib,
when I turn off ACL support altogether (comment out the USE_ACL
definition in lib/config.h), all of the tests pass except tests/dd/misc.

That failure appears to be due to another Darwin-specific bug:
The open flag, O_NOFOLLOW is defined, yet open doesn't honor the
semantics we've come to expect for that flag.  That makes the test
of dd's new iflag=nofollow option fail.
The only mitigating factor is that O_NOFOLLOW is not yet documented in
their open man page:  the definition may have slipped into their
system headers before library support was completed.


btw fwiw 'apropos O_NOFOLLOW' shows 'nothing appropriate' here,
supporting your note about it not being documented.
otoh 'apropos acl_get_file' does show the manpage available.


Unfortunately, O_NOFOLLOW (when present) is an important key to avoiding
certain types of race conditions.  Defining the symbol yet letting
open ignore it is *bad*.  Here's a small program to demonstrate the bug.
Run the command in the comment.  If it prints "bug", then your system's
open function is buggy.

$ cat o_nofollow.c
/* rm -f s f; cc o_nofollow.c && { touch f; ln -s f s; ./a.out || echo 
bug; } */

# define _GNU_SOURCE 1
#include 
#include 
#include 
int
main ()
{
  int fd = open ("s", O_RDONLY|O_NOFOLLOW);
  return 0 < fd;
}



Yep, it prints 'bug'.  And this is with coreutils-5.97 btw.


However, the good news is that this led me to discover a bug in gnulib's
test for just this type of brokenness.  Its fcntl_h module tests for a
nonfunctional O_NOFOLLOW, but the test had a typo.  I've just checked
in a fix for that.

So now, all tests now pass, using the very latest, and with the hack
to disable ACL support.  I suppose we'll need an added run-test
to detect the acl_get_file brokenness.


Thank you for the detailed explanations.  If nothing shows up on the
Darwin maillists (or I give up waiting ;) ), I will file a Bug at ADC.
Would you mind if I quote your entire message here, please?

Let me also check their CVS and/or new MacOSForge sites (the latter has
been down for nearly a week, still is as of this typing ~double~sigh~).

I will pull coreutils and gnulib from CVS also and report back.

Thank you again.


--

here's my latest make check log on Darwin (Re: now getting a build error with coreutils-cvs (Re: coreutils-6.2: various runtime problems on Darwin-8.7.0 HFS+ (including attachment this time)))

[sci-fi]

(Unison won't let me attach a file to a message, nor will it let an 
upload come with a message, too lazy to switch to Thoth right now 
;) so I'll do this via e-mail)

I cvs update'd within the last half-hour.  Builds seem to go fine.  
I ran make -k check with all the RUN*TESTS enabled I could dig-up.  
Attached should be a file named 'cucvs_makecheck.log.bz2' which is 
the tee'd capture from the check.

I think these failures are 100% explainable on Darwin.  Also, the 
filesystem bugs causing the long ls and other problems seem 
successfully bypassed now.  So I feel we're very close for release. 
 I'll do a make install and use it for real for a while.  :)

Thank you.




cucvs_makecheck.log.bz2
Description: BZip2 compressed data

Re: MacOSX Leopard gcc-4.2 build problems with coreutils-7.0 and its gnulib

[sci-fi]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1


Hi,

Thank you-all for responding.

Just a quick reply from me for now, I'll get deeper into it later
as my head stops swimming (too much going on right now lol).

Before you-all branch-off too much, please understand I'm
strictly trying to use the (llvm-)gcc-4.2 family that Apple has
provided with the latest XCode/iPhone SDKs.  The only other
choice is the gcc-4.0 family in which Apple has support for all
four -arch options -- this means we can't use the gcc-3.3 family
for these builds.  (The Tiger XCode does not have full 64-bit
support.)  But since I have only a C2D iMac to use for testing
here (my PPC gear got "stolen" a while back), I can only build &
test Universal Binaries with the two Intel -arch options.

We are following Apple's Tech Note TN2137:
http://developer.apple.com/technotes/tn2005/tn2137.html
In particular:
"The --disable-dependency-tracking option to configure causes it
to not use gcc's built-in dependency generation code, which does
not work with multiple -arch targets."
I'm discovering this flag is being required for quite a lot of
projects.
(I'll only use MacPorts/Fink for clues when I'm stumped,
otherwise my builds are straight out of official src tarballs.)

Somehow since my original post on Sunday, I found
coreutils-7.0.59, which still fails building the test suite as I
reported.

I also used your git website to get snapshots of coreutils and
gnulib.  I told coreutils's bootstrap to point to my local gnulib
snapshot.  I think it worked correctly.  Still fails as reported.

Since the compiler is balking during building of the test suite,
I went ahead to install coreutils-7.0.59 anyway:

$ ls -al `which cp`
- -rwxr-xr-x 1 root admin 392176 2008-11-16 08:37 /usr/local/bin/cp

$ cp --version
cp (GNU coreutils) 7.0.59-4e38e6
Copyright (C) 2008 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later

This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.

Written by Torbjörn Granlund, David MacKenzie, and Jim Meyering.

$ file /usr/local/bin/cp
/usr/local/bin/cp: Mach-O universal binary with 2 architectures
/usr/local/bin/cp (for architecture i386):  Mach-O executable i386
/usr/local/bin/cp (for architecture x86_64):Mach-O 64-bit
executable x86_64

As for the missing @LIBINTL@, I believe adding -L/usr/local/lib
to LDFLAGS is letting some projects configure find it, even tho I
have specifically specified --with-lib{intl,iconv}=/usr/local
(yes both are built with the two -arch options).

As for "fixes not yet done", well, doesn't someone need to be
testing this kind of stuff before release?  ;)  See, that's my
pet peeve against using such things as MacPorts & Fink, because
they will not allow me to specify what compilers & environments
to use as a Developer needs (without tons of hacking of course;
it's just easier to manually build straight from tarballs).

Now FYI I am following the requisites (manually) to build
everything needed for the pan2 newsreader -- lots & lots of GNOME
stuff but not the full GUI system itself.  Just last night I
discovered the two current Glib2 releases have ltmain.sh scripts
that are in sad shape: I had to get libtool-2.2.6a and do the
autogen.sh to refresh things there, but still had to hack it to
report '.dylib' shared lib suffixes properly, which is probably
an indication that Glib2 configure isn't calling that function
quite right with newer libtools.  But I dunno, just an educated
guess after doing this for what 10+ years or so.  ;)  A few
projects had to fall back to using gcc-4.0 (as provided by Apple
in XCode-3.1.1 or iPhone-2.1).  So far, tho, at least we have
dual 32-&-64-bit support for the whole chain.

This is the general script/environment I'm using for these
builds:

#
export PATH="\
/usr/local/bin\
:/usr/X11R6/bin\
:/usr/local/sbin\
:/usr/bin\
:/usr/sbin\
:/bin\
:/sbin\
:/Developer/usr/bin\
:/Developer/Tools\
"
#
export CC=llvm-gcc-4.2
export CPP=llvm-cpp-4.2
export CXX=llvm-g++-4.2
export OBJC=${CC}
export OBJCPP=${CPP}
#
export CFLAGS=" -Os  -g  -mtune=core2 -march=core2 -
force_cpusubtype_ALL  -arch i386  -arch x86_64  -isysroot
/Developer/SDKs/MacOSX10.5.sdk "
export  CXXFLAGS="${CFLAGS}"
unset  CPPFLAGS
export OBJCFLAGS="${CFLAGS}"
#
export ACLOCAL_FLAGS=" -I /usr/local/share/aclocal  -I
/usr/share/aclocal "
#
export MACOSX_DEPLOYMENT_TARGET="10.5"
export SDKROOT="/Developer/SDKs/MacOSX10.5.sdk"
#
export LDFLAGS=" -arch i386  -arch x86_64 "
#
export LIBTOOL=/usr/local/bin/libtool
export LIBTOOLIZE=/usr/local/bin/libtoolize
#
export MANPATH="\
/usr/local/share/man\
:/usr/local/man\
:/usr/X11R6/man\
:/usr/man\
:/usr/share/man\
:/System/Library/Frameworks/JavaVM.framework/Versions/1.5.0/Home/man
\
"
#
export PKG_CONFIG_PATH="\
/usr/local/lib/pkgconfig\
:/usr/local/share/pkgconfig\
:/usr/X11R6/lib/pkgconfig\
:/usr/lib/pkgconfig\
"
# ( I really need to document how to c