What is the difference between Microsoft and Apple? When Microsoft adopts some piece of free software, they act according to the motto "embrace then extend" [1]. When Apple adopts some piece of free software, they apparently act according to the principle "embrace then brick".
The macOS 14 machines on GitHub are running darwin23.4.0, which per [2]
means macOS 14.4. According to [3] it uses the implementation tagged
libiconv-99 from [4]. This implementation was based on FreeBSD's native
iconv in summer 2023, with many Apple-specific modification since then.
Some of these modifications apparently attempt to make their iconv()
similar to the GNU implementations, in order to fix breakage in
applications such as mutt. In particular, they added three iconvctl()
operations that ought to be able to change how unconvertible input
is handled:
ICONV_SET_TRANSLITERATE
ICONV_SET_DISCARD_ILSEQ
ICONV_SET_ILSEQ_INVALID
I did some extensive testing of this implementation with unconvertible input;
see the attached test-iconv.zip file. The result:
* Transliteration is always enabled, even when not requested.
* When transliteration happens, it is not reflected in the return code
from iconv(), violating POSIX.
* None of the three iconvctl() operations (that are supposed to have an
effect on this) have any effect. I tested all 8 combinations, with
// suffixes on iconv_open as well as with iconvctl().
There is no point in supporting such a severely buggy implementation.
We can just tell users to install GNU libiconv (from Homebrew or from source).
[1] https://en.wikipedia.org/wiki/Embrace,_extend,_and_extinguish
[2] https://en.wikipedia.org/wiki/Darwin_(operating_system)#Darwin_20_onwards
[3] https://opensource.apple.com/releases/
[4] https://github.com/apple-oss-distributions/libiconv
2024-05-22 Bruno Haible <br...@clisp.org>
iconv: Reject the broken macOS 14.4 iconv implementation.
Reported by Daniel Collins <solemnwarn...@solemnwarning.net> at
<https://savannah.gnu.org/bugs/?65686>.
* m4/iconv.m4 (AM_ICONV_LINK): In the test "for working iconv", test
against the macOS 14.4 iconv bug.
* doc/posix-functions/iconv.texi: Document the bug.
diff --git a/doc/posix-functions/iconv.texi b/doc/posix-functions/iconv.texi
index c77b4492dd..29228273aa 100644
--- a/doc/posix-functions/iconv.texi
+++ b/doc/posix-functions/iconv.texi
@@ -18,7 +18,7 @@
@itemize
@item
Failures are not distinguishable from successful returns on some platforms:
-AIX 5.1..7.2, Solaris 10.
+macOS 14.4, AIX 5.1..7.2, Solaris 10.
@item
A buffer overrun can occur on some platforms:
AIX 6.1..7.1.
diff --git a/m4/iconv.m4 b/m4/iconv.m4
index 314679735d..b414bfca62 100644
--- a/m4/iconv.m4
+++ b/m4/iconv.m4
@@ -1,5 +1,5 @@
# iconv.m4
-# serial 27
+# serial 28
dnl Copyright (C) 2000-2002, 2007-2014, 2016-2024 Free Software Foundation,
dnl Inc.
dnl This file is free software; the Free Software Foundation
@@ -75,7 +75,7 @@ AC_DEFUN([AM_ICONV_LINK]
if test "$am_cv_func_iconv" = yes; then
AC_CACHE_CHECK([for working iconv], [am_cv_func_iconv_works], [
dnl This tests against bugs in AIX 5.1, AIX 6.1..7.1, HP-UX 11.11,
- dnl Solaris 10.
+ dnl Solaris 10, macOS 14.4.
gl_saved_LIBS="$LIBS"
if test $am_cv_lib_iconv = yes; then
LIBS="$LIBS $LIBICONV"
@@ -114,6 +114,35 @@ AC_DEFUN([AM_ICONV_LINK]
iconv_close (cd_utf8_to_88591);
}
}
+ /* Test against macOS 14.4 bug: Failures are not distinguishable from
+ successful returns.
+ POSIX:2018 says: "The iconv() function shall ... return the number of
+ non-identical conversions performed."
+ But here, the conversion always does transliteration (the suffixes
+ "//TRANSLIT" and "//IGNORE" have no effect, nor does iconvctl()) and
+ does not report when it does a non-identical conversion. */
+ {
+ iconv_t cd_utf8_to_88591 = iconv_open ("ISO-8859-1", "UTF-8");
+ if (cd_utf8_to_88591 != (iconv_t)(-1))
+ {
+ static ICONV_CONST char input[] = "\305\202"; /* LATIN SMALL LETTER L
WITH STROKE */
+ char buf[10];
+ ICONV_CONST char *inptr = input;
+ size_t inbytesleft = strlen (input);
+ char *outptr = buf;
+ size_t outbytesleft = sizeof (buf);
+ size_t res = iconv (cd_utf8_to_88591,
+ &inptr, &inbytesleft,
+ &outptr, &outbytesleft);
+ /* Here:
+ With glibc, GNU libiconv (including macOS up to 13): res ==
(size_t)-1, errno == EILSEQ.
+ With musl libc, NetBSD 10, Solaris 11: res == 1.
+ With macOS 14.4: res == 0, output is "l". */
+ if (res == 0)
+ result |= 2;
+ iconv_close (cd_utf8_to_88591);
+ }
+ }
/* Test against Solaris 10 bug: Failures are not distinguishable from
successful returns. */
{
@@ -130,7 +159,7 @@ AC_DEFUN([AM_ICONV_LINK]
&inptr, &inbytesleft,
&outptr, &outbytesleft);
if (res == 0)
- result |= 2;
+ result |= 4;
iconv_close (cd_ascii_to_88591);
}
}
@@ -149,7 +178,7 @@ AC_DEFUN([AM_ICONV_LINK]
&inptr, &inbytesleft,
&outptr, &outbytesleft);
if (res != (size_t)(-1) || outptr - buf > 1 || buf[1] != (char)0xAD)
- result |= 4;
+ result |= 8;
iconv_close (cd_88591_to_utf8);
}
}
@@ -169,7 +198,7 @@ AC_DEFUN([AM_ICONV_LINK]
&inptr, &inbytesleft,
&outptr, &outbytesleft);
if ((int)res > 0)
- result |= 8;
+ result |= 16;
iconv_close (cd_88591_to_utf8);
}
}
@@ -187,7 +216,7 @@ AC_DEFUN([AM_ICONV_LINK]
iconv_t cd4 = iconv_open ("utf8", "eucJP");
if (cd1 == (iconv_t)(-1) && cd2 == (iconv_t)(-1)
&& cd3 == (iconv_t)(-1) && cd4 == (iconv_t)(-1))
- result |= 16;
+ result |= 32;
if (cd1 != (iconv_t)(-1))
iconv_close (cd1);
if (cd2 != (iconv_t)(-1))
<<attachment: test-iconv.zip>>
