bug#73159: 30.0.90; uniscribe / harfbuzz are not initialized on Windows, resulting in fallback to gdi
I recently pulled the latest emacs-30 branch (f47297782bdb5e5a07e02f119c8013d11f7d7fae), and I'm building emacs using MSYS2 mingw64 on Windows. With a build on this branch, certain symbols (from the nerd-icons package) were rendering as boxes and not their actual symbol. I suspected that this was because gdi was being used, and indeed using describe-char on any character shows me a font starting with `gdi:`, indicating the uniscribe or harfbuzz are not being used. This is not the case on the emacs-29 branch, where I see `uniscribe:` instead (and the symbols render correctly). I was not able to get harfbuzz to load on that branch, which is why I was trying emacs-30 to see if something was different there. To investigate this, I first used procmon to look at the syscalls to see if anything was trying to load libharfbuzz-0.dll, and I did see it in the trace I took, however the callstack was an OS callstack, and not from where I expected (src\w32uniscribe.c:syms_of_w32uniscribe_for_pdumber). I rebuilt with debug symbols, and set a breakpoint in that function - it was called once, but `initialized` was false so the LoadLibrary calls never ran. I'm not too familiar with how pdumper works, or the emacs source in general, so I'm not sure what the actual root cause is here. I did a bit more debugging to see when `initialized` was set to true, and it does get set later during initialization. I noted that I do not have a .pdmp file in my installation direction, which it seems to be trying to load during startup. One more piece of information: attempting to yank any text results in the error: `Coding system is invalid or doesn't have an eol variant for dos line ends: nil` This error occurs when starting with -Q and attempting to yank any text. If I call `(set-selection-coding-system 'utf-16-le)`, it resolves the issue. However, I didn't have to do this in emacs-29, so this makes me think something has possibly broken with init on Windows, since I believe this is not supposed to be nil by default? I'm happy to assist by providing any additional info if needed! Thanks, Casey In GNU Emacs 30.0.90 (build 1, x86_64-w64-mingw32) of 2024-09-09 built on DESKTOP-EK25TL1 Windowing system distributor 'Microsoft Corp.', version 10.0.19045 System Description: Microsoft Windows 10 Pro (v10.0.2009.19045.4780) Configured using: 'configure -prefix=/e/dev/emacs-src --without-dbus --without-pop --with-native-compilation --with-wide-int --without-compress-install --with-json --with-tree-sitter --without-imagemagick 'CFLAGS=-O2 -mtune=native -march=native -fomit-frame-pointer -ftree-vectorize -Wno-error=implicit-function-declaration -pipe -g' PKG_CONFIG_PATH=/mingw64/lib/pkgconfig:/mingw64/share/pkgconfig' Configured features: ACL GIF GMP GNUTLS HARFBUZZ JPEG LCMS2 LIBXML2 MODULES NATIVE_COMP NOTIFY W32NOTIFY PDUMPER PNG RSVG SOUND SQLITE3 THREADS TIFF TOOLKIT_SCROLL_BARS TREE_SITTER WEBP XPM ZLIB (NATIVE_COMP present but libgccjit not available) Important settings: value of $LANG: en_US.UTF-8 locale-coding-system: cp1252 Major mode: Fundamental Minor modes in effect: tooltip-mode: t global-eldoc-mode: t show-paren-mode: t electric-indent-mode: t mouse-wheel-mode: t tool-bar-mode: t menu-bar-mode: t file-name-shadow-mode: t global-font-lock-mode: t blink-cursor-mode: t minibuffer-regexp-mode: t buffer-read-only: t line-number-mode: t indent-tabs-mode: t transient-mark-mode: t auto-composition-mode: t auto-encryption-mode: t auto-compression-mode: t Load-path shadows: None found. Features: (shadow sort mail-extr emacsbug message mailcap yank-media puny dired dired-loaddefs rfc822 mml mml-sec password-cache epa derived epg rfc6068 epg-config gnus-util text-property-search time-date subr-x mm-decode mm-bodies mm-encode mail-parse rfc2231 mailabbrev gmm-utils mailheader cl-loaddefs cl-lib sendmail rfc2047 rfc2045 ietf-drums mm-util mail-prsvr mail-utils rmc iso-transl tooltip cus-start cconv eldoc paren electric uniquify ediff-hook vc-hooks lisp-float-type elisp-mode mwheel touch-screen dos-w32 ls-lisp disp-table term/w32-win w32-win w32-vars term/common-win tool-bar dnd fontset image regexp-opt fringe tabulated-list replace newcomment text-mode lisp-mode prog-mode register page tab-bar menu-bar rfn-eshadow isearch easymenu timer select scroll-bar mouse jit-lock font-lock syntax font-core term/tty-colors frame minibuffer nadvice seq simple cl-generic indonesian philippine cham georgian utf-8-lang misc-lang vietnamese tibetan thai tai-viet lao korean japanese eucjp-ms cp51932 hebrew greek romanian slovak czech european ethiopic indian cyrillic chinese composite emoji-zwj charscript charprop case-table epa-hook jka-cmpr-hook help abbrev obarray oclosure cl-preloaded button loaddefs theme-loaddefs faces cus-face macroexp files window text-properties overlay sha1 md5 base64 format env code-pages mule custom widget keymap hashtable-print-readable backquote threads w32notify w32 lcm
bug#73159:
This actually seems to be caused by the pdmp failing to load.
`make install` installs the pdmp file as `EMACS_PDMP =
`./src/emacs${EXEEXT} --fingerprint`.pdmp`. However, this is not the
filename that emacs tries to load - it attempts to load it from the wrong
directory, which fails.
That seems to be why `initialized` is false, and why uniscribe / harfbuzz
is not loaded. If I rename the emacs-.pdmp file to emacs.pdmp,
and place it next to emacs.exe everything works as expected (and emacs
loads a lot faster!).
I did a procmon dump to see what .pdmp files emacs.exe is trying to load,
and it attempts these locations:
- E:\dev\emacs-src\bin\emacs.pdmp
-
E:\dev\emacs-src\libexec\emacs\30.0.50\x86_64-w64-mingw32\emacs-ef314e5e05618ae9e98f90769b2adce721d1b3bac00e305e61b4d457b0a4.pdmp
- E:\dev\emacs-src\libexec\emacs\30.0.50\x86_64-w64-mingw32\emacs.pdmp
The thing is, the emacs version is 30.0.90, not 30.0.50.
A strings dump of emacs.exe:
strings emacs.exe | grep 30.0
%emacs_dir%/libexec/emacs/30.0.50/x86_64-w64-mingw32
$Id: GNU Emacs 30.0.90 (x86_64-w64-mingw32 ACL GIF GMP GNUTLS HARFBUZZ JPEG
LCMS2 LIBXML2 MODULES NATIVE_COMP NOTIFY W32NOTIFY PDUMPER PNG RSVG SOUND
SQLITE3 THREADS TIFF TOOLKIT_SCROLL_BARS TREE_SITTER WEBP XPM ZLIB) $
30.0.90
%emacs_dir%/share/emacs/30.0.90/lisp
%emacs_dir%/share/emacs/30.0.90/site-lisp;%emacs_dir%/share/emacs/site-lisp
/30.0.90/lisp/
%emacs_dir%/share/emacs/30.0.90/etc
%emacs_dir%/libexec/emacs/30.0.90/x86_64-w64-mingw32
%emacs_dir%/libexec/emacs/30.0.50/x86_64-w64-mingw32
So it seems something is going wrong with setting the path. I noticed that
exec/configure.ac has:
AC_INIT([libexec], [30.0.50], [bug-gnu-emacs@gnu.org], [],
[https://www.gnu.org/software/emacs/])
I tried changing this to 30.0.90 and re-building (using make bootstrap),
but the resulting binary still has 30.0.50 in it.
bug#73159: 30.0.90; uniscribe / harfbuzz are not initialized on Windows, resulting in fallback to gdi
> How come your LANG is set to en_US.UTF-8? Did you set this in the > environment or something. Using UTF-8 as the default encoding on > Windows is not a good idea. It seems that the msys2 .profile has `export LANG=$(locale -uU)`, and that returns en_US.UTF-8 for me. > Please look at src/epaths.h and see how PATH_EXEC is defined there. It is indeed #define PATH_EXEC "%emacs_dir%/libexec/emacs/30.0.50/x86_64-w64-mingw32" src/epaths.in has #define PATH_EXEC "/usr/local/libexec/emacs" I had been running configure and make in a subdirectory. If I run them in the top-level directory, then it does update PATH_EXEC to the correct version. I think I made the wrong assumption that running configure in a subdirectory would leave the main source clean. Thank you for your help debugging this! - Casey On Tue, Sep 10, 2024 at 8:33 AM Eli Zaretskii wrote: > > From: Casey Banner > > Date: Tue, 10 Sep 2024 01:50:43 -0400 > > > > I did a procmon dump to see what .pdmp files emacs.exe is trying to > load, and it attempts these locations: > > > > - E:\dev\emacs-src\bin\emacs.pdmp > > - > > > E:\dev\emacs-src\libexec\emacs\30.0.50\x86_64-w64-mingw32\emacs-ef314e5e05618ae9e98f90769b2adce721d1b3bac00e305e61b4d457b0a4.pdmp > > > > - E:\dev\emacs-src\libexec\emacs\30.0.50\x86_64-w64-mingw32\emacs.pdmp > > > > The thing is, the emacs version is 30.0.90, not 30.0.50. > > > > A strings dump of emacs.exe: > > > > strings emacs.exe | grep 30.0 > > %emacs_dir%/libexec/emacs/30.0.50/x86_64-w64-mingw32 > > $Id: GNU Emacs 30.0.90 (x86_64-w64-mingw32 ACL GIF GMP GNUTLS HARFBUZZ > JPEG LCMS2 LIBXML2 > > MODULES NATIVE_COMP NOTIFY W32NOTIFY PDUMPER PNG RSVG SOUND SQLITE3 > THREADS TIFF > > TOOLKIT_SCROLL_BARS TREE_SITTER WEBP XPM ZLIB) $ > > 30.0.90 > > %emacs_dir%/share/emacs/30.0.90/lisp > > > %emacs_dir%/share/emacs/30.0.90/site-lisp;%emacs_dir%/share/emacs/site-lisp > > /30.0.90/lisp/ > > %emacs_dir%/share/emacs/30.0.90/etc > > %emacs_dir%/libexec/emacs/30.0.90/x86_64-w64-mingw32 > > %emacs_dir%/libexec/emacs/30.0.50/x86_64-w64-mingw32 > > > > So it seems something is going wrong with setting the path. I noticed > that exec/configure.ac has: > > > > AC_INIT([libexec], [30.0.50], [bug-gnu-emacs@gnu.org], [], > > [https://www.gnu.org/software/emacs/]) > > exec/configure.ac is not used in the MinGW build (but the above is a > bug nonetheless, so I will fix it shortly). > > I cannot reproduce this result: > > > strings emacs.exe | grep 30.0 > > %emacs_dir%/libexec/emacs/30.0.50/x86_64-w64-mingw32 > > Please look at src/epaths.h and see how PATH_EXEC is defined there. > > > I tried changing this to 30.0.90 and re-building (using make bootstrap), > but the resulting binary still has 30.0.50 > > in it. > > As expected: the MinGW build doesn't use that file. You should > regenerate or update src/epaths.h. Perhaps touch src/epaths.in and > then re-run "make" in the top-level directory of the source tree. > >
bug#73159: Fwd: bug#73159: 30.0.90; uniscribe / harfbuzz are not initialized on Windows, resulting in fallback to gdi
> OK, so that's one mystery down. We are left with the HarfBuzz issue; > please answer the questions I asked about that. Ah, yes sorry - I acquired the DLLs from the mingw64/mingw-w64-x86_64-harfbuzz package. $ objdump -x /mingw64/bin/libharfbuzz-0.dll | grep "DLL Name": DLL Name: libgcc_s_seh-1.dll DLL Name: GDI32.dll DLL Name: KERNEL32.dll DLL Name: msvcrt.dll DLL Name: RPCRT4.dll DLL Name: libstdc++-6.dll DLL Name: USER32.dll DLL Name: USP10.dll DLL Name: libfreetype-6.dll DLL Name: libglib-2.0-0.dll DLL Name: libgraphite2.dll DLL Name: libintl-8.dll harfbuzz does load correctly now that the pdmp is being loaded correctly - I believe because now it passes the `intialized` check and so actually attempts to load them. describe-char shows me: harfbuzz:-outline-Iosevka Fixed SS02-regular-normal-normal-mono-14-*-*-*-c-*-iso8859-1 (#x56) On Tue, Sep 10, 2024 at 2:47 PM Eli Zaretskii wrote: > > From: Casey Banner > > Date: Tue, 10 Sep 2024 14:31:07 -0400 > > Cc: 73...@debbugs.gnu.org > > > > > How come your LANG is set to en_US.UTF-8? Did you set this in the > > > environment or something. Using UTF-8 as the default encoding on > > > Windows is not a good idea. > > > > It seems that the msys2 .profile has `export LANG=$(locale -uU)`, and > that returns en_US.UTF-8 for me. > > I don't recommend running Emacs from the MSYS2 Bash prompt. Instead, > run it from a desktop shortcut or pin it to the task bar and run from > there. > > > > Please look at src/epaths.h and see how PATH_EXEC is defined there. > > > > It is indeed #define PATH_EXEC > "%emacs_dir%/libexec/emacs/30.0.50/x86_64-w64-mingw32" > > > > src/epaths.in has #define PATH_EXEC "/usr/local/libexec/emacs" > > > > I had been running configure and make in a subdirectory. If I run them > in the top-level directory, > > then it does update PATH_EXEC to the correct version. I think I made the > wrong assumption that > > running configure in a subdirectory would leave the main source clean. > > > > Thank you for your help debugging this! > > OK, so that's one mystery down. We are left with the HarfBuzz issue; > please answer the questions I asked about that. >
