Hello Mitch,
> I have been working on a high level of windows compatibility for several
> GNU and other related open source software without the traditional
> emulation layers/mingw requirements. Overall it has had great success in
> my mind, multiple dozens of GNU related packages compiled from the latest
> source on a daily basis in a CI with success generally with minimal changes
> to the base source. All compiling is done (by default) using MSVC's
> compiler, resulting in full visual studio support for debugging and trace
> support.
>
> At the core of many builds are the gnulib "improvements".
> https://github.com/mitchcapper/WIN64LinuxBuild/blob/master/repo_notes/gnulib_README.md
> outlines the various gnulib changes I have made.
These changes are welcome. I can see that they represent a lot of work.
For GNU packages, it is important to support mingw as an alternative to MSVC:
we don't want to force people to use proprietary toolchains. (They can use
proprietary tools, but that is their own choice.) In practice, most code
that works with MSVC also works with mingw. It's just a matter of writing
the proper #if conditions:
- defined _WIN32 && !defined __CYGWIN__
for native Windows
- defined _MSC_VER
for the MSVC header files, together with either MSVC or clang as compiler,
- defined __MINGW32__
for mingw.
> There are some poor hacks
> used, and my lack of deep gnulib knowledge means I likely have done some
> things quite wrong.
>From a quick look at your patches, most of the things look like done with
the correct approach.
> Of all the projects I have worked on Windows improvements for, I feel
> Gnulib is the most critical to see adoption.
I agree. Integrating patches for GNU tar or GNU wget/wget2 makes sense only
after patches for Gnulib were integrated.
> These changes have allowed for things like tar to be built on windows with
> a bunch of added gnulib modules and maybe only a dozen lines of code
> changes (minus some of the fork/exec logic that was longer).
When a package has complicated fork/exec logic, generally it's advisable to
use the posix_spawn facility when possible. This provides not only portability
to native Windows; it is also a speedup on GNU/Linux (because glibc implements
posix_spawn through optimized clone() syscalls).
I don't know whether this is applicable to GNU tar, though. It's too long ago
that I looked.
> It is able to
> be used with external compressors (without hangs), windows paths, relative
> and absolute symbolic link archiving and extracting (cross platform)
> something I haven't seen in a native tar build before.
Support for native Windows symlinks is something that Gnulib does not have
so far. The complexity of the several kinds of "reparse points" is not
something that invites developers to support it...
> Any feedback appreciated, especially with some of the issues/coding
> practices I am aware of the guidelines at
> https://www.gnu.org/software/gnulib/ which most of the changes I feel meet
> (minus things like colorization potentially). From a coding standard
> specifically following: https://www.gnu.org/prep/standards/standards.html
> there are some things I would change for any code officially submitted up:
> - Removing the ifdefs changing to #if wherever possible
> - Will run through the generalized clang formatter for whitespace / etc
I'm not sure an automatic tool (like GNU indent or clang-format) supports
our coding style well. Things to watch out for:
- Placement of braces,
- space before opening parenthesis in function and macro calls,
- Use spaces, not tabs, for indentation (in Gnulib; other GNU packages
handle this differently).
> *Gating w/ CYGWIN support*
> While I have gone back and tried to gate the changes with proper windows
> only checks I generally use #ifdef _WIN32 and I don't know the effect
> that would have on other windows systems (ie cygwin).
In Gnulib, we use
#if defined _WIN32 && !defined __CYGWIN__
It's mostly because many years ago, it was customary to build Cygwin
programs with -D_WIN32. I don't know whether that's still relevant.
We are keeping it this way because it's a bit safer and because it makes
it clear which code will be compiled on Cygwin.
> I know very
> little about cygwin dev and I am not sure what should be gated to avoid
> cygwin picking it up. Some things might already be pollyfilled by
> mingw/gcc generally I try to use the most advanced implementation for
> changes but it is possible they polyfill something better that I didn't
> notice.
Generally, Cygwin should be treated like a Unix platform. Over the years,
Cygwin has added many APIs, so there is hardly any need any more to use
Windows API when compiling for the Cygwin platform.
> *Proper ways to detect when gnulib polyfills have been used*
> Then there are issues less about gnulib and more about things that use it
> and how they do feature detection. There is plenty of 3rd party code that
> may have code with HAVE_SYMLINK gates in place and only try to symlink if
> defined.
Yes, that's an adaptation that needs be made in the packages: When the
package requests a polyfill for function XY (and this polyfill is always
available), tests of HAVE_XY should be simplified to 1. Developers like
this, because it makes their code easier.
Only in a few cases we define a macro such as 'GNULIB_overrides_fprintf'.
I'll reply to the rest of your mail later.
Bruno
