Re: GCC testing on FreeBSD
Hi Jonathan,
On Fri, 26 Apr 2024, Jonathan Wakely wrote:
> How are you testing on FreeBSD?
>
> When I build GCC trunk on FreeBSD 14.0 and try to run the libstdc++
> testsuite it fails due to lots of these errors:
>
> Excess errors:
> /usr/local/bin/ld: /tmp//ccev946q.o: relocation R_X86_64_32 against
> symbol `_ZTIN10__cxxabiv115__forced_unwindE@@CXXABI_1.3.2' can not be
> used when making a PDE object; recompile with -fPIE
> /usr/local/bin/ld: failed to set dynamic section sizes: bad value
my first reaction was to recommend using binutils instead of /usr/bin/ld
which is LLD 16.0.6 or similar (since a while ago FreeBSD switched to that
toolchain as part of the base system).
My nightly tester has been using GNU ld since
# 2012-03-11 Configure using --with-as=$localbase/bin/as and
# --with-ld=$localbase/bin/ld on *.freebsd.org.
in the script invoked by cron, even before FreeBSD made that switch.
Seeing /usr/local/bin/ld in the error message it appears you are doing
that already, though?
> Which suggests that -fPIE is missing from the default test flags.
>
> Have you seen this? Do you do something locally to work around it?
All I have in terms of adjustments to the FreeBSD systems I build on via
that script are the following
CONFIGUREFLAGS="--with-gmp=$LOCALBASE --with-as=$LOCALBASE/bin/as
$CONFIGUREFLAGS"
LD_LIBRARY_PATH=$LD_LIBRARY_PATH:$LOCALBASE/lib
PATH=$LOCALBASE/bin:$PATH
where LOCALBASE looks like it would be /usr/local in your case. Nothing
explicit around PIE there.
Looking at the logs of a serialized build I triggered manually where I
remove the --with-ld configure option, I see
checking linker PIE support with copy reloc... no
checking for -fno-PIE option... yes
checking for -no-pie option... yes
and then build invocations like
c++ -std=c++11 -fno-PIE -c ...
during all-stage1-target-libgcc which ultimately fails with - mystery?! -
ld: error: unable to find library -lc
collect2: error: ld returned 1 exit status
gmake[3]: *** [Makefile:1005: libgcc_s.so] Error 1
(Disclaimer: all my tests on FreeBSD 13.2, not FreeBSD 14 as in your
case.)
Looking at the lang/gcc* ports that I maintained for two decades until
Lorenzo (copied now) kindly took them over two years ago I see the
following change among others for newer versions:
% git show b6a5871a0cf40
commit b6a5871a0cf40dfc194217704e2dc03e2e91fb62
Author: Lorenzo Salvadore
Date: Fri Feb 3 20:12:49 2023 +0100
lang/gcc10: Mark PIE_UNSAFE
Building the port with WITH_PIE fails if the BOOTSTRAP option is
enabled. Mark PIE_UNSAFE when this option is enabled until a better
solution is found.
PR: 268901
where https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=268901 is the PR
referenced.
PIE_UNSAFE=yes in a port's Makefile (the equivalent to a spec file in RPM
land) is used as follows in ports/Mk/Features/pie.mk:
. if !defined(PIE_UNSAFE)
PIE_CFLAGS?=-fPIE -fPIC
CFLAGS+=${PIE_CFLAGS}
CXXFLAGS+= ${PIE_CFLAGS}
LDFLAGS+= -pie
STATIC_PIE_ARGS+= -static-pie
. endif
You are not using the Ports Infrastructure, I believe, so the above does
not directly apply, may provide some additional background, though?
Hope this helps - and please chime in Lorenzo and Andreas!
Gerald
Re: GCC testing on FreeBSD
Hi Gerald, > On Fri, 26 Apr 2024, Jonathan Wakely wrote: >> How are you testing on FreeBSD? >> >> When I build GCC trunk on FreeBSD 14.0 and try to run the libstdc++ >> testsuite it fails due to lots of these errors: >> >> Excess errors: >> /usr/local/bin/ld: /tmp//ccev946q.o: relocation R_X86_64_32 against >> symbol `_ZTIN10__cxxabiv115__forced_unwindE@@CXXABI_1.3.2' can not be >> used when making a PDE object; recompile with -fPIE >> /usr/local/bin/ld: failed to set dynamic section sizes: bad value > > my first reaction was to recommend using binutils instead of /usr/bin/ld > which is LLD 16.0.6 or similar (since a while ago FreeBSD switched to that > toolchain as part of the base system). right: some time ago I tried bootstrapping trunk on FreeBSD 14.0 myself, which was a very rough journey unfortunately. I've filed a number of PRs for the issues found. They should all be referenced in PR target/112959install.tex needs updates on FreeBSD which is primarily a call for documention so the next unwary non-FreeBSD developer won't run into all the same known issues again. Rainer -- - Rainer Orth, Center for Biotechnology, Bielefeld University
Re: GCC testing on FreeBSD
On Sunday, April 28th, 2024 at 12:24, Gerald Pfeifer wrote: > On Fri, 26 Apr 2024, Jonathan Wakely wrote: > > > How are you testing on FreeBSD? > > > > When I build GCC trunk on FreeBSD 14.0 and try to run the libstdc++ > > testsuite it fails due to lots of these errors: > > > > Excess errors: > > /usr/local/bin/ld: /tmp//ccev946q.o: relocation R_X86_64_32 against > > symbol `_ZTIN10__cxxabiv115__forced_unwindE@@CXXABI_1.3.2' can not be > > used when making a PDE object; recompile with -fPIE > > /usr/local/bin/ld: failed to set dynamic section sizes: bad value Hi Gerald and Jonathan! I normally test every weekly GCC snapshots through the FreeBSD ports framework on Cirrus, so that all my tests are publicly accessible: http://cirrus-ci.com/github/lsalvadore/freebsd-ports/lang/gcc11-devel http://cirrus-ci.com/github/lsalvadore/freebsd-ports/lang/gcc12-devel http://cirrus-ci.com/github/lsalvadore/freebsd-ports/lang/gcc13-devel http://cirrus-ci.com/github/lsalvadore/freebsd-ports/lang/gcc14-devel And of course the cirrus configuration is public as well: https://github.com/lsalvadore/freebsd-ports/blob/lang/gcc11-devel/.cirrus.yml By the way, is it possible for FreeBSD to enter GCC's CI pipeline? Where would one start? I am willing to help of course. Cheers, Lorenzo Salvadore
Re: GCC testing on FreeBSD
On Sun, 28 Apr 2024, 11:24 Gerald Pfeifer, wrote:
> Hi Jonathan,
>
> On Fri, 26 Apr 2024, Jonathan Wakely wrote:
> > How are you testing on FreeBSD?
> >
> > When I build GCC trunk on FreeBSD 14.0 and try to run the libstdc++
> > testsuite it fails due to lots of these errors:
> >
> > Excess errors:
> > /usr/local/bin/ld: /tmp//ccev946q.o: relocation R_X86_64_32 against
> > symbol `_ZTIN10__cxxabiv115__forced_unwindE@@CXXABI_1.3.2' can not be
> > used when making a PDE object; recompile with -fPIE
> > /usr/local/bin/ld: failed to set dynamic section sizes: bad value
>
> my first reaction was to recommend using binutils instead of /usr/bin/ld
> which is LLD 16.0.6 or similar (since a while ago FreeBSD switched to that
> toolchain as part of the base system).
>
> My nightly tester has been using GNU ld since
>
> # 2012-03-11 Configure using --with-as=$localbase/bin/as and
> # --with-ld=$localbase/bin/ld on *.freebsd.org.
>
> in the script invoked by cron, even before FreeBSD made that switch.
>
> Seeing /usr/local/bin/ld in the error message it appears you are doing
> that already, though?
>
Yes, I'm using GNU ld.
On IRC Andreas suggested that also using GNU objdump would solve my issue,
but I haven't tried yet.
> > Which suggests that -fPIE is missing from the default test flags.
> >
> > Have you seen this? Do you do something locally to work around it?
>
> All I have in terms of adjustments to the FreeBSD systems I build on via
> that script are the following
>
> CONFIGUREFLAGS="--with-gmp=$LOCALBASE --with-as=$LOCALBASE/bin/as
> $CONFIGUREFLAGS"
> LD_LIBRARY_PATH=$LD_LIBRARY_PATH:$LOCALBASE/lib
> PATH=$LOCALBASE/bin:$PATH
>
> where LOCALBASE looks like it would be /usr/local in your case. Nothing
> explicit around PIE there.
>
> Looking at the logs of a serialized build I triggered manually where I
> remove the --with-ld configure option, I see
>
> checking linker PIE support with copy reloc... no
> checking for -fno-PIE option... yes
> checking for -no-pie option... yes
>
> and then build invocations like
>
> c++ -std=c++11 -fno-PIE -c ...
>
> during all-stage1-target-libgcc which ultimately fails with - mystery?! -
>
> ld: error: unable to find library -lc
> collect2: error: ld returned 1 exit status
> gmake[3]: *** [Makefile:1005: libgcc_s.so] Error 1
>
> (Disclaimer: all my tests on FreeBSD 13.2, not FreeBSD 14 as in your
> case.)
>
>
> Looking at the lang/gcc* ports that I maintained for two decades until
> Lorenzo (copied now) kindly took them over two years ago I see the
> following change among others for newer versions:
>
> % git show b6a5871a0cf40
> commit b6a5871a0cf40dfc194217704e2dc03e2e91fb62
> Author: Lorenzo Salvadore
> Date: Fri Feb 3 20:12:49 2023 +0100
>
> lang/gcc10: Mark PIE_UNSAFE
>
> Building the port with WITH_PIE fails if the BOOTSTRAP option is
> enabled. Mark PIE_UNSAFE when this option is enabled until a better
> solution is found.
>
> PR: 268901
>
> where https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=268901 is the PR
> referenced.
>
> PIE_UNSAFE=yes in a port's Makefile (the equivalent to a spec file in RPM
> land) is used as follows in ports/Mk/Features/pie.mk:
>
> . if !defined(PIE_UNSAFE)
> PIE_CFLAGS?=-fPIE -fPIC
> CFLAGS+=${PIE_CFLAGS}
> CXXFLAGS+= ${PIE_CFLAGS}
> LDFLAGS+= -pie
> STATIC_PIE_ARGS+= -static-pie
> . endif
>
>
> You are not using the Ports Infrastructure, I believe, so the above does
> not directly apply, may provide some additional background, though?
>
>
> Hope this helps - and please chime in Lorenzo and Andreas!
>
> Gerald
>
Re: GCC testing on FreeBSD
Hi Lorenzo, > On Sunday, April 28th, 2024 at 12:24, Gerald Pfeifer > wrote: > >> On Fri, 26 Apr 2024, Jonathan Wakely wrote: >> >> > How are you testing on FreeBSD? >> > >> > When I build GCC trunk on FreeBSD 14.0 and try to run the libstdc++ >> > testsuite it fails due to lots of these errors: >> > >> > Excess errors: >> > /usr/local/bin/ld: /tmp//ccev946q.o: relocation R_X86_64_32 against >> > symbol `_ZTIN10__cxxabiv115__forced_unwindE@@CXXABI_1.3.2' can not be >> > used when making a PDE object; recompile with -fPIE >> > /usr/local/bin/ld: failed to set dynamic section sizes: bad value > > Hi Gerald and Jonathan! > > I normally test every weekly GCC snapshots through the FreeBSD ports > framework on Cirrus, so that all my tests are publicly accessible: > http://cirrus-ci.com/github/lsalvadore/freebsd-ports/lang/gcc11-devel > http://cirrus-ci.com/github/lsalvadore/freebsd-ports/lang/gcc12-devel > http://cirrus-ci.com/github/lsalvadore/freebsd-ports/lang/gcc13-devel > http://cirrus-ci.com/github/lsalvadore/freebsd-ports/lang/gcc14-devel > > And of course the cirrus configuration is public as well: > https://github.com/lsalvadore/freebsd-ports/blob/lang/gcc11-devel/.cirrus.yml this isn't particularly helpful if you just try to build upstream GCC for comparision with your own targets or to verify a patch of yours. Having to go hunting for configs like this if you're not a regular FreeBSD user is a no-no IMO. GCC trunk should either build out of the box or the quirks be documented in install.texi. Otherwise, non-FreeBSD developers will get frustrated and give up on the target, to the detriment both of their patches and the platform. Unfortunately, it's pretty common that targets keep necessary patches in some ports collection of their own (usually a different one per target) and neglect to submit them upstream. Rainer -- - Rainer Orth, Center for Biotechnology, Bielefeld University
Re: [PATCH 5/4] libbacktrace: improve getting debug information for loaded dlls
On Thu, Apr 25, 2024 at 1:15 PM Björn Schäpers wrote:
>
> > Attached is the combined version of the two patches, only implementing the
> > variant with the tlhelp32 API.
> >
> > Tested on x86 and x86_64 windows.
> >
> > Kind regards,
> > Björn.
>
> A friendly ping.
Thanks. Committed as follows.
Which of your other patches are still relevant? Thanks.
Ian
942a9cf2a958113d2ab46f5b015c36e569abedcf
diff --git a/libbacktrace/configure.ac b/libbacktrace/configure.ac
index 3e0075a2b79..59e9c415db8 100644
--- a/libbacktrace/configure.ac
+++ b/libbacktrace/configure.ac
@@ -380,6 +380,10 @@ if test "$have_loadquery" = "yes"; then
fi
AC_CHECK_HEADERS(windows.h)
+AC_CHECK_HEADERS(tlhelp32.h, [], [],
+[#ifdef HAVE_WINDOWS_H
+# include
+#endif])
# Check for the fcntl function.
if test -n "${with_target_subdir}"; then
diff --git a/libbacktrace/pecoff.c b/libbacktrace/pecoff.c
index 9e437d810c7..4f267841178 100644
--- a/libbacktrace/pecoff.c
+++ b/libbacktrace/pecoff.c
@@ -49,6 +49,18 @@ POSSIBILITY OF SUCH DAMAGE. */
#endif
#include
+
+#ifdef HAVE_TLHELP32_H
+#include
+
+#ifdef UNICODE
+/* If UNICODE is defined, all the symbols are replaced by a macro to use the
+ wide variant. But we need the ansi variant, so undef the macros. */
+#undef MODULEENTRY32
+#undef Module32First
+#undef Module32Next
+#endif
+#endif
#endif
/* Coff file header. */
@@ -592,7 +604,8 @@ coff_syminfo (struct backtrace_state *state, uintptr_t addr,
static int
coff_add (struct backtrace_state *state, int descriptor,
backtrace_error_callback error_callback, void *data,
- fileline *fileline_fn, int *found_sym, int *found_dwarf)
+ fileline *fileline_fn, int *found_sym, int *found_dwarf,
+ uintptr_t module_handle ATTRIBUTE_UNUSED)
{
struct backtrace_view fhdr_view;
off_t fhdr_off;
@@ -870,12 +883,7 @@ coff_add (struct backtrace_state *state, int descriptor,
}
#ifdef HAVE_WINDOWS_H
- {
-uintptr_t module_handle;
-
-module_handle = (uintptr_t) GetModuleHandle (NULL);
-base_address = module_handle - image_base;
- }
+ base_address = module_handle - image_base;
#endif
if (!backtrace_dwarf_add (state, base_address, &dwarf_sections,
@@ -917,12 +925,61 @@ backtrace_initialize (struct backtrace_state *state,
int found_sym;
int found_dwarf;
fileline coff_fileline_fn;
+ uintptr_t module_handle = 0;
+#ifdef HAVE_TLHELP32_H
+ fileline module_fileline_fn;
+ int module_found_sym;
+ HANDLE snapshot;
+#endif
+
+#ifdef HAVE_WINDOWS_H
+ module_handle = (uintptr_t) GetModuleHandle (NULL);
+#endif
ret = coff_add (state, descriptor, error_callback, data,
- &coff_fileline_fn, &found_sym, &found_dwarf);
+ &coff_fileline_fn, &found_sym, &found_dwarf, module_handle);
if (!ret)
return 0;
+#ifdef HAVE_TLHELP32_H
+ do
+{
+ snapshot = CreateToolhelp32Snapshot (TH32CS_SNAPMODULE, 0);
+}
+ while (snapshot == INVALID_HANDLE_VALUE
+&& GetLastError () == ERROR_BAD_LENGTH);
+
+ if (snapshot != INVALID_HANDLE_VALUE)
+{
+ MODULEENTRY32 entry;
+ BOOL ok;
+ entry.dwSize = sizeof (MODULEENTRY32);
+
+ for (ok = Module32First (snapshot, &entry); ok; ok = Module32Next
(snapshot, &entry))
+ {
+ if (strcmp (filename, entry.szExePath) == 0)
+ continue;
+
+ module_handle = (uintptr_t) entry.hModule;
+ if (module_handle == 0)
+ continue;
+
+ descriptor = backtrace_open (entry.szExePath, error_callback, data,
+ NULL);
+ if (descriptor < 0)
+ continue;
+
+ coff_add (state, descriptor, error_callback, data,
+ &module_fileline_fn, &module_found_sym, &found_dwarf,
+ module_handle);
+ if (module_found_sym)
+ found_sym = 1;
+ }
+
+ CloseHandle (snapshot);
+}
+#endif
+
if (!state->threaded)
{
if (found_sym)
Re: GCC testing on FreeBSD
On Sunday, April 28th, 2024 at 19:19, Rainer Orth wrote: > > > Hi Lorenzo, > > > On Sunday, April 28th, 2024 at 12:24, Gerald Pfeifer ger...@pfeifer.com > > wrote: > > > > > On Fri, 26 Apr 2024, Jonathan Wakely wrote: > > > > > > > How are you testing on FreeBSD? > > > > > > > > When I build GCC trunk on FreeBSD 14.0 and try to run the libstdc++ > > > > testsuite it fails due to lots of these errors: > > > > > > > > Excess errors: > > > > /usr/local/bin/ld: /tmp//ccev946q.o: relocation R_X86_64_32 against > > > > symbol `_ZTIN10__cxxabiv115__forced_unwindE@@CXXABI_1.3.2' can not be > > > > used when making a PDE object; recompile with -fPIE > > > > /usr/local/bin/ld: failed to set dynamic section sizes: bad value > > > > Hi Gerald and Jonathan! > > > > I normally test every weekly GCC snapshots through the FreeBSD ports > > framework on Cirrus, so that all my tests are publicly accessible: > > http://cirrus-ci.com/github/lsalvadore/freebsd-ports/lang/gcc11-devel > > http://cirrus-ci.com/github/lsalvadore/freebsd-ports/lang/gcc12-devel > > http://cirrus-ci.com/github/lsalvadore/freebsd-ports/lang/gcc13-devel > > http://cirrus-ci.com/github/lsalvadore/freebsd-ports/lang/gcc14-devel > > > > And of course the cirrus configuration is public as well: > > https://github.com/lsalvadore/freebsd-ports/blob/lang/gcc11-devel/.cirrus.yml > > > this isn't particularly helpful if you just try to build upstream GCC > for comparision with your own targets or to verify a patch of yours. > Having to go hunting for configs like this if you're not a regular > FreeBSD user is a no-no IMO. GCC trunk should either build out of the > box or the quirks be documented in install.texi. Otherwise, non-FreeBSD > developers will get frustrated and give up on the target, to the > detriment both of their patches and the platform. > > Unfortunately, it's pretty common that targets keep necessary patches in > some ports collection of their own (usually a different one per target) > and neglect to submit them upstream. I totally agree with you: upstreaming patches is important! It is not only for the upstream project itself, but for the target as well: having patches sitting in a ports collection also requires more maintainance, they require to be kept up to date with the upstream progresses. Unfortunately, it is not always possible to upstream a patch. Sometimes, patches are too specific to a target to be suitable for upstream. For example, smaller projects might be interested in supporting only very popular platforms and would not accept (or could not accept) the complexity of supporting a less known target. Hopefully this is not the case for GCC. Other times, developers do try to upstream a patch, but fail. This happened to myself in GCC too, when I was unable to get any attention to a patch I submitted, and could not do any better than keep the patch into the FreeBSD ports collection: https://gcc.gnu.org/pipermail/jit/2023q3/001642.html https://gcc.gnu.org/bugzilla/show_bug.cgi?id=101491#c5 If you are able to help with this upstreaming, I would appreciate it a lot. Thanks. Cheers, Lorenzo Salvadore
gcc-15-20240428 is now available
Snapshot gcc-15-20240428 is now available on https://gcc.gnu.org/pub/gcc/snapshots/15-20240428/ and on various mirrors, see https://gcc.gnu.org/mirrors.html for details. This snapshot has been generated from the GCC 15 git branch with the following options: git://gcc.gnu.org/git/gcc.git branch master revision d71308d5a681de00ea291136c162e5b46c7c You'll find: gcc-15-20240428.tar.xz Complete GCC SHA256=e31cdd424a22d132bbe66bb1a2d061033e5e5d5b62d82a1cd3e0711665b87f26 SHA1=24e6766fb090d98710ec8e16a3af4dbc141453a7 Diffs from 15- are available in the diffs/ subdirectory. When a particular snapshot is ready for public consumption the LATEST-15 link is updated and a message is sent to the gcc list. Please do not use a snapshot before it has been announced that way.
