https://gcc.gnu.org/bugzilla/show_bug.cgi?id=104832
Bug ID: 104832
Summary: gccgo / libgo Reproducibility Problem
Product: gcc
Version: unknown
Status: UNCONFIRMED
Severity: normal
Priority: P3
Component: go
Assignee: ian at airs dot com
Reporter: toolybird at tuta dot io
CC: cmang at google dot com
Target Milestone: ---
TL;DR -> Different binaries are produced depending on whether gccgo is already
installed on the host or not.
Background: I'm looking into broad issues of reproducibility in the context of
bootstrapping whole toolchains.
I first noticed this problem with gcc-11.2.0 but it's still present on GCC
trunk.
To reproduce, you will need to have the same version of gccgo installed on your
host in /usr.
For example, with current GCC trunk:
$ cat gcc/BASE-VER
$ 12.0.1
Step 1:
bootstrap GCC with `--enable-languages=c,c++,go --prefix=/usr
--libexecdir=/usr/lib' then install into a temp DESTDIR.
Step 2:
Uninstall gccgo from the host, or you can fudge it with:
$ sudo mv /usr/lib/go /usr/lib/go.XX
Step 3:
bootstrap GCC again with `--enable-languages=c,c++,go --prefix=/usr
--libexecdir=/usr/lib' then install into a 2nd temp DESTDIR.
Step 4: diff / compare the binaries in the DESTDIRs and you will see
differences.
Delving deeper and diffing the gcc-build dirs, there are a number of object
files affected. A list sorted with `ls -altr' is at the end of this report.
Here is a demonstration with "libgo/fmt.o" (although I suspect the real problem
is further up the chain):
$ cd gcc-build/x86_64-pc-linux-gnu/libgo
$ /build/gcc/src/gcc-build/./gcc/gccgo -B/build/gcc/src/gcc-build/./gcc/
-minline-all-stringops -O2 -g -I . -c -fgo-pkgpath=fmt
../../../gcc/libgo/go/fmt/doc.go ../../../gcc/libgo/go/fmt/errors.go
../../../gcc/libgo/go/fmt/format.go ../../../gcc/libgo/go/fmt/print.go
../../../gcc/libgo/go/fmt/scan.go -o fmt-test1.o
$ sudo mv /usr/lib/go /usr/lib/go.XX
$ /build/gcc/src/gcc-build/./gcc/gccgo -B/build/gcc/src/gcc-build/./gcc/
-minline-all-stringops -O2 -g -I . -c -fgo-pkgpath=fmt
../../../gcc/libgo/go/fmt/doc.go ../../../gcc/libgo/go/fmt/errors.go
../../../gcc/libgo/go/fmt/format.go ../../../gcc/libgo/go/fmt/print.go
../../../gcc/libgo/go/fmt/scan.go -o fmt-test2.o
$ diff fmt-test1.o fmt-test2.o
Binary files fmt-test1.o and fmt-test2.o differ
The differences are in the ".go_export" section of the ELF file.
Using "strings" helps to visualize the differences:
$ strings fmt-test1.o > 1.txt
$ strings fmt-test2.o > 2.txt
$ diff -u 1.txt 2.txt
--- 1.txt 2022-03-08 11:07:01.577545105 +1100
+++ 2.txt 2022-03-08 11:07:06.364263769 +1100
@@ -860,7 +860,7 @@
import utf8 unicode/utf8 "unicode/utf8"
init fmt fmt..import errors errors..import cpu internal_1cpu..import oserror
internal_1oserror..import poll internal_1poll..import reflectlite
internal_1reflectlite..import testlog internal_1testlog..import io io..import
fs io_1fs..import math math..import os os..import path path..import reflect
reflect..import runtime runtime..import sort sort..import strconv
strconv..import sync sync..import syscall syscall..import time time..import
unicode unicode..import abi ~internal_1abi bytealg ~internal_1bytealg fmtsort
~internal_1fmtsort goarch ~internal_1goarch goexperiment
~internal_1goexperiment goos ~internal_1goos itoa ~internal_1itoa race
~internal_1race execenv ~internal_1syscall_1execenv unix
~internal_1syscall_1unix unsafeheader ~internal_1unsafeheader bits ~math_1bits
atomic ~runtime_1internal_1atomic math ~runtime_1internal_1math sys
~runtime_1internal_1sys atomic ~sync_1atomic utf8 ~unicode_1utf8
init_graph 0 1 0 2 0 3 0 4 0 5 0 6 0 7 0 8 0 9 0 10 0 11 0 12 0 13 0 14 0 15 0
16 0 17 0 18 0 19 1 2 1 5 1 13 3 1 3 2 3 5 3 13 4 1 4 2 4 3 4 5 4 7 4 13 4 16 4
17 4 18 5 2 5 13 6 2 6 13 6 16 7 1 7 2 7 5 7 13 7 16 8 1 8 2 8 3 8 5 8 7 8 11 8
13 8 14 8 16 8 17 8 18 9 2 10 1 10 2 10 3 10 4 10 5 10 6 10 7 10 8 10 11 10 13
10 14 10 16 10 17 10 18 11 1 11 2 11 5 11 13 12 1 12 2 12 5 12 9 12 13 12 15 12
16 12 19 13 2 14 2 14 5 14 13 15 1 15 2 15 5 15 9 15 13 16 2 16 13 17 1 17 2 17
3 17 5 17 13 17 16 18 1 18 2 18 3 18 5 18 13 18 16 18 17
-types 41 7 29 30 29 27 25 28 1067 3172 30 335 26 30 34 35 30 88 92 47 88 47 34
34 18 18 19 115 171 21 22 195 60 46 83 61 73 295 73 44 74 22
+types 41 7 29 30 29 27 25 28 1067 3172 30 335 26 30 34 35 30 88 92 47 88 47 38
34 18 18 19 115 171 21 22 195 60 46 83 61 73 295 73 44 74 22
type 1 "Formatter" <type 31>
type 2 "GoStringer" <type 32>
type 3 "ScanState" <type 36>
@@ -972,7 +972,7 @@
type 18 (? <type 29>, ? <type -21>) <type -11>
type 19 (? <type 29>, ? <type -16>, ? <type -20>, ? <type -15>, ? <type -15>)
<type 29>
type 20 (? <type -10>, ? <type -3>) <type -10>
-type 21 (? <type -21>) <type -15>
+type 21 (? <type -21>) (? <type -15>)
type 22 (? <type -16>) <type -19>
type 23 *<type 7>
type 24 *<type 8>
@@ -1067,7 +1067,7 @@
$ret10 = $convert(<type -15>, $false) //577
return //577
} //0
-checksum 22D2C8F1A933DC5F8B8E981917B8ADA04DEB2FC1
+checksum A916C151E089AA7EC237CBEBBE2B293A9595478C
ScanState's Read should not be called. Use ReadRune
fmt: scanning called UnreadRune with no rune available
0123456789ABCDEFX
strace shows the host dirs being accessed:
First case:
newfstatat(AT_FDCWD, "/build/gcc/src/gcc-build/./gcc/go/12.0.1",
0x7ffc48250da0, 0) = -1 ENOENT (No such file or directory)
newfstatat(AT_FDCWD, "/lib/../lib/go/12.0.1", {st_mode=S_IFDIR|0755,
st_size=4096, ...}, 0) = 0
newfstatat(AT_FDCWD, "/lib/../lib/go/12.0.1/x86_64-pc-linux-gnu",
{st_mode=S_IFDIR|0755, st_size=4096, ...}, 0) = 0
newfstatat(AT_FDCWD, "/usr/lib/../lib/go/12.0.1", {st_mode=S_IFDIR|0755,
st_size=4096, ...}, 0) = 0
newfstatat(AT_FDCWD, "/usr/lib/../lib/go/12.0.1/x86_64-pc-linux-gnu",
{st_mode=S_IFDIR|0755, st_size=4096, ...}, 0) = 0
Second case:
newfstatat(AT_FDCWD, "/build/gcc/src/gcc-build/./gcc/go/12.0.1",
0x7ffee9c2a960, 0) = -1 ENOENT (No such file or directory)
newfstatat(AT_FDCWD, "/lib/../lib/go/12.0.1", 0x7ffee9c2a960, 0) = -1 ENOENT
(No such file or directory)
newfstatat(AT_FDCWD, "/usr/lib/../lib/go/12.0.1", 0x7ffee9c2a960, 0) = -1
ENOENT (No such file or directory)
Any thoughts?
Thanks
gcc-build/x86_64-pc-linux-gnu/libgo/.libs/runtime.o
gcc-build/x86_64-pc-linux-gnu/libgo/.libs/syscall.o
gcc-build/x86_64-pc-linux-gnu/libgo/syscall.o
gcc-build/x86_64-pc-linux-gnu/libgo/index/suffixarray.o
gcc-build/x86_64-pc-linux-gnu/libgo/.libs/os.o
gcc-build/x86_64-pc-linux-gnu/libgo/os.o
gcc-build/x86_64-pc-linux-gnu/libgo/.libs/fmt.o
gcc-build/x86_64-pc-linux-gnu/libgo/fmt.o
gcc-build/x86_64-pc-linux-gnu/libgo/image/.libs/gif.o
gcc-build/x86_64-pc-linux-gnu/libgo/image/gif.o
gcc-build/x86_64-pc-linux-gnu/libgo/archive/.libs/zip.o
gcc-build/x86_64-pc-linux-gnu/libgo/archive/zip.o
gcc-build/x86_64-pc-linux-gnu/libgo/debug/.libs/macho.o
gcc-build/x86_64-pc-linux-gnu/libgo/text/.libs/template.o
gcc-build/x86_64-pc-linux-gnu/libgo/testing.o
gcc-build/x86_64-pc-linux-gnu/libgo/text/template.o
gcc-build/x86_64-pc-linux-gnu/libgo/go/ast.o
gcc-build/x86_64-pc-linux-gnu/libgo/net/.libs/mail.o
gcc-build/x86_64-pc-linux-gnu/libgo/net/mail.o
gcc-build/x86_64-pc-linux-gnu/libgo/html/.libs/template.o
gcc-build/x86_64-pc-linux-gnu/libgo/html/template.o
gcc-build/x86_64-pc-linux-gnu/libgo/crypto/.libs/x509.o
gcc-build/x86_64-pc-linux-gnu/libgo/crypto/tls.o
gcc-build/x86_64-pc-linux-gnu/libgo/net/.libs/http.o
gcc-build/x86_64-pc-linux-gnu/libgo/net/http.o
gcc-build/x86_64-pc-linux-gnu/libgo/net/.libs/rpc.o
gcc-build/x86_64-pc-linux-gnu/libgo/net/rpc.o
gcc-build/x86_64-pc-linux-gnu/libgo/cmd/go/internal/modfetch.o
gcc-build/x86_64-pc-linux-gnu/libgo/cmd/go/internal/help.o
gcc-build/x86_64-pc-linux-gnu/libgo/cmd/go/internal/workcmd.o
gcc-build/x86_64-pc-linux-gnu/libgo/cmd/go/internal/work.o
gcc-build/x86_64-pc-linux-gnu/libgo/cmd/go/internal/clean.o
gcc-build/x86_64-pc-linux-gnu/libgo/cmd/go/internal/get.o
gcc-build/x86_64-pc-linux-gnu/libgo/cmd/go/internal/envcmd.o
gcc-build/x86_64-pc-linux-gnu/libgo/cmd/go/internal/bug.o
gcc-build/x86_64-pc-linux-gnu/libgo/cmd/go/internal/modget.o
gcc-build/x86_64-pc-linux-gnu/libgo/cmd/go/internal/test.o
