Nuno Teixeira <eduardo_at_freebsd.org> wrote on
Date: Mon, 05 May 2025 20:37:09 UTC :
> (...)
>
> Don't forget to `env NO_PKG_UPGRADE=yes beinstall.sh` to not mess with
> ports and stuff.
>
> Nuno Teixeira <edua...@freebsd.org> escreveu (segunda, 5/05/2025 à(s)
> 21:34):
>
> > Hello,
> >
> > Using incremental WITH_META_MODE builds, after installation with
> > beinstall.sh, building same src, a complete compilation happens.
> >
> > If someone uses this script, could you please do the following test:
> >
> > WITH_META_MODE=yes (/etc/src-env.conf)
> > filemon module loaded
> >
> > # cd /usr/src
> > # make buildworld-jobs buildkernel-jobs
The above used older commands and files from before
the following install. META_MODE recorded the use of
those commands.
Example .meta mode file content:
# Meta data file
/usr/obj/amd64_clang/amd64.amd64/usr/fbsd/mm-src/amd64.amd64/lib/libc++/libc++.a.meta
CMD @echo building static c++ library
CMD @rm -f libc++.a
CMD ar -crsD libc++.a algorithm.o any.o atomic.o barrier.o bind.o charconv.o
chrono.o condition_variable.o condition_variable_destructor.o debug.o
exception.o filesystem/directory_iterator.o filesyste
m/int128_builtins.o filesystem/operations.o functional.o future.o hash.o ios.o
iostream.o locale.o memory.o mutex.o mutex_destructor.o new.o optional.o
random.o random_shuffle.o regex.o shared_mutex.o
stdexcept.o string.o strstream.o system_error.o thread.o typeinfo.o utility.o
valarray.o variant.o vector.o cxxrt_auxhelper.o cxxrt_dynamic_cast.o
cxxrt_exception.o cxxrt_guard.o cxxrt_libelftc_dem_g
nu3.o cxxrt_memory.o cxxrt_stdexcept.o cxxrt_terminate.o cxxrt_typeinfo.o
CWD /usr/obj/amd64_clang/amd64.amd64/usr/fbsd/mm-src/amd64.amd64/lib/libc++
TARGET libc++.a
-- command output --
building static c++ library
-- filemon acquired metadata --
# filemon version 5
# Target pid 22471
# Start 1611359217.214996
V 5
E 22961 /bin/sh
R 22961 /etc/libmap.conf
R 22961 /var/run/ld-elf.so.hints
R 22961 /lib/libedit.so.7
R 22961 /lib/libc.so.7
R 22961 /lib/libncursesw.so.9
R 22961 /usr/share/locale/C.UTF-8/LC_CTYPE
F 22961 22962
E 22962
/usr/obj/amd64_clang/amd64.amd64/usr/fbsd/mm-src/amd64.amd64/tmp/legacy/usr/sbin/rm
R 22962 /etc/libmap.conf
R 22962 /var/run/ld-elf.so.hints
R 22962 /lib/libc.so.7
R 22962 /usr/share/locale/C.UTF-8/LC_CTYPE
D 22962 libc++.a
X 22962 0 0
. . .
So META_MODE has lots of files that were used
that it can later detect being newer than the
prior build results, leading to rebuilds based
on those newer files.
> > # ./tools/build/beinstall.sh
The new be will have various updated files
that could be different by content and, for
commands, could behave differently than those
used to do the prior build. Detecting newer
time stamps on such used files leads to
rebuild activity.
More than /usr/src/ and /usr/obj/ content
are involved as well.
Note that the new be is based on somewhat
different files than the original
buildworld-jobs buildkernel-jobs was based
on.
> > # reboot
> >
(I presume booting into the new be here.)
> > # cd /usr/src
> > # make buildworld-jobs buildkernel-jobs
META_MODE will notice when commands are used
that are newer than when the prior build was
done. Similarly for other files that may be
read. It will make sure that the newer commands
and files are allowed to produce new results
that potentially could be distinct in content
from what the old context produced for results.
> >
> > Since src and obj are the same from one BE to newer BE, minimal
> > compilation should happen, not a full one.
META_MODE is more careful than that.
Note: I'm not claiming that new behavior that is
needed is likely for lots of the files with new
dates. But META_MODE is biased to avoiding leaving
in place something that should have been updated.
> >
> > Am I missing something here?
> >
Note that make has a -dM option:
M Print debugging information about “meta” mode decisions
about targets.
So, for example,
file
'/usr/obj/amd64_clang/amd64.amd64/usr/fbsd/mm-src/amd64.amd64/tmp/legacy/usr/sbin/awk'
is newer than the target...
file
'/usr/obj/amd64_clang/amd64.amd64/usr/fbsd/mm-src/amd64.amd64/tmp/legacy/usr/sbin/cap_mkdb'
is newer than the target...
file
'/usr/obj/amd64_clang/amd64.amd64/usr/fbsd/mm-src/amd64.amd64/tmp/legacy/usr/sbin/cat'
is newer than the target...
file
'/usr/obj/amd64_clang/amd64.amd64/usr/fbsd/mm-src/amd64.amd64/tmp/legacy/usr/sbin/cp'
is newer than the target...
file
'/usr/obj/amd64_clang/amd64.amd64/usr/fbsd/mm-src/amd64.amd64/tmp/legacy/usr/sbin/crunchgen'
is newer than the target...
file
'/usr/obj/amd64_clang/amd64.amd64/usr/fbsd/mm-src/amd64.amd64/tmp/legacy/usr/sbin/crunchide'
is newer than the target...
file
'/usr/obj/amd64_clang/amd64.amd64/usr/fbsd/mm-src/amd64.amd64/tmp/legacy/usr/sbin/dd'
is newer than the target...
. . .
Various . . ./tmp/legacy/. . ./*bin/ actually were
links to files.
Ultimately buildworld then installworld lead to new
dates for a bunch of files used. Later use of
META_MODE notices such and rebuilds based on the
newer files. (It is a lot of detail to go through
it all.)
Back in 2021 and 2023 I got help with exploring
avoiding lots of these. But, in the end, it
involved use of experimental code in
share/mk/src.sys.obj.mk to provide a new
definition to use to build some paths with.
The experiments were an unsupported activity that
produced an unsupported change to allow
configurable enabling of taking risks with not
updating files that possibly should be updated.
===
Mark Millard
marklmi at yahoo.com
