On Jun 16, 2025, at 04:50, Ronald Klop <[email protected]> wrote: > Van: Mark Millard <[email protected]> > Datum:vrijdag, 30 mei 2025 20:58 > Aan:"Dag-Erling Smørgrav" <[email protected]> > CC:[email protected], FreeBSD Mailing List <[email protected]> > Onderwerp:Re: error -9 on build cluster? > On May 30, 2025, at 11:29, Mark Millard <[email protected]> wrote: > > > On May 30, 2025, at 10:43, Dag-Erling Smørgrav <[email protected]> wrote: > > > >> Mark Millard <[email protected]> writes: > >>> Dag-Erling Smørgrav <[email protected]> writes: > >>>> This is preceded by an 800-line list of warnings from a single source > >>>> file (multi_index_block.cpp). This list is clearly truncated as it's > >>>> missing the final “N warnings generated” line, so my guess is there is > >>>> at least one error there as well which just doesn't get displayed. > >>> My guess: the failure occurred while the warnings were otherwise > >>> still being generated and that is what stopped the warnings. > >> > >> That's possible. If we assume that the error code returned by scons is > >> an errno, it corresponds to EBADF. Perhaps a bug in scons causes it to > >> lose track of the pipe it uses to read the compiler's stderr. > > > > devel/scans has not been updated after 2024-Jun-06: > > > > QUOTE > > author Po-Chuan Hsieh <[email protected]> 2024-06-06 22:03:56 +0000 > > committer Po-Chuan Hsieh <[email protected]> 2024-06-06 22:03:56 +0000 > > . . . > > devel/scons: Update to 4.7.0 > > Changes: https://github.com/SCons/scons/releases > > https://scons.org/tag/releases.html PR: 279488 Exp-run by: antoine > > END QUOTE > > > > That would mean that quarterly got the update in 2024 as well. > > > > https://pkg-status.freebsd.org/ampere1/data/141arm64-quarterly/2e2d801cf977/logs/mongodb60-6.0.18.log > > > > did not fail but shows the warnings. > > > > devel/scons does not seem to be the source of the change in beavior. > > > Worked: > > https://pkg-status.freebsd.org/ampere3/data/134arm64-default/1afdc808e67b/logs/mongodb60-6.0.20_1.log > build started at Sun Jan 5 17:35:49 UTC 2025 > > The next 134arm64-default build Failed: > > https://pkg-status.freebsd.org/ampere3/data/134arm64-default/86e28c84a7e0/logs/errors/mongodb60-6.0.20_1.log > build started at Sun Jan 19 11:21:16 UTC 2025 > > That at least gives a time frame to investigate. > > More details: > > =>> Building databases/mongodb60 > build started at Sun Jan 5 17:35:49 UTC 2025 > port directory: /usr/ports/databases/mongodb60 > package name: mongodb60-6.0.20_1 > building for: FreeBSD 134arm64-default-job-06 13.4-RELEASE-p2 FreeBSD > 13.4-RELEASE-p2 arm64 > maintained by: [email protected] > Makefile datestamp: -rw-r--r-- 1 root wheel 5845 Jan 4 01:02 > /usr/ports/databases/mongodb60/Makefile > Ports top last git commit: 1afdc808e67 > Ports top unclean checkout: no > Port dir last git commit: 3ab18c284b0 > Port dir unclean checkout: no > Poudriere version: poudriere-git-3.4.2 > Host OSVERSION: 1500028 > Jail OSVERSION: 1304000 > > vs. > > =>> Building databases/mongodb60 > build started at Sun Jan 19 11:21:16 UTC 2025 > port directory: /usr/ports/databases/mongodb60 > package name: mongodb60-6.0.20_1 > building for: FreeBSD 134arm64-default-job-13 13.4-RELEASE-p2 FreeBSD > 13.4-RELEASE-p2 arm64 > maintained by: [email protected] > Makefile datestamp: -rw-r--r-- 1 root wheel 5845 Jan 4 01:02 > /usr/ports/databases/mongodb60/Makefile > Ports top last git commit: 86e28c84a7e > Ports top unclean checkout: no > Port dir last git commit: 3ab18c284b0 > Port dir unclean checkout: no > Poudriere version: poudriere-git-3.4.2 > Host OSVERSION: 1500028 > Jail OSVERSION: 1304000 > > Same Host and Jail OSVERSION's. Same "Port dir last git commit". > > But different "Ports top last git commit". So. some change in > another package that is involved? > > === > Mark Millard > marklmi at yahoo.com > > > > Hi, > > The builder got the same error on mongodb70.
Well recent 60 vs. the 70: scons: *** [build/59f4f0dd/mongo/db/catalog/multi_index_block.o] Error -9 vs.: scons: *** [build/opt/mongo/db/query/canonical_query_encoder.o] Error -9 It is not at an analogous point in the build. Looking at the build times: build time: 00:45:05 vs.: build time: 05:05:52 Looking at the 60 history . . . scons: *** [build/59f4f0dd/mongo/db/catalog/multi_index_block.o] Error -9 scons: building terminated because of errors. build/59f4f0dd/mongo/db/catalog/multi_index_block.o failed: Error -9 scons: *** [build/59f4f0dd/mongo/db/views/view_catalog_helpers.o] Error -9 . . . scons: building terminated because of errors. build/59f4f0dd/mongo/db/views/view_catalog_helpers.o failed: Error -9 (The above 2 are amd64, the below older ones are all arm64:) src/mongo/db/commands/collection_to_capped.cpp . . . *** Signal 9 build/59f4f0dd/mongo/db/commands/feature_compatibility_version_document_gen.cpp . . . *** Signal 9 build/59f4f0dd/mongo/db/storage/storage_parameters_gen.cpp . . . *** Signal 9 (The older ones only have the report available now, not the log:) scons: building terminated because of errors. build/59f4f0dd/mongo/db/s/config/sharding_catalog_manager_zone_operations.o failed: Error -9 scons: building terminated because of errors. build/59f4f0dd/mongo/db/commands/mr_common.o failed: Error -9 scons: building terminated because of errors. build/59f4f0dd/mongo/db/query/find.o failed: Error -9 src/third_party/mozjs/platform/aarch64/freebsd/build/Unified_cpp_js_src16.cpp *** Signal 9 (The above goes back to 2025-02-05 17:50.) The first is 6.0.23_1. The 2nd is 6.0.22_1. The last 2 are 6.0.20_1. The rest are 6.0.20_2. There is no uniformity to where the errors occur. There is also variability for "Signal 9" vs. "Error -9". A guess would be that "Error -9" is another way of reporting "Signal 9". Back in 2022-Jul there was this to avoid "Signal 9" problems: A commit in branch main references this bug: URL: https://cgit.FreeBSD.org/ports/commit/?id=0aabc02be18794a56c436233e59379ebaa86c2e0 commit 0aabc02be18794a56c436233e59379ebaa86c2e0 Author: Daniel Engberb <[email protected]> AuthorDate: 2022-07-14 14:43:02 +0000 Commit: Fernando Apesteguía <[email protected]> CommitDate: 2022-07-16 15:11:58 +0000 databases/mongodb50: Disable LTO by default. Building mongo with LTO enabled takes a ridiculously amount of memory (32GB in amd64). Disable -ggdb so memory requirements don't sky rocket (around 10-12GB) PR: 264340 Reported by: [email protected] Approved by: [email protected] (maintainer) databases/mongodb50/files/patch-SConstruct | 15 +++++++++++---- 1 file changed, 11 insertions(+), 4 deletions(-) There was also a comment reporting console log text like: swap_pager: out of swap space swp_pager_getswapspace(2): failed swap_pager: out of swap space swp_pager_getswapspace(2): failed swp_pager_getswapspace(5): failed swp_pager_getswapspace(1): failed swp_pager_getswapspace(23): failed pid 73953 (ld.lld), jid 0, uid 0, was killed: failed to reclaim memory But, to my knowledge, such information is not publicly available and would be not easily tied to match the timing of mongodb* build failures. So personal/local builds may be needed to figure out if such might be involved. > But this build does not have all the warnings. I still don't see the cause of > Error -9. > > https://pkg-status.freebsd.org/beefy22/data/142amd64-default/02e9dc3b903b/logs/mongodb70-7.0.21_1.log > > https://portsfallout.com/fallout?port=databases%2Fmongodb70%24 > > My next idea is that this occurs because a process gets killed (-9), which > should be visible in the logs of the cluster maintainers. Does this makes > sense? === Mark Millard marklmi at yahoo.com
