Hi,
Do you think this could help? The default JOBS=3 on the package builder cluster.
diff --git a/databases/mongodb70/Makefile b/databases/mongodb70/Makefile
index 0b54ffa2fa64..9843b818c129 100644
--- a/databases/mongodb70/Makefile
+++ b/databases/mongodb70/Makefile
@@ -62,6 +62,10 @@ CONFIGURE_ARGS+= ${MOZJS_ARCH} \
freebsd
CONFIGURE_ENV+= PYTHON3="${PYTHON_CMD}"
+.if defined(PACKAGE_BUILDING)
+MAKE_JOBS_NUMBER= 2
+.endif
+
MAKE_ARGS= --allocator=system \
--cxx-std=20 \
--disable-warnings-as-errors \
Regards,
Ronald.
Van: Mark Millard <[email protected]>
Datum:dinsdag, 17 juni 2025 02:31
Aan:Ronald Klop <[email protected]>
CC:"Dag-Erling Smørgrav" <[email protected]>, FreeBSD Mailing List
<[email protected]>
Onderwerp:Re: error -9 on build cluster?
On Jun 16, 2025, at 08:22, Mark Millard <[email protected]> wrote:
> On Jun 16, 2025, at 04:50, Ronald Klop <[email protected]> wrote:
>
>> Van: Mark Millard <[email protected]>
>> Datum:vrijdag, 30 mei 2025 20:58
>> Aan:"Dag-Erling Smørgrav" <[email protected]>
>> CC:[email protected], FreeBSD Mailing List <[email protected]>
>> Onderwerp:Re: error -9 on build cluster?
>> On May 30, 2025, at 11:29, Mark Millard <[email protected]> wrote:
>>
>>> On May 30, 2025, at 10:43, Dag-Erling Smørgrav <[email protected]> wrote:
>>>
>>>> Mark Millard <[email protected]> writes:
>>>>> Dag-Erling Smørgrav <[email protected]> writes:
>>>>>> This is preceded by an 800-line list of warnings from a single source
>>>>>> file (multi_index_block.cpp). This list is clearly truncated as it's
>>>>>> missing the final “N warnings generated” line, so my guess is there is
>>>>>> at least one error there as well which just doesn't get displayed.
>>>>> My guess: the failure occurred while the warnings were otherwise
>>>>> still being generated and that is what stopped the warnings.
>>>>
>>>> That's possible. If we assume that the error code returned by scons is
>>>> an errno, it corresponds to EBADF. Perhaps a bug in scons causes it to
>>>> lose track of the pipe it uses to read the compiler's stderr.
>>>
>>> devel/scans has not been updated after 2024-Jun-06:
>>>
>>> QUOTE
>>> author Po-Chuan Hsieh <[email protected]> 2024-06-06 22:03:56 +0000
>>> committer Po-Chuan Hsieh <[email protected]> 2024-06-06 22:03:56 +0000
>>> . . .
>>> devel/scons: Update to 4.7.0
>>> Changes: https://github.com/SCons/scons/releases
https://scons.org/tag/releases.html PR: 279488 Exp-run by: antoine
>>> END QUOTE
>>>
>>> That would mean that quarterly got the update in 2024 as well.
>>>
>>>
https://pkg-status.freebsd.org/ampere1/data/141arm64-quarterly/2e2d801cf977/logs/mongodb60-6.0.18.log
>>>
>>> did not fail but shows the warnings.
>>>
>>> devel/scons does not seem to be the source of the change in beavior.
>>
>>
>> Worked:
>>
>>
https://pkg-status.freebsd.org/ampere3/data/134arm64-default/1afdc808e67b/logs/mongodb60-6.0.20_1.log
>> build started at Sun Jan 5 17:35:49 UTC 2025
>>
>> The next 134arm64-default build Failed:
>>
>>
https://pkg-status.freebsd.org/ampere3/data/134arm64-default/86e28c84a7e0/logs/errors/mongodb60-6.0.20_1.log
>> build started at Sun Jan 19 11:21:16 UTC 2025
>>
>> That at least gives a time frame to investigate.
>>
>> More details:
>>
>> =>> Building databases/mongodb60
>> build started at Sun Jan 5 17:35:49 UTC 2025
>> port directory: /usr/ports/databases/mongodb60
>> package name: mongodb60-6.0.20_1
>> building for: FreeBSD 134arm64-default-job-06 13.4-RELEASE-p2 FreeBSD
13.4-RELEASE-p2 arm64
>> maintained by: [email protected]
>> Makefile datestamp: -rw-r--r-- 1 root wheel 5845 Jan 4 01:02
/usr/ports/databases/mongodb60/Makefile
>> Ports top last git commit: 1afdc808e67
>> Ports top unclean checkout: no
>> Port dir last git commit: 3ab18c284b0
>> Port dir unclean checkout: no
>> Poudriere version: poudriere-git-3.4.2
>> Host OSVERSION: 1500028
>> Jail OSVERSION: 1304000
>>
>> vs.
>>
>> =>> Building databases/mongodb60
>> build started at Sun Jan 19 11:21:16 UTC 2025
>> port directory: /usr/ports/databases/mongodb60
>> package name: mongodb60-6.0.20_1
>> building for: FreeBSD 134arm64-default-job-13 13.4-RELEASE-p2 FreeBSD
13.4-RELEASE-p2 arm64
>> maintained by: [email protected]
>> Makefile datestamp: -rw-r--r-- 1 root wheel 5845 Jan 4 01:02
/usr/ports/databases/mongodb60/Makefile
>> Ports top last git commit: 86e28c84a7e
>> Ports top unclean checkout: no
>> Port dir last git commit: 3ab18c284b0
>> Port dir unclean checkout: no
>> Poudriere version: poudriere-git-3.4.2
>> Host OSVERSION: 1500028
>> Jail OSVERSION: 1304000
>>
>> Same Host and Jail OSVERSION's. Same "Port dir last git commit".
>>
>> But different "Ports top last git commit". So. some change in
>> another package that is involved?
>>
>> ===
>> Mark Millard
>> marklmi at yahoo.com
>>
>>
>>
>> Hi,
>>
>> The builder got the same error on mongodb70.
>
> Well recent 60 vs. the 70:
>
> scons: *** [build/59f4f0dd/mongo/db/catalog/multi_index_block.o] Error -9
> vs.:
> scons: *** [build/opt/mongo/db/query/canonical_query_encoder.o] Error -9
>
> It is not at an analogous point in the build. Looking at the build times:
>
> build time: 00:45:05
> vs.:
> build time: 05:05:52
>
>
> Looking at the 60 history . . .
>
> scons: *** [build/59f4f0dd/mongo/db/catalog/multi_index_block.o] Error -9
> scons: building terminated because of errors.
> build/59f4f0dd/mongo/db/catalog/multi_index_block.o failed: Error -9
>
> scons: *** [build/59f4f0dd/mongo/db/views/view_catalog_helpers.o] Error -9
> . . .
> scons: building terminated because of errors.
> build/59f4f0dd/mongo/db/views/view_catalog_helpers.o failed: Error -9
>
> (The above 2 are amd64, the below older ones are all arm64:)
>
> src/mongo/db/commands/collection_to_capped.cpp
> . . .
> *** Signal 9
>
>
build/59f4f0dd/mongo/db/commands/feature_compatibility_version_document_gen.cpp
> . . .
> *** Signal 9
>
> build/59f4f0dd/mongo/db/storage/storage_parameters_gen.cpp
> . . .
> *** Signal 9
>
> (The older ones only have the report available now, not the log:)
>
> scons: building terminated because of errors.
> build/59f4f0dd/mongo/db/s/config/sharding_catalog_manager_zone_operations.o
failed: Error -9
>
> scons: building terminated because of errors.
> build/59f4f0dd/mongo/db/commands/mr_common.o failed: Error -9
>
> scons: building terminated because of errors.
> build/59f4f0dd/mongo/db/query/find.o failed: Error -9
>
> src/third_party/mozjs/platform/aarch64/freebsd/build/Unified_cpp_js_src16.cpp
> *** Signal 9
>
> (The above goes back to 2025-02-05 17:50.)
>
> The first is 6.0.23_1. The 2nd is 6.0.22_1. The last 2 are 6.0.20_1.
> The rest are 6.0.20_2.
>
> There is no uniformity to where the errors occur. There is also
> variability for "Signal 9" vs. "Error -9". A guess would be that
> "Error -9" is another way of reporting "Signal 9".
Speaking of variability:
https://pkg-status.freebsd.org/beefy18/build.html?mastername=main-amd64-default&build=p056670b6dfea_s1749465947a
shows building all the amd64 mongodb[5678]0* just fine.
That build started at: 05 Jun 2025 01:11:32 GMT on
beefy18 .
As does the beefy8 13 May 2025 09:01:49 GMT:
https://pkg-status.freebsd.org/beefy8/build.html?mastername=142amd64-default&build=2025-05-13_09h01m49s
and the beefy16 24 Apr 2025 07:24:46 GMT:
https://pkg-status.freebsd.org/beefy16/build.html?mastername=134amd64-default&build=dc1d367f2961
and the beefy14 12 Apr 2025 01:03:08 GMT:
https://pkg-status.freebsd.org/beefy14/build.html?mastername=134amd64-quarterly&build=37270c265761
mongodb[5678]0 does not reliably fail.
> Back in 2022-Jul there was this to avoid "Signal 9" problems:
>
> A commit in branch main references this bug:
>
> URL:
https://cgit.FreeBSD.org/ports/commit/?id=0aabc02be18794a56c436233e59379ebaa86c2e0
>
> commit 0aabc02be18794a56c436233e59379ebaa86c2e0
> Author: Daniel Engberb <[email protected]>
> AuthorDate: 2022-07-14 14:43:02 +0000
> Commit: Fernando Apesteguía <[email protected]>
> CommitDate: 2022-07-16 15:11:58 +0000
>
> databases/mongodb50: Disable LTO by default.
>
> Building mongo with LTO enabled takes a ridiculously amount of memory
> (32GB in amd64).
> Disable -ggdb so memory requirements don't sky rocket (around 10-12GB)
>
> PR: 264340
> Reported by: [email protected]
> Approved by: [email protected] (maintainer)
>
> databases/mongodb50/files/patch-SConstruct | 15 +++++++++++----
> 1 file changed, 11 insertions(+), 4 deletions(-)
>
>
> There was also a comment reporting console log text like:
>
> swap_pager: out of swap space
> swp_pager_getswapspace(2): failed
> swap_pager: out of swap space
> swp_pager_getswapspace(2): failed
> swp_pager_getswapspace(5): failed
> swp_pager_getswapspace(1): failed
> swp_pager_getswapspace(23): failed
> pid 73953 (ld.lld), jid 0, uid 0, was killed: failed to reclaim memory
>
> But, to my knowledge, such information is not publicly
> available and would be not easily tied to match the
> timing of mongodb* build failures.
>
> So personal/local builds may be needed to figure out if
> such might be involved.
>
>> But this build does not have all the warnings. I still don't see the cause
of Error -9.
>>
>>
https://pkg-status.freebsd.org/beefy22/data/142amd64-default/02e9dc3b903b/logs/mongodb70-7.0.21_1.log
>>
>> https://portsfallout.com/fallout?port=databases%2Fmongodb70%24
>>
>> My next idea is that this occurs because a process gets killed (-9), which
should be visible in the logs of the cluster maintainers. Does this makes sense?
===
Mark Millard
marklmi at yahoo.com
