On Wed, 19 Oct 2022, Michal Orzel wrote:
> Hi Bertrand,
>
> On 19/10/2022 12:40, Bertrand Marquis wrote:
> >
> >
> > Hi Michal,
> >
> >> On 19 Oct 2022, at 10:06, Michal Orzel <[email protected]> wrote:
> >>
> >> Hi Stefano,
> >>
> >> On 19/10/2022 02:02, Stefano Stabellini wrote:
> >>>
> >>>
> >>> On Mon, 17 Oct 2022, Stefano Stabellini wrote:
> >>>> It should be
> >>>>
> >>>> BB_NUMBER_THREADS="2"
> >>>>
> >>>> but that worked! Let me a couple of more tests.
> >>>
> >>> I could run successfully a Yocto build test with qemuarm64 as target in
> >>> gitlab-ci, hurray! No size issues, no build time issues, everything was
> >>> fine. See:
> >>> https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgitlab.com%2Fxen-project%2Fpeople%2Fsstabellini%2Fxen%2F-%2Fjobs%2F3193051236&data=05%7C01%7Cmichal.orzel%40amd.com%7C5f7fc3a161fe44b5954808dab1be5c3a%7C3dd8961fe4884e608e11a82d994e183d%7C0%7C0%7C638017728406088513%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=2mb3N26wiz39RJNSA4KoIOt%2BG9X7EMDOWIpfKc2ZZOc%3D&reserved=0
> >>> https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgitlab.com%2Fxen-project%2Fpeople%2Fsstabellini%2Fxen%2F-%2Fjobs%2F3193083119&data=05%7C01%7Cmichal.orzel%40amd.com%7C5f7fc3a161fe44b5954808dab1be5c3a%7C3dd8961fe4884e608e11a82d994e183d%7C0%7C0%7C638017728406088513%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=QhTFefS8NU1f7oLemB0Vtn%2BDCD%2BCnq1v1gEmlKCJt84%3D&reserved=0
> >>>
> >>> I made the appended changes in top of this series.
> >>>
> >>> - I pushed registry.gitlab.com/xen-project/xen/yocto:kirkstone and
> >>> registry.gitlab.com/xen-project/xen/yocto:kirkstone-qemuarm64
> >>> - for the gitlab-ci runs, we need to run build-yocto.sh from the copy in
> >>> xen.git, not from a copy stored inside a container
> >>> - when building the kirkstone-qemuarm64 container the first time
> >>> (outside of gitlab-ci) I used COPY and took the script from the local
> >>> xen.git tree
> >>> - after a number of tests, I settled on: BB_NUMBER_THREADS="8" more than
> >>> this and it breaks on some workstations, please add it
> >>> - I am running the yocto build on arm64 so that we can use the arm64
> >>> hardware to do it in gitlab-ci
> >>>
> >>> Please feel free to incorporate these changes in your series, and add
> >>> corresponding changes for the qemuarm32 and qemux86 targets.
> >>>
> >>> I am looking forward to it! Almost there!
> >>>
> >>> Cheers,
> >>>
> >>> Stefano
> >>>
> >>>
> >>> diff --git a/automation/build/yocto/build-yocto.sh
> >>> b/automation/build/yocto/build-yocto.sh
> >>> index 0d31dad607..16f1dcc0a5 100755
> >>> --- a/automation/build/yocto/build-yocto.sh
> >>> +++ b/automation/build/yocto/build-yocto.sh
> >>> @@ -107,6 +107,9 @@ IMAGE_INSTALL:append:pn-xen-image-minimal = "
> >>> ssh-pregen-hostkeys"
> >>> # Save some disk space
> >>> INHERIT += "rm_work"
> >>>
> >>> +# Reduce number of jobs
> >>> +BB_NUMBER_THREADS="8"
> >>> +
> >>> EOF
> >>>
> >>> if [ "${do_localsrc}" = "y" ]; then
> >>> diff --git a/automation/build/yocto/kirkstone-qemuarm64.dockerfile
> >>> b/automation/build/yocto/kirkstone-qemuarm64.dockerfile
> >>> index f279a7af92..aea3fc1f3e 100644
> >>> --- a/automation/build/yocto/kirkstone-qemuarm64.dockerfile
> >>> +++ b/automation/build/yocto/kirkstone-qemuarm64.dockerfile
> >>> @@ -16,7 +16,8 @@ ARG target=qemuarm64
> >>>
> >>> # This step can take one to several hours depending on your download
> >>> bandwith
> >>> # and the speed of your computer
> >>> -RUN /home/$USER_NAME/bin/build-yocto.sh --dump-log $target
> >>> +COPY ./build-yocto.sh /
> >>> +RUN /build-yocto.sh --dump-log $target
> >>>
> >>> FROM $from_image
> >>>
> >>> diff --git a/automation/build/yocto/kirkstone.dockerfile
> >>> b/automation/build/yocto/kirkstone.dockerfile
> >>> index 367a7863b6..ffbd91aa90 100644
> >>> --- a/automation/build/yocto/kirkstone.dockerfile
> >>> +++ b/automation/build/yocto/kirkstone.dockerfile
> >>> @@ -84,9 +84,6 @@ RUN mkdir -p /home/$USER_NAME/yocto-layers \
> >>> /home/$USER_NAME/xen && \
> >>> chown $USER_NAME.$USER_NAME /home/$USER_NAME/*
> >>>
> >>> -# Copy the build script
> >>> -COPY build-yocto.sh /home/$USER_NAME/bin/
> >>> -
> >>> # clone yocto repositories we need
> >>> ARG yocto_version="kirkstone"
> >>> RUN for rep in \
> >>> diff --git a/automation/gitlab-ci/build.yaml
> >>> b/automation/gitlab-ci/build.yaml
> >>> index ddc2234faf..4b8bcde252 100644
> >>> --- a/automation/gitlab-ci/build.yaml
> >>> +++ b/automation/gitlab-ci/build.yaml
> >>> @@ -584,6 +584,22 @@ alpine-3.12-gcc-arm64-boot-cpupools:
> >>> EXTRA_XEN_CONFIG: |
> >>> CONFIG_BOOT_TIME_CPUPOOLS=y
> >>>
> >>> +yocto-kirkstone-qemuarm64:
> >>> + stage: build
> >>> + image: registry.gitlab.com/xen-project/xen/${CONTAINER}
> >>> + script:
> >>> + - ./automation/build/yocto/build-yocto.sh -v --log-dir=./logs
> >>> --xen-dir=`pwd` qemuarm64
> >>> + variables:
> >>> + CONTAINER: yocto:kirkstone-qemuarm64
> >>> + artifacts:
> >>> + paths:
> >>> + - '*.log'
> >>> + - '*/*.log'
> >> The above lines are not needed as the logs/* below will handle them all
> >> (logs are only stored in logs/).
> >
> > Ack
> >
> >>
> >>> + - 'logs/*'
> >>> + when: always
> >>> + tags:
> >>> + - arm64
> >>> +
> >> build-yocto.sh performs both build and run actions. I think it'd be better
> >> to move this into test.yaml in that case.
> >> The best would be to create one build job (specifying --no-run) in
> >> build.yaml and one test job (specifying --no-build) in test.yaml.
> >> This however would probably require marking path
> >> build/tmp/deploy/***/qemuarm64 as an build artifact. The question then is
> >> whether having this path would be enough for runqemu (Bertrand's opinion
> >> needed).
> >
> > This will not be enough to run qemu as the qemu binary and its dependencies
> > are in the build artifacts and not in deploy.
> > Splitting the build and run is not a good idea because the size of the
> > artifact between the 2 will be huge.
> >
> >>
> >> Apart from that there is an aspect of Yocto releases and the
> >> containers/tests names.
> >> Yocto needs to be up-to-date in order to properly build Xen+tools.
> >> This basically means that we will need to update the containers once
> >> per Yocto release. The old containers would still need to be stored in our
> >> CI container registry
> >> so that we can use CI for older versions of Xen. However, updating the
> >> containers would also require
> >> modifying the existing tests (for now we have e.g.
> >> yocto-kirkstone-qemuarm64 but in a month we will have
> >> to change them to yocto-langdale-qemuarm64). In a few years time this will
> >> result in several CI jobs
> >> that are the same but differ only in name/container. I would thus suggest
> >> to name the CI jobs like this:
> >> yocto-qemuarm64 (without yocto release name) and define the top-level
> >> YOCTO_CONTAINER variable to store
> >> the current yocto release container. This will solve the issue I described
> >> above.
> >
> > I think we have no other way around this and we will need to have one Yocto
> > release supported by Xen officially so
> > we will have to keep old docker images for old releases of Xen and move to
> > newer versions of Yocto in staging when
> > it is needed.
> >
> > We have to find a way for gitlab-ci to use the build.yaml contained inside
> > the tree that is to be tested somehow so that gitlab would automatically
> > take the right one.
> > Which means that build.yaml will be different between branches and contain
> > the right version for the current branch.
> >
>
> What I suggest is that with each new yocto release, we add new docker
> container files and push them to registry.
> So we will end up in a registry having e.g. (arm64 as an example):
> - kirkstone-qemuarm64
> - langdale-qemuarm64
> We maintain only the one group of CI jobs whose names are generic
> (yocto-qemuarm64).
> After adding new containers for a new Yocto release, we modify the
> YOCTO_RELEASE variable
> to point to the latest yocto release containers.
>
> test.yaml:
> ...
> # Yocto test jobs
> variables:
> YOCTO_RELEASE: "kirkstone"
>
> yocto-qemuarm64:
> extends: .test-jobs-common
> script:
> - ./automation/build/yocto/build-yocto.sh -v --log-dir=./logs
> --xen-dir=`pwd` qemuarm64
> variables:
> CONTAINER: yocto:${YOCTO_RELEASE}-qemuarm64
> artifacts:
> paths:
> - 'logs/*'
> when: always
> tags:
> - arm64
>
> This means that:
> - on the current staging branch the YOCTO_RELEASE points to the latest
> containers (for the latest yocto release)
> - on the old stable branches the YOCTO_RELEASE points to the old containers
> (for the old yocto release).
I think that's a good idea
