Hi Bertrand,
On 19/10/2022 12:40, Bertrand Marquis wrote:
>
>
> Hi Michal,
>
>> On 19 Oct 2022, at 10:06, Michal Orzel <[email protected]> wrote:
>>
>> Hi Stefano,
>>
>> On 19/10/2022 02:02, Stefano Stabellini wrote:
>>>
>>>
>>> On Mon, 17 Oct 2022, Stefano Stabellini wrote:
>>>> It should be
>>>>
>>>> BB_NUMBER_THREADS="2"
>>>>
>>>> but that worked! Let me a couple of more tests.
>>>
>>> I could run successfully a Yocto build test with qemuarm64 as target in
>>> gitlab-ci, hurray! No size issues, no build time issues, everything was
>>> fine. See:
>>> https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgitlab.com%2Fxen-project%2Fpeople%2Fsstabellini%2Fxen%2F-%2Fjobs%2F3193051236&data=05%7C01%7Cmichal.orzel%40amd.com%7C5f7fc3a161fe44b5954808dab1be5c3a%7C3dd8961fe4884e608e11a82d994e183d%7C0%7C0%7C638017728406088513%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=2mb3N26wiz39RJNSA4KoIOt%2BG9X7EMDOWIpfKc2ZZOc%3D&reserved=0
>>> https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgitlab.com%2Fxen-project%2Fpeople%2Fsstabellini%2Fxen%2F-%2Fjobs%2F3193083119&data=05%7C01%7Cmichal.orzel%40amd.com%7C5f7fc3a161fe44b5954808dab1be5c3a%7C3dd8961fe4884e608e11a82d994e183d%7C0%7C0%7C638017728406088513%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=QhTFefS8NU1f7oLemB0Vtn%2BDCD%2BCnq1v1gEmlKCJt84%3D&reserved=0
>>>
>>> I made the appended changes in top of this series.
>>>
>>> - I pushed registry.gitlab.com/xen-project/xen/yocto:kirkstone and
>>> registry.gitlab.com/xen-project/xen/yocto:kirkstone-qemuarm64
>>> - for the gitlab-ci runs, we need to run build-yocto.sh from the copy in
>>> xen.git, not from a copy stored inside a container
>>> - when building the kirkstone-qemuarm64 container the first time
>>> (outside of gitlab-ci) I used COPY and took the script from the local
>>> xen.git tree
>>> - after a number of tests, I settled on: BB_NUMBER_THREADS="8" more than
>>> this and it breaks on some workstations, please add it
>>> - I am running the yocto build on arm64 so that we can use the arm64
>>> hardware to do it in gitlab-ci
>>>
>>> Please feel free to incorporate these changes in your series, and add
>>> corresponding changes for the qemuarm32 and qemux86 targets.
>>>
>>> I am looking forward to it! Almost there!
>>>
>>> Cheers,
>>>
>>> Stefano
>>>
>>>
>>> diff --git a/automation/build/yocto/build-yocto.sh
>>> b/automation/build/yocto/build-yocto.sh
>>> index 0d31dad607..16f1dcc0a5 100755
>>> --- a/automation/build/yocto/build-yocto.sh
>>> +++ b/automation/build/yocto/build-yocto.sh
>>> @@ -107,6 +107,9 @@ IMAGE_INSTALL:append:pn-xen-image-minimal = "
>>> ssh-pregen-hostkeys"
>>> # Save some disk space
>>> INHERIT += "rm_work"
>>>
>>> +# Reduce number of jobs
>>> +BB_NUMBER_THREADS="8"
>>> +
>>> EOF
>>>
>>> if [ "${do_localsrc}" = "y" ]; then
>>> diff --git a/automation/build/yocto/kirkstone-qemuarm64.dockerfile
>>> b/automation/build/yocto/kirkstone-qemuarm64.dockerfile
>>> index f279a7af92..aea3fc1f3e 100644
>>> --- a/automation/build/yocto/kirkstone-qemuarm64.dockerfile
>>> +++ b/automation/build/yocto/kirkstone-qemuarm64.dockerfile
>>> @@ -16,7 +16,8 @@ ARG target=qemuarm64
>>>
>>> # This step can take one to several hours depending on your download
>>> bandwith
>>> # and the speed of your computer
>>> -RUN /home/$USER_NAME/bin/build-yocto.sh --dump-log $target
>>> +COPY ./build-yocto.sh /
>>> +RUN /build-yocto.sh --dump-log $target
>>>
>>> FROM $from_image
>>>
>>> diff --git a/automation/build/yocto/kirkstone.dockerfile
>>> b/automation/build/yocto/kirkstone.dockerfile
>>> index 367a7863b6..ffbd91aa90 100644
>>> --- a/automation/build/yocto/kirkstone.dockerfile
>>> +++ b/automation/build/yocto/kirkstone.dockerfile
>>> @@ -84,9 +84,6 @@ RUN mkdir -p /home/$USER_NAME/yocto-layers \
>>> /home/$USER_NAME/xen && \
>>> chown $USER_NAME.$USER_NAME /home/$USER_NAME/*
>>>
>>> -# Copy the build script
>>> -COPY build-yocto.sh /home/$USER_NAME/bin/
>>> -
>>> # clone yocto repositories we need
>>> ARG yocto_version="kirkstone"
>>> RUN for rep in \
>>> diff --git a/automation/gitlab-ci/build.yaml
>>> b/automation/gitlab-ci/build.yaml
>>> index ddc2234faf..4b8bcde252 100644
>>> --- a/automation/gitlab-ci/build.yaml
>>> +++ b/automation/gitlab-ci/build.yaml
>>> @@ -584,6 +584,22 @@ alpine-3.12-gcc-arm64-boot-cpupools:
>>> EXTRA_XEN_CONFIG: |
>>> CONFIG_BOOT_TIME_CPUPOOLS=y
>>>
>>> +yocto-kirkstone-qemuarm64:
>>> + stage: build
>>> + image: registry.gitlab.com/xen-project/xen/${CONTAINER}
>>> + script:
>>> + - ./automation/build/yocto/build-yocto.sh -v --log-dir=./logs
>>> --xen-dir=`pwd` qemuarm64
>>> + variables:
>>> + CONTAINER: yocto:kirkstone-qemuarm64
>>> + artifacts:
>>> + paths:
>>> + - '*.log'
>>> + - '*/*.log'
>> The above lines are not needed as the logs/* below will handle them all
>> (logs are only stored in logs/).
>
> Ack
>
>>
>>> + - 'logs/*'
>>> + when: always
>>> + tags:
>>> + - arm64
>>> +
>> build-yocto.sh performs both build and run actions. I think it'd be better
>> to move this into test.yaml in that case.
>> The best would be to create one build job (specifying --no-run) in
>> build.yaml and one test job (specifying --no-build) in test.yaml.
>> This however would probably require marking path
>> build/tmp/deploy/***/qemuarm64 as an build artifact. The question then is
>> whether having this path would be enough for runqemu (Bertrand's opinion
>> needed).
>
> This will not be enough to run qemu as the qemu binary and its dependencies
> are in the build artifacts and not in deploy.
> Splitting the build and run is not a good idea because the size of the
> artifact between the 2 will be huge.
>
>>
>> Apart from that there is an aspect of Yocto releases and the
>> containers/tests names.
>> Yocto needs to be up-to-date in order to properly build Xen+tools.
>> This basically means that we will need to update the containers once
>> per Yocto release. The old containers would still need to be stored in our
>> CI container registry
>> so that we can use CI for older versions of Xen. However, updating the
>> containers would also require
>> modifying the existing tests (for now we have e.g. yocto-kirkstone-qemuarm64
>> but in a month we will have
>> to change them to yocto-langdale-qemuarm64). In a few years time this will
>> result in several CI jobs
>> that are the same but differ only in name/container. I would thus suggest to
>> name the CI jobs like this:
>> yocto-qemuarm64 (without yocto release name) and define the top-level
>> YOCTO_CONTAINER variable to store
>> the current yocto release container. This will solve the issue I described
>> above.
>
> I think we have no other way around this and we will need to have one Yocto
> release supported by Xen officially so
> we will have to keep old docker images for old releases of Xen and move to
> newer versions of Yocto in staging when
> it is needed.
>
> We have to find a way for gitlab-ci to use the build.yaml contained inside
> the tree that is to be tested somehow so that gitlab would automatically take
> the right one.
> Which means that build.yaml will be different between branches and contain
> the right version for the current branch.
>
What I suggest is that with each new yocto release, we add new docker container
files and push them to registry.
So we will end up in a registry having e.g. (arm64 as an example):
- kirkstone-qemuarm64
- langdale-qemuarm64
We maintain only the one group of CI jobs whose names are generic
(yocto-qemuarm64).
After adding new containers for a new Yocto release, we modify the
YOCTO_RELEASE variable
to point to the latest yocto release containers.
test.yaml:
...
# Yocto test jobs
variables:
YOCTO_RELEASE: "kirkstone"
yocto-qemuarm64:
extends: .test-jobs-common
script:
- ./automation/build/yocto/build-yocto.sh -v --log-dir=./logs
--xen-dir=`pwd` qemuarm64
variables:
CONTAINER: yocto:${YOCTO_RELEASE}-qemuarm64
artifacts:
paths:
- 'logs/*'
when: always
tags:
- arm64
This means that:
- on the current staging branch the YOCTO_RELEASE points to the latest
containers (for the latest yocto release)
- on the old stable branches the YOCTO_RELEASE points to the old containers
(for the old yocto release).
~Michal
