On Sun, 2021-10-03 at 13:22 +0100, Richard Purdie via lists.openembedded.org wrote: > I was worried about the failures I was seeing in reproducible builds. I > therefore put more instrumentation about where SDE is calculated in builds. > There are some situations we'd not expect to see a value of SDE so I tried to > write logic that describes it. It is harder than you'd think as the value > isn't > valid in fetch, unpack, prepare_recipe_sysroot or *_setscene tasks. I tried to > make the warnings conditional upon BB_CURRENTTASK only to find that gets set > later that the SDE function is called. I did add some hacks to make it get set > earlier but we'll come back to how early it runs. > > Once it has a value, the function writes it to the datastore, since re-reading > it all the time is high overhead. This means that the value is cached and once > set, doesn't change. > > In testing on the autobuilder, I found this log extract: > > NOTE: Running task 196 of 2214 > (virtual:native:/home/pokybuild/yocto-worker/buildtools/build/meta/recipes-devtools/rsync/rsync_3.2.3.bb:do_patch) > [...] > WARNING: Cannot find > /home/pokybuild/yocto-worker/buildtools/build/build/tmp/work/core2-64-poky-linux/rsync/3.2.3-r0/source-date-epoch/__source_date_epoch.txt. > SOURCE_DATE_EPOCH will default to 1302044400 > [...] > NOTE: recipe rsync-native-3.2.3-r0: task do_patch: Started > > buried in: > > https://autobuilder.yoctoproject.org/typhoon/#/builders/20/builds/4430/steps/11/logs/stdio > > What worries me there is that rsync-native is looking in core2-64-poky-linux, > i.e. the target rsync. How could that happen? > > My theory is that SDE is being called early in parsing before the > BBCLASSEXTEND > has modified PN and friends. > > At this point I'm very worried as the whole situation looks error and race > prone > . It may explain some of the instabilities I've been seeing though.
The above isn't as bad as it seems. When parsing the datastore during the recipe run the main datastore is finalized before the variants are finalized, only the latter ends up being used. This is the real problem: https://autobuilder.yoctoproject.org/typhoon/#/builders/15/builds/4386/steps/11/logs/warnings i.e. the SDE caching is switching between different WORKDIRs as the datastore is changed. I've sent a workaround for this but it needs further thought. Cheers, Richard
-=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#156558): https://lists.openembedded.org/g/openembedded-core/message/156558 Mute This Topic: https://lists.openembedded.org/mt/86040862/21656 Group Owner: [email protected] Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [[email protected]] -=-=-=-=-=-=-=-=-=-=-=-
