On Wed, 2024-05-22 at 13:00 +0000, Peter Kjellerstedt wrote:
> > -----Original Message-----
> > From: [email protected]
> > <[email protected]> On Behalf Of Richard Purdie
> > Sent: den 22 maj 2024 11:28
> > To: [email protected]
> > Subject: [OE-core] [PATCH 1/3] base: Switch UNPACKDIR to a subdir of WORKDIR
> >
> > Change do_unpack to unpack files to a subdirectory of WORKDIR instead of
> > WORKDIR
> > itself. There are several good reasons for this but it is mainly about
> > being able
> > to isolate the output of the unpack task and tell the files apart from
> > other things
> > which are created in workdir (logs, sysroots, temp dirs and more).
> >
> > This means that when the do_unpack task reruns, we can clean UNPACKDIR and
> > know
> > we have a standard point to start builds from.
> >
> > It also makes code in tools like devtool and recipetool easier.
> >
> > To reduce the impact to users, if a subdirectory under UNPACKDIR matches
> > the first subdirectory under WORKDIR of S, that directory is moved into
> > position
> > inside WORKDIR. This preserves the behaviour of S = "${WORKDIR}/git",
> > S = "${WORKDIR}/${BPN}" and other commonly used source directory setups.
> >
> > The directory is moved since sadly many autotools based projects can't cope
> > with
> > symlinks in their paths.
> >
> > The patch also updates reproducible and SOURCE_DATE_EPOCH handling to
> > match the new potential source locations. We can get rid of the horrible
> > list of hardcoded directories in WORKDIR to ignore from that code.
> >
> > Signed-off-by: Richard Purdie <[email protected]>
> > ---
> > meta/classes-global/base.bbclass | 28 +++++++++++++++++++++++-----
> > meta/conf/bitbake.conf | 2 +-
> > meta/lib/oe/reproducible.py | 19 ++++++++++---------
> > 3 files changed, 34 insertions(+), 15 deletions(-)
> >
> > diff --git a/meta/classes-global/base.bbclass
> > b/meta/classes-global/base.bbclass
> > index 066f3848f7c..13e91b24a3b 100644
> > --- a/meta/classes-global/base.bbclass
> > +++ b/meta/classes-global/base.bbclass
> > @@ -153,20 +153,38 @@ python base_do_fetch() {
> > }
> >
> > addtask unpack after do_fetch
> > -do_unpack[dirs] = "${UNPACKDIR}"
> > -
> > -do_unpack[cleandirs] = "${@d.getVar('S') if
> > os.path.normpath(d.getVar('S')) != os.path.normpath(d.getVar('WORKDIR'))
> > else os.path.join('${S}', 'patches')}"
> > +do_unpack[cleandirs] = "${UNPACKDIR}"
> >
> > python base_do_unpack() {
> > + import shutil
> > +
> > + sourcedir = d.getVar('S')
> > + # Intentionally keep SOURCE_BASEDIR internal to the task just for SDE
> > + d.setVar("SOURCE_BASEDIR", sourcedir)
> > +
> > src_uri = (d.getVar('SRC_URI') or "").split()
> > if not src_uri:
> > return
> >
> > + basedir = None
> > + unpackdir = d.getVar('UNPACKDIR')
> > + workdir = d.getVar('WORKDIR')
> > + if sourcedir.startswith(workdir) and not
> > sourcedir.startswith(unpackdir):
> > + basedir = sourcedir.replace(workdir, '').strip("/").split('/')[0]
> > + if basedir:
> > + bb.utils.remove(workdir + '/' + basedir, True)
> > + d.setVar("SOURCE_BASEDIR", workdir + '/' + basedir)
> > +
> > try:
> > fetcher = bb.fetch2.Fetch(src_uri, d)
> > fetcher.unpack(d.getVar('UNPACKDIR'))
> > except bb.fetch2.BBFetchException as e:
> > bb.fatal("Bitbake Fetcher Error: " + repr(e))
> > +
> > + if basedir and os.path.exists(unpackdir + '/' + basedir):
> > + # Compatibility magic to ensure ${WORKDIR}/git and ${WORKDIR}/${BP}
> > + # as often used in S work as expected.
> > + shutil.move(unpackdir + '/' + basedir, workdir + '/' + basedir)
> > }
> >
> > SSTATETASKS += "do_deploy_source_date_epoch"
> > @@ -199,8 +217,8 @@ addtask do_deploy_source_date_epoch_setscene
> > addtask do_deploy_source_date_epoch before do_configure after do_patch
> >
> > python create_source_date_epoch_stamp() {
> > - # Version: 1
> > - source_date_epoch = oe.reproducible.get_source_date_epoch(d,
> > d.getVar('S'))
> > + # Version: 2
> > + source_date_epoch = oe.reproducible.get_source_date_epoch(d,
> > d.getVar('SOURCE_BASEDIR') or d.getVar('S'))
> > oe.reproducible.epochfile_write(source_date_epoch,
> > d.getVar('SDE_FILE'), d)
> > }
> > do_unpack[postfuncs] += "create_source_date_epoch_stamp"
>
> The following code is part of the anonymous python() function in base.bbclass:
>
> if os.path.normpath(d.getVar("WORKDIR")) !=
> os.path.normpath(d.getVar("S")):
> d.appendVar("PSEUDO_IGNORE_PATHS", ",${S}")
> if os.path.normpath(d.getVar("WORKDIR")) !=
> os.path.normpath(d.getVar("B")):
> d.appendVar("PSEUDO_IGNORE_PATHS", ",${B}")
>
> Since it is (or will be) an error to have S = "${WORKDIR}", it should now be
> possible to move the addition of ${S} to PSEUDO_IGNORE_PATHS to bitbake.conf.
Yes, we should be able to start finding and cleaning up things like
this.
> I also believe ${UNPACKDIR} should be added to PSEUDO_IGNORE_PATHS.
Why? I don't think we want UNPACKDIR there, we only need S.
> And on a related note, would it make sense to make B = ${WORKDIR} an error
> too?
> In which case the addition of ${B} to PSEUDO_IGNORE_PATHS could also be moved
> to bitbake.conf.
I'd hope no one is doing that so yes, I'd also like to move that. We
can make that an error too.
Cheers,
Richard
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#199750):
https://lists.openembedded.org/g/openembedded-core/message/199750
Mute This Topic: https://lists.openembedded.org/mt/106239726/21656
Group Owner: [email protected]
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub
[[email protected]]
-=-=-=-=-=-=-=-=-=-=-=-
