I think probably you meant to send this to yocto-patches@ ML?
On 3/30/26 15:51, Piotr Buliński via lists.openembedded.org wrote: > When I was using `bitbake-layers layerindex-fetch ...` I noticed it being > quite > slow. Upon further investigation, I found that the call to `load_layerindex()` > inside layerindexlib was called with the deafult `load` argument. > > The deafault was set to ['layerDependencies', 'recipes', 'machines', > 'distros'] > where the `layerindex-fetch` effectively requires only ['layerDependencies']. > > So whenever a `layerindex-fetch` is used, we pull for example 'recipes', which > is about 18-20MB in size - it takes time to produce that response by the > layerindex-web and ship over the internet, just to be discarded. > > This change addresses the issue by setting a "sane" default (empty `load` > list) > and sets a correct, explicit `load` list whenever it's needed (e.g. toaster) > by the caller. > > Lastly, there seems to be an error when the cooker index is loaded in the > process with the `load='layerDependencies'`, that's incorrect, as the > implementation expects also a list, so that's corrected as well. > > Signed-off-by: Piotr Buliński <[email protected]> > --- > lib/bblayers/layerindex.py | 4 ++-- > lib/layerindexlib/__init__.py | 6 +++--- > lib/toaster/orm/management/commands/lsupdates.py | 3 ++- > 3 files changed, 7 insertions(+), 6 deletions(-) > > diff --git a/lib/bblayers/layerindex.py b/lib/bblayers/layerindex.py > index 308a5532d..2ba1103a3 100644 > --- a/lib/bblayers/layerindex.py > +++ b/lib/bblayers/layerindex.py > @@ -116,7 +116,7 @@ class LayerIndexPlugin(ActionPlugin): > > # Load the cooker DB > cookerIndex = layerindexlib.LayerIndex(self.tinfoil.config_data) > - cookerIndex.load_layerindex('cooker://', load='layerDependencies') > + cookerIndex.load_layerindex('cooker://', load=['layerDependencies']) > > # Fast path, check if we already have what has been requested! > (dependencies, invalidnames) = > cookerIndex.find_dependencies(names=args.layername, ignores=ignore_layers) > @@ -137,7 +137,7 @@ class LayerIndexPlugin(ActionPlugin): > > for remoteurl in _construct_url(apiurl, branches): > logger.plain("Loading %s..." % remoteurl) > - remoteIndex.load_layerindex(remoteurl) > + remoteIndex.load_layerindex(remoteurl, > load=['layerDependencies']) > > if remoteIndex.is_empty(): > logger.error("Remote layer index %s is empty for branches > %s" % (apiurl, branches)) > diff --git a/lib/layerindexlib/__init__.py b/lib/layerindexlib/__init__.py > index c3265ddaa..e693e5c11 100644 > --- a/lib/layerindexlib/__init__.py > +++ b/lib/layerindexlib/__init__.py > @@ -174,15 +174,15 @@ class LayerIndex(): > return res > > > - def load_layerindex(self, indexURI, load=['layerDependencies', > 'recipes', 'machines', 'distros'], reload=False): > + def load_layerindex(self, indexURI, load=[], reload=False): > '''Load the layerindex. > > indexURI - An index to load. (Use multiple calls to load > multiple indexes) > > reload - If reload is True, then any previously loaded indexes > will be forgotten. > > - load - List of elements to load. Default loads all items. > - Note: plugs may ignore this. > + load - List of elements to load. By default, an empty list is > used to keep things lean. > + Callers need to specify a minimal set of elements to load, > such as ['layerDependencies'] for dependency resolution. > > The format of the indexURI: > > diff --git a/lib/toaster/orm/management/commands/lsupdates.py > b/lib/toaster/orm/management/commands/lsupdates.py > index 0ee00aa15..153b26f60 100644 > --- a/lib/toaster/orm/management/commands/lsupdates.py > +++ b/lib/toaster/orm/management/commands/lsupdates.py > @@ -105,7 +105,8 @@ class Command(BaseCommand): > url_branches = ";branch=%s" % ','.join(allowed_branch_names) > else: > url_branches = "" > - layerindex.load_layerindex("%s%s" % (self.apiurl, url_branches)) > + layerindex.load_layerindex("%s%s" % (self.apiurl, url_branches), > + load=['layerDependencies', 'recipes', > 'machines', 'distros']) > > http_progress.stop() > > > > > >
-=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#125867): https://lists.openembedded.org/g/openembedded-devel/message/125867 Mute This Topic: https://lists.openembedded.org/mt/118578936/21656 Group Owner: [email protected] Unsubscribe: https://lists.openembedded.org/g/openembedded-devel/unsub [[email protected]] -=-=-=-=-=-=-=-=-=-=-=-
