I'd like to start a discussion about the state of searching packages in
OpenBSD.
To date, we have a number of ways to search, all of them with some kind of
downside, some having more than others. None of them offer the ability to search
for arbitrary strings in both COMMENT and DESCR at the same time.
These search methods are:
- pkg_info -Q:
- Cons:
- Search can only be passed a single word.
- Isn't exhaustive, searching for "quake" returns 5 results (on
openbsd-current as of 2022-04-08). Meanwhile there are many other ports
that have "quake" somewhere.
- Does not search DESCR.
- Has to make outbound connections for each search.
- Pros:
- It's in base.
- pkglocate:
- Cons:
- Only searches file names.
- Not in base.
- You need to know what you are looking for to find it.
- Does not search DESCR.
- Pros:
- Fast.
- Uses base tools.
- Super handy when you know the name (or part of it) of the file you are
looking for!
- sqlports:
- Cons:
- Requires users to know SQL to query it.
- Not available by default.
- Pros:
- Has all the information required, COMMENT, DESCR.. other meta information
that could be used.
- portslist:
- Cons:
- Only contains partial information. No DESCR.
- No tooling exists to make use of it, there is also functionality that can
be built on top of this.
- Requires users to grep, or know what they are already looking for.
- Pros:
- It has COMMENT.
- Browser based tools like ports-readmes-dancer (available locally or via
https://openports.pl/ ):
- Cons:
- Requires a browser
- If you run it local you need to run a webserver which you then access via
your browser.
- Pros:
- The definitive web source for openbsd package info.
Of these methods listed, the "official" way to find packages (per packages(7))
is to install one of: pkglocatedb, sqlports and ports-readmes-dancer.
IMO it would be much better if users didn't have to reach for external tools do
to what is an extremely common task (and one handled by package managers from
other operating systems out of the box).
I have created a tool that checks a few more of these boxes, but not all of
them.
The tool is here: https://github.com/qbit/pkg
Its own pros/cons:
Cons:
- It uses sqlite3 and perl's DBD driver. It is unlikely either of these will
end up being in base.
- It currently duplicates (doesn't have to, but the current implementation
does) the sqlports file while adding a fts5 (
https://www.sqlite.org/fts5.html ) index to it.
- It's a POC.
Pros:
- It's extremely fast compared to the other tools.
- It searches arbitrary text contained in COMMENT and DESCR.
- It's very easy to use and operates how end-users expect a package search
tool to operate.
I am by no means proposing that my tool be imported or even used. I am just
pointing out deficiencies and possible places we can do better. And since
previously I was accused of "trying to peddle my own tool", I want to be
_extremely clear_ about my intention. I DO NOT care what tool ends up
being used to make the default searching capabilities better, just that _it
happens_.
I am also not saying that the above tools can't be improved in some
way.. They indeed can! But they are still not available by default.
I don't know where to go from here. I think it would be very much worth it to
have a tool installed by default that did a better job of searching package
information (both COMMENT and DESCR). I have no clue what it would look like.
Perhaps there is another perl indexing library that would be acceptable to use
in base? I still don't know what that would look like since the index would need
to be built.. and installed by a.. package..?
Maybe it's sufficient to error out and say "please install pkgindex"? Then
actually search when that is there?
Thoughts?
