Bug#1074599: command-not-found: UnicodeDecodeError: 'utf-8' codec can't decode byte 0xed in position 15: invalid continuation byte

[Otto Kekäläinen]

Source: command-not-found
Version: 23.04.0-1
Severity: severe

Running /usr/lib/cnf-update-db failed on:

Traceback (most recent call last):
  File "/usr/lib/cnf-update-db", line 32, in <module>
    col.create(db)
  File "/usr/share/command-not-found/CommandNotFound/db/creator.py",
line 96, in create
    self._fill_commands(con)
  File "/usr/share/command-not-found/CommandNotFound/db/creator.py",
line 141, in _fill_commands
    self._parse_single_contents_file(con, f, sub.stdout)
  File "/usr/share/command-not-found/CommandNotFound/db/creator.py",
line 242, in _parse_single_contents_file
    if not (l.startswith('usr/sbin') or l.startswith('usr/bin') or
        ^^^^^^^^^^^^^^^^^
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xed in position
15: invalid continuation byte

The command is currently unusable.

I added a print statement and seems the parser is failing on file from aspell:


b'usr/lib/aspell/xh.dat\t\t\t\t\t    universe/text/aspell-xh\n'
b'usr/lib/aspell/xh.multi\t\t\t\t\t    universe/text/aspell-xh\n'
b'usr/lib/aspell/xh.rws\t\t\t\t\t    universe/text/aspell-xh\n'
b'usr/lib/aspell/xhosa.alias\t\t\t\t    universe/text/aspell-xh\n'
b'usr/lib/aspell/zu.dat\t\t\t\t\t    universe/text/aspell-zu\n'
b'usr/lib/aspell/zu.multi\t\t\t\t\t    universe/text/aspell-zu\n'
b'usr/lib/aspell/zu.rws\t\t\t\t\t    universe/text/aspell-zu\n'
b'usr/lib/aspell/zulu.alias\t\t\t\t    universe/text/aspell-zu\n'
b'usr/lib/aspell/\xedslenska.alias\t\t\t\t    universe/text/aspell-is\n'
Traceback (most recent call last):
  File "/usr/lib/cnf-update-db", line 32, in <module>
    col.create(db)
  File "/usr/share/command-not-found/CommandNotFound/db/creator.py",
line 96, in create
    self._fill_commands(con)
  File "/usr/share/command-not-found/CommandNotFound/db/creator.py",
line 141, in _fill_commands
    self._parse_single_contents_file(con, f, sub.stdout)
  File "/usr/share/command-not-found/CommandNotFound/db/creator.py",
line 242, in _parse_single_contents_file
    if not (l.startswith('usr/sbin') or l.startswith('usr/bin') or
        ^^^^^^^^^^^^^^^^^
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xed in position
15: invalid continuation byte


This seems to be trigered by the i in "Islenska":

/usr/lib/aspell/íslenska.alias aspell-is

See 
https://packages.debian.org/search?mode=path&suite=bookworm&section=all&arch=any&searchon=contents&keywords=slenska.alias

As a temporary workaround I used this try-statement:

    def _parse_single_contents_file(self, con, f, fp):
        # read header
        suite=None      # FIXME

        for l in fp:
            try:
               l = l.decode("utf-8")
            except UnicodeDecodeError:
                continue

            if not (l.startswith('usr/sbin') or l.startswith('usr/bin') or
                    l.startswith('bin') or l.startswith('sbin')):
                continue
            try:
                command, pkgnames = l.split(None, 1)
            except ValueError:
                continue