https://gcc.gnu.org/bugzilla/show_bug.cgi?id=109098
Bug ID: 109098
Summary: Encoding errors on SARIF output for non-UTF-8 source
files
Product: gcc
Version: 13.0
Status: UNCONFIRMED
Severity: normal
Priority: P3
Component: analyzer
Assignee: dmalcolm at gcc dot gnu.org
Reporter: dmalcolm at gcc dot gnu.org
Target Milestone: ---
I've run into issues with -fanalyzer integration testing when a diagnostic
occurs in a source file with non-UTF-8 bytes in it.
Specifically, haproxy-2.7.1's include/import/ist.h has a comment within
function "istalloc" that isn't quite UTF-8.
When a diagnostic occurs on this, with -fdiagnostics-format=sarif-file, the
file is quoted in the SARIF output, but the individual bytes are copied
directly into the JSON, leading to e.g.:
$ sarif ls integration-tests/haproxy-2.7.1/haproxy-2.7.1/http_client.c.sarif
Traceback (most recent call last):
File "/usr/local/lib/python3.8/site-packages/sarif/loader.py", line 57, in
load_sarif_file
data = json.load(file_in)
File "/usr/lib64/python3.8/json/__init__.py", line 293, in load
return loads(fp.read(),
File "/usr/lib64/python3.8/codecs.py", line 322, in decode
(result, consumed) = self._buffer_decode(data, self.errors, final)
UnicodeDecodeError: 'utf-8' codec can't decode byte 0x80 in position 96006:
invalid start byte
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/usr/local/bin/sarif", line 8, in <module>
sys.exit(main())
File "/usr/local/lib/python3.8/site-packages/sarif/cmdline/main.py", line 40,
in main
exitcode = args.func(args)
File "/usr/local/lib/python3.8/site-packages/sarif/cmdline/main.py", line
384, in _ls
ls_op.print_ls(args.files_or_dirs, args.output)
File "/usr/local/lib/python3.8/site-packages/sarif/operations/ls_op.py", line
17, in print_ls
sarif_files = loader.load_sarif_files(path)
File "/usr/local/lib/python3.8/site-packages/sarif/loader.py", line 30, in
load_sarif_files
path_exists = _add_path_to_sarif_file_set(path, ret)
File "/usr/local/lib/python3.8/site-packages/sarif/loader.py", line 17, in
_add_path_to_sarif_file_set
sarif_file_set.add_file(load_sarif_file(path))
File "/usr/local/lib/python3.8/site-packages/sarif/loader.py", line 60, in
load_sarif_file
raise IOError(f"Cannot load {file_path}") from exception
OSError: integration-tests/haproxy-2.7.1/haproxy-2.7.1/http_client.c.sarif
I'm working on a fix.
