------- Comment From i...@de.ibm.com 2024-10-07 10:58 EDT-------
I've extracted the compression logic into a stand-alone script, and it doesn't
fail:
#!/usr/bin/env python3
import gzip
import struct
import zlib
value = b"ABABABABABABABABABAB\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00Z"
compressed = b"\037\213\010\010\000\000\000\000\002\377File\000"
bc = zlib.compressobj(6, zlib.DEFLATED, -zlib.MAX_WBITS, zlib.DEF_MEM_LEVEL, 0)
compressed += bc.compress(value)
compressed += bc.flush()
compressed += struct.pack("<L", zlib.crc32(value) & 0xFFFFFFFF)
compressed += struct.pack("<L", len(value) & 0xFFFFFFFF)
print(compressed)
assert gzip.decompress(compressed) == value
I think the problem is rather here:
@staticmethod
def _write_binary_item_base64_encoded(
[...]
for chunk in chunks:
file.write(b"\n ")
file.write(base64.b64encode(chunk))
Example:
>>> base64.b64decode(base64.b64encode(b"a")+base64.b64encode(b"b"))
b'a'
Sometimes this works though:
>>> base64.b64decode(base64.b64encode(b"aaa")+base64.b64encode(b"b"))
b'aaab'
Seems like s390x was unlucky and hit the problematic case. I would
suggest to call b64decode() on each line individually.
--
You received this bug notification because you are a member of Ubuntu
Touch seeded packages, which is subscribed to apport in Ubuntu.
https://bugs.launchpad.net/bugs/2076269
Title:
invalid base64 encoded gzip data on s390x causes autopkgtest failures
Status in Ubuntu on IBM z Systems:
New
Status in apport package in Ubuntu:
New
Status in python3.12 package in Ubuntu:
New
Bug description:
The following tests all fail with an encoding/decoding issue:
tests/unit/test_problem_report.py::T:test_modify
tests/unit/test_report.py::T::test_report_from_systemd_coredump_storage_journal
tests/integration/test_problem_report::T::test_write_file
Example failure:
564s =================================== FAILURES
===================================
564s ________________________________ T.test_modify
_________________________________
564s
564s self = <tests.unit.test_problem_report.T testMethod=test_modify>
564s
564s def test_modify(self):
564s """reading, modifying fields, and writing back."""
564s report = textwrap.dedent(
564s """\
564s ProblemType: Crash
564s Date: now!
564s Long:
564s xxx
564s .
564s yyy
564s Short: Bar
564s File: base64
564s H4sICAAAAAAC/0ZpbGUA
564s c3RyxIAMcBAFAK/2p9MfAAAA
564s """
564s ).encode()
564s
564s pr = problem_report.ProblemReport()
564s pr.load(io.BytesIO(report))
564s
564s self.assertEqual(pr["Long"], "xxx\n.\nyyy")
564s
564s # write back unmodified
564s out = io.BytesIO()
564s pr.write(out)
564s > self.assertEqual(out.getvalue(), report)
564s E AssertionError: b'Pro[73 chars]e64\n H4sICAAAAAAC/0ZpbGUA\n
cnTChAxwEA==\n BRgAr/an0x8AAAA=\n' != b'Pro[73 chars]e64\n
H4sICAAAAAAC/0ZpbGUA\n c3RyxIAMcBAFAK/2p9MfAAAA\n'
564s
564s tests/unit/test_problem_report.py:509: AssertionError
Autopkgtest noble log:
https://autopkgtest.ubuntu.com/results/autopkgtest-noble/noble/s390x/a/apport/20240807_023039_8850a@/log.gz
oracular log:
https://autopkgtest.ubuntu.com/results/autopkgtest-oracular/oracular/s390x/a/apport/20240805_132135_d3a80@/log.gz
This failure was seen on noble (log above) but also on oracular.
To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu-z-systems/+bug/2076269/+subscriptions
--
Mailing list: https://launchpad.net/~touch-packages
Post to : touch-packages@lists.launchpad.net
Unsubscribe : https://launchpad.net/~touch-packages
More help : https://help.launchpad.net/ListHelp
