** Description changed: - sos 4.3 seems to have a regression in mac address obfuscation. We found - this in the ubuntu package testing. The file concerned seems to be the - etc/netplan/50-cloud-init.yaml which seems to end up with an - unobfuscated mac address inspite of using --mask flag to sos report - command. - - ---- - - autopkgtest run shows, - - Found 1 total reports to obfuscate, processing up to 4 concurrently - - sosreport-autopkgtest-2022-03-02-kluxwcz : Beginning obfuscation... - sosreport-autopkgtest-2022-03-02-kluxwcz : Obfuscation completed [removed 16 unprocessable files] - - Successfully obfuscated 1 report(s) - - Creating compressed archive... - - A mapping of obfuscated elements is available at - /tmp/sosreport-host0-2022-03-02-kluxwcz-private_map - - Your sosreport has been generated and saved in: - /tmp/sosreport-host0-2022-03-02-kluxwcz-obfuscated.tar.xz - - Size 2.28MiB - Owner root - sha256 42db961f8cde1aa72f78afbef825d7bd54884e76996f96ce657a37fca5e1fa44 - - Please send this file to your support representative. - - ### end stdout - ### start extraction - ### stop extraction - ######### DONE WITH --mask ######### - !!! TEST FAILED: MAC address not obfuscated in all places !!! - /tmp/sosreport_test/etc/netplan/50-cloud-init.yaml: macaddress: '52:54:00:12:34:56' + [Impact] + + sos 4.3 has a regression in mac address obfuscation. The file + etc/netplan/50-cloud-init.yaml ends up with an unobfuscated mac address + inspite of using --mask. + + [TEST PLAN] + + Documentation for Special Cases: + https://wiki.ubuntu.com/SosreportUpdates + + [WHERE PROBLEMS COULD OCCUR] + + Since we are changing the regex parser code in + sos/cleaner/parsers/mac_parser.py we would need to ensure no other regex + behavior is changed. The unit tests in autopkgtest will suffice to + determine that. ------------- - + [Other Info] + + Upstream issue is https://github.com/sosreport/sos/issues/2873 + Upstream MR is https://github.com/sosreport/sos/pull/2875 + + Reproducer details: sos 4.2 shows correct behavior. testing shows.. /etc/netplan/50-cloud-init.yaml contains network: - ethernets: - ens3: - dhcp4: true - match: - macaddress: '52:54:00:12:34:56' - set-name: ens3 - version: 2 - - - 4.2 sos contains the file but with the obfuscated mac address. correct behavior. - - # This file is generated from information provided by the datasource. Changes - # to it will not persist across an instance reboot. To disable cloud-init's - # network configuration capabilities, write a file - # /etc/cloud/cloud.cfg.d/99-disable-network-config.cfg with the following: - # network: {config: disabled} - network: - ethernets: - ens3: - dhcp4: true - match: - macaddress: '53:4f:53:cf:3a:9e' - set-name: ens3 - version: 2 - + ethernets: + ens3: + dhcp4: true + match: + macaddress: '52:54:00:12:34:56' + set-name: ens3 + version: 2 + + 4.2 sos contains the file but with the obfuscated mac address. correct + behavior. + + # This file is generated from information provided by the datasource. Changes + # to it will not persist across an instance reboot. To disable cloud-init's + # network configuration capabilities, write a file + # /etc/cloud/cloud.cfg.d/99-disable-network-config.cfg with the following: + # network: {config: disabled} + network: + ethernets: + ens3: + dhcp4: true + match: + macaddress: '53:4f:53:cf:3a:9e' + set-name: ens3 + version: 2 ---------------------- 4.3 testing shows the bug, the /etc/netplan/50-cloud-init.yaml contains # This file is generated from information provided by the datasource. Changes # to it will not persist across an instance reboot. To disable cloud-init's # network configuration capabilities, write a file # /etc/cloud/cloud.cfg.d/99-disable-network-config.cfg with the following: # network: {config: disabled} network: - ethernets: - ens3: - dhcp4: true - match: - macaddress: '52:54:00:12:34:56' - set-name: ens3 - version: 2 - + ethernets: + ens3: + dhcp4: true + match: + macaddress: '52:54:00:12:34:56' + set-name: ens3 + version: 2 --- generated sosreport (run with --mask) contains # This file is generated from information provided by the datasource. Changes # to it will not persist across an instance reboot. To disable cloud-init's # network configuration capabilities, write a file # /etc/cloud/cloud.cfg.d/99-disable-network-config.cfg with the following: # network: {config: disabled} network: - ethernets: - ens3: - dhcp4: true - match: - macaddress: '52:54:00:12:34:56' - set-name: ens3 - version: 2 + ethernets: + ens3: + dhcp4: true + match: + macaddress: '52:54:00:12:34:56' + set-name: ens3 + version: 2 So the mac address isnt being obfuscated. Whats interesting is.. sosreport-host0-2022-03-02-kluxwcz-private_map stored in /tmp does indeed contain that entry and the obfuscated entry too.. - { - "hostname_map": { - "autopkgtest": "host0" - }, - "ip_map": { - "10.0.2.0/24": "100.0.0.0/24", - "10.0.2.15/24": "100.0.0.1/24", - "10.0.2.255": "100.0.0.255", - "10.0.2.3": "100.0.0.2/24", - "91.189.89.199": "96.79.58.28", - "5.4.0.102": "73.90.65.47", - "5.4.0.100": "69.14.29.26", - "5.4.0.26": "58.82.14.73", - "224.0.0.1": "34.30.38.86", - "91.189.89.198": "59.29.50.62", - "3.192.30.10": "14.98.98.25", - "5.4.0.99": "74.19.96.35", - "10.0.2.0/28": "101.0.0.1/28", - "10.0.2.0/30": "102.0.0.1/30", - "192.168.200.1": "20.50.36.98", - "192.168.200.4": "28.58.14.33", - "192.168.200.9": "97.65.46.86", - "192.168.201.0/24": "103.0.0.1/24", - "192.168.201.0/25": "104.0.0.1/25", - "224.0.0.251": "67.40.56.81", - "239.255.255.250": "26.23.39.82", - "123.45.67.89": "20.91.68.26", - "192.168.0.133": "79.76.79.11" - }, - "mac_map": { - "52:54:00:12:34:56": "53:4f:53:79:76:69", <--- - "33:33:00:00:00:16": "53:4f:53:73:7a:0e", - "33:33:00:00:00:02": "53:4f:53:5e:53:b8", - "33:33:ff:12:34:56": "53:4f:53:eb:10:e9", - "33:33:00:00:00:01": "53:4f:53:dc:de:c4", - "01:00:5e:00:00:01": "53:4f:53:4d:05:93", - "01:80:c2:00:00:00": "53:4f:53:b6:04:55", - "01:80:c2:00:00:03": "53:4f:53:34:cb:ef", - "01:80:c2:00:00:0e": "53:4f:53:f1:77:bc", - "52:55:0a:00:02:02": "53:4f:53:08:59:72", - "52:55:0a:00:02:03": "53:4f:53:97:23:da", - "52:56:00:00:00:02": "53:4f:53:80:ba:7d" - }, - "keyword_map": {}, - "username_map": {} + "hostname_map": { + "autopkgtest": "host0" + }, + "ip_map": { + "10.0.2.0/24": "100.0.0.0/24", + "10.0.2.15/24": "100.0.0.1/24", + "10.0.2.255": "100.0.0.255", + "10.0.2.3": "100.0.0.2/24", + "91.189.89.199": "96.79.58.28", + "5.4.0.102": "73.90.65.47", + "5.4.0.100": "69.14.29.26", + "5.4.0.26": "58.82.14.73", + "224.0.0.1": "34.30.38.86", + "91.189.89.198": "59.29.50.62", + "3.192.30.10": "14.98.98.25", + "5.4.0.99": "74.19.96.35", + "10.0.2.0/28": "101.0.0.1/28", + "10.0.2.0/30": "102.0.0.1/30", + "192.168.200.1": "20.50.36.98", + "192.168.200.4": "28.58.14.33", + "192.168.200.9": "97.65.46.86", + "192.168.201.0/24": "103.0.0.1/24", + "192.168.201.0/25": "104.0.0.1/25", + "224.0.0.251": "67.40.56.81", + "239.255.255.250": "26.23.39.82", + "123.45.67.89": "20.91.68.26", + "192.168.0.133": "79.76.79.11" + }, + "mac_map": { + "52:54:00:12:34:56": "53:4f:53:79:76:69", <--- + "33:33:00:00:00:16": "53:4f:53:73:7a:0e", + "33:33:00:00:00:02": "53:4f:53:5e:53:b8", + "33:33:ff:12:34:56": "53:4f:53:eb:10:e9", + "33:33:00:00:00:01": "53:4f:53:dc:de:c4", + "01:00:5e:00:00:01": "53:4f:53:4d:05:93", + "01:80:c2:00:00:00": "53:4f:53:b6:04:55", + "01:80:c2:00:00:03": "53:4f:53:34:cb:ef", + "01:80:c2:00:00:0e": "53:4f:53:f1:77:bc", + "52:55:0a:00:02:02": "53:4f:53:08:59:72", + "52:55:0a:00:02:03": "53:4f:53:97:23:da", + "52:56:00:00:00:02": "53:4f:53:80:ba:7d" + }, + "keyword_map": {}, + "username_map": {} } But somehow the file itself is collected without changing the ip address from original ("52:54:00:12:34:56") to the obfuscated one ("53:4f:53:79:76:69") - ---- Tested with upstream sos: root@autopkgtest:/home/nikhil/sos# git remote -v origin https://github.com/sosreport/sos/ (fetch) origin https://github.com/sosreport/sos/ (push) root@autopkgtest:/home/nikhil/sos# ls AUTHORS bin docs LICENSE man MANIFEST.in plugins_overview.py po README.md requirements.txt setup.py snap sos sos.conf sos.spec tests tmpfilesd-sos.conf - root@autopkgtest:/home/nikhil/sos# - - - root@autopkgtest:/home/nikhil/sos/bin# cat /etc/netplan/50-cloud-init.yaml - # This file is generated from information provided by the datasource. Changes - # to it will not persist across an instance reboot. To disable cloud-init's - # network configuration capabilities, write a file - # /etc/cloud/cloud.cfg.d/99-disable-network-config.cfg with the following: - # network: {config: disabled} - network: - ethernets: - ens3: - dhcp4: true - match: - macaddress: '52:54:00:12:34:56' - set-name: ens3 - version: 2 - + root@autopkgtest:/home/nikhil/sos# + + root@autopkgtest:/home/nikhil/sos/bin# cat /etc/netplan/50-cloud-init.yaml + # This file is generated from information provided by the datasource. Changes + # to it will not persist across an instance reboot. To disable cloud-init's + # network configuration capabilities, write a file + # /etc/cloud/cloud.cfg.d/99-disable-network-config.cfg with the following: + # network: {config: disabled} + network: + ethernets: + ens3: + dhcp4: true + match: + macaddress: '52:54:00:12:34:56' + set-name: ens3 + version: 2 Note it contains the mac addr 52:54:00:12:34:56 Now we will run the sos report, (upstream version) oot@autopkgtest:/home/nikhil/sos/bin# ls sos sos-collector sosreport root@autopkgtest:/home/nikhil/sos/bin# pwd /home/nikhil/sos/bin root@autopkgtest:/home/nikhil/sos/bin# ./sos report --mask sosreport (version 4.3) This command will collect system configuration and diagnostic information from this Ubuntu system. For more information on Canonical visit: - Community Website : https://www.ubuntu.com/ - Commercial Support : https://www.canonical.com + Community Website : https://www.ubuntu.com/ + Commercial Support : https://www.canonical.com The generated archive may contain data considered sensitive and its content should be reviewed by the originating organization before being passed to any third party. No changes will be made to system configuration. - Press ENTER to continue, or CTRL-C to quit. Optionally, please enter the case id that you are generating this report for []: - Setting up archive ... - Setting up plugins ... + Setting up archive ... + Setting up plugins ... [plugin:firewall_tables] skipped command 'nft list ruleset': required kmods missing: nf_tables, nfnetlink. Use '--allow-system-changes' to enable collection. - [plugin:firewall_tables] skipped command 'iptables -vnxL': required kmods missing: nf_tables, iptable_filter. - [plugin:firewall_tables] skipped command 'ip6tables -vnxL': required kmods missing: ip6table_filter, nf_tables. + [plugin:firewall_tables] skipped command 'iptables -vnxL': required kmods missing: nf_tables, iptable_filter. + [plugin:firewall_tables] skipped command 'ip6tables -vnxL': required kmods missing: ip6table_filter, nf_tables. [plugin:networking] skipped command 'ip -s macsec show': required kmods missing: macsec. Use '--allow-system-changes' to enable collection. [plugin:networking] skipped command 'ss -peaonmi': required kmods missing: unix_diag, udp_diag, netlink_diag, af_packet_diag, inet_diag, xsk_diag, tcp_diag. Use '--allow-system-changes' to enable collection. - [plugin:ufw] skipped command 'ufw status numbered': required kmods missing: bpfilter, iptable_filter. - [plugin:ufw] skipped command 'ufw app list': required kmods missing: bpfilter, iptable_filter. - [plugin:wireless] skipped command 'iw list': required kmods missing: cfg80211. - [plugin:wireless] skipped command 'iw dev': required kmods missing: cfg80211. - [plugin:wireless] skipped command 'iwconfig': required kmods missing: cfg80211. - [plugin:wireless] skipped command 'iwlist scanning': required kmods missing: cfg80211. - Running plugins. Please wait ... - - Finishing plugins [Running: systemd] - Finished running plugins + [plugin:ufw] skipped command 'ufw status numbered': required kmods missing: bpfilter, iptable_filter. + [plugin:ufw] skipped command 'ufw app list': required kmods missing: bpfilter, iptable_filter. + [plugin:wireless] skipped command 'iw list': required kmods missing: cfg80211. + [plugin:wireless] skipped command 'iw dev': required kmods missing: cfg80211. + [plugin:wireless] skipped command 'iwconfig': required kmods missing: cfg80211. + [plugin:wireless] skipped command 'iwlist scanning': required kmods missing: cfg80211. + Running plugins. Please wait ... + + Finishing plugins [Running: systemd] + Finished running plugins Found 1 total reports to obfuscate, processing up to 4 concurrently sosreport-autopkgtest-2022-03-02-abhwscl : Beginning obfuscation... sosreport-autopkgtest-2022-03-02-abhwscl : Obfuscation completed [removed 16 unprocessable files] Successfully obfuscated 1 report(s) Creating compressed archive... A mapping of obfuscated elements is available at - /tmp/sosreport-host0-2022-03-02-abhwscl-private_map + /tmp/sosreport-host0-2022-03-02-abhwscl-private_map Your sosreport has been generated and saved in: - /tmp/sosreport-host0-2022-03-02-abhwscl-obfuscated.tar.xz - - Size 2.27MiB - Owner root - sha256 e9d19933cfed512a59790edf65f70a0139f8da162f406153c298bb093bfbd939 + /tmp/sosreport-host0-2022-03-02-abhwscl-obfuscated.tar.xz + + Size 2.27MiB + Owner root + sha256 e9d19933cfed512a59790edf65f70a0139f8da162f406153c298bb093bfbd939 Please send this file to your support representative. - - Lets open the file and see if mac address in that file is left unobfuscated, - - - root@autopkgtest:/tmp# cat sosreport-host0-2022-03-02-abhwscl/etc/netplan/50-cloud-init.yaml - # This file is generated from information provided by the datasource. Changes - # to it will not persist across an instance reboot. To disable cloud-init's - # network configuration capabilities, write a file - # /etc/cloud/cloud.cfg.d/99-disable-network-config.cfg with the following: - # network: {config: disabled} - network: - ethernets: - ens3: - dhcp4: true - match: - macaddress: '52:54:00:12:34:56' - set-name: ens3 - version: 2 - root@autopkgtest:/tmp# + Lets open the file and see if mac address in that file is left + unobfuscated, + + root@autopkgtest:/tmp# cat sosreport-host0-2022-03-02-abhwscl/etc/netplan/50-cloud-init.yaml + # This file is generated from information provided by the datasource. Changes + # to it will not persist across an instance reboot. To disable cloud-init's + # network configuration capabilities, write a file + # /etc/cloud/cloud.cfg.d/99-disable-network-config.cfg with the following: + # network: {config: disabled} + network: + ethernets: + ens3: + dhcp4: true + match: + macaddress: '52:54:00:12:34:56' + set-name: ens3 + version: 2 + root@autopkgtest:/tmp# Note, root@autopkgtest:/tmp# ls -lrt total 9448 drwx------ 3 root root 4096 Mar 2 15:48 systemd-private-e709306472c1435993a0b8d1f15e9dd3-systemd-resolved.service-7kMEUf drwx------ 3 root root 4096 Mar 2 15:48 systemd-private-e709306472c1435993a0b8d1f15e9dd3-systemd-timesyncd.service-FqCM6e drwx------ 3 root root 4096 Mar 2 15:48 systemd-private-e709306472c1435993a0b8d1f15e9dd3-systemd-logind.service-xFJpBh drwx------ 3 root root 4096 Mar 2 15:48 systemd-private-e709306472c1435993a0b8d1f15e9dd3-ModemManager.service-x5UZXh -rwxr-xr-x 1 root root 691 Mar 2 15:48 eofcat -rwxr-xr-x 1 root root 285 Mar 2 15:48 autopkgtest-reboot -rwxr-xr-x 1 root root 269 Mar 2 15:48 autopkgtest-reboot-prepare drwxrwxrwt 5 root root 4096 Mar 2 15:48 autopkgtest.RixDKr drwx------ 10 root root 4096 Mar 2 15:48 sosreport-autopkgtest-2022-03-02-zwngejm drwx------ 3 root root 4096 Mar 2 15:48 systemd-private-e709306472c1435993a0b8d1f15e9dd3-fwupd.service-Zasqxf drwx------ 3 root root 4096 Mar 2 15:48 systemd-private-e709306472c1435993a0b8d1f15e9dd3-upower.service-qb69Wg -rw------- 1 root root 1645 Mar 2 15:49 sosreport-host0-2022-03-02-bwcteqj-private_map -rw------- 1 root root 2389116 Mar 2 15:50 sosreport-host0-2022-03-02-bwcteqj-obfuscated.tar.xz drwxr-xr-x 12 root root 4096 Mar 2 15:50 sosreport_test drwx------ 12 root root 4096 Mar 2 15:55 sosreport-autopkgtest-2022-03-02-nwzytde -rw------- 1 root root 2409380 Mar 2 15:55 sosreport-autopkgtest-2022-03-02-nwzytde.tar.xz -rw-r--r-- 1 root root 65 Mar 2 15:55 sosreport-autopkgtest-2022-03-02-nwzytde.tar.xz.sha256 -rw------- 1 root root 2411848 Mar 2 15:58 sosreport-autopkgtest-2022-03-02-hkqkbak.tar.xz -rw-r--r-- 1 root root 65 Mar 2 15:58 sosreport-autopkgtest-2022-03-02-hkqkbak.tar.xz.sha256 drwx------ 12 root root 4096 Mar 2 15:58 sosreport-host0-2022-03-02-abhwscl -rw------- 1 root root 1645 Mar 2 15:59 sosreport-host0-2022-03-02-abhwscl-private_map <--- -rw------- 1 root root 2378324 Mar 2 15:59 sosreport-host0-2022-03-02-abhwscl-obfuscated.tar.xz -rw------- 1 root root 65 Mar 2 15:59 sosreport-host0-2022-03-02-abhwscl-obfuscated.tar.xz.sha256 - - root@autopkgtest:/tmp# cat sosreport-host0-2022-03-02-abhwscl-private_map + root@autopkgtest:/tmp# cat sosreport-host0-2022-03-02-abhwscl-private_map { - "hostname_map": { - "autopkgtest": "host0" - }, - "ip_map": { - "10.0.2.0/24": "100.0.0.0/24", - "10.0.2.15/24": "100.0.0.1/24", - "10.0.2.255": "100.0.0.255", - "10.0.2.3": "100.0.0.2/24", - "91.189.89.198": "33.43.50.21", - "5.4.0.102": "80.74.90.96", - "5.4.0.100": "69.87.15.65", - "5.4.0.26": "13.16.68.51", - "224.0.0.1": "92.20.91.63", - "91.189.94.4": "42.38.68.46", - "3.192.30.10": "93.87.22.28", - "5.4.0.99": "37.44.72.50", - "10.0.2.0/28": "101.0.0.1/28", - "10.0.2.0/30": "102.0.0.1/30", - "192.168.200.1": "37.72.13.85", - "192.168.200.4": "19.35.86.99", - "192.168.200.9": "39.80.73.13", - "192.168.201.0/24": "103.0.0.1/24", - "192.168.201.0/25": "104.0.0.1/25", - "224.0.0.251": "19.45.84.66", - "239.255.255.250": "93.52.70.42", - "123.45.67.89": "92.20.45.84", - "192.168.0.133": "29.81.60.51" - }, - "mac_map": { - "52:54:00:12:34:56": "53:4f:53:45:22:61", <----- never made it into the file collected - "33:33:00:00:00:16": "53:4f:53:63:ca:e1", - "33:33:00:00:00:02": "53:4f:53:46:bc:12", - "33:33:ff:12:34:56": "53:4f:53:9a:2e:fa", - "33:33:00:00:00:01": "53:4f:53:e6:29:f1", - "01:00:5e:00:00:01": "53:4f:53:67:6f:e7", - "01:80:c2:00:00:00": "53:4f:53:20:c4:71", - "01:80:c2:00:00:03": "53:4f:53:fb:50:f9", - "01:80:c2:00:00:0e": "53:4f:53:2d:57:8a", - "52:55:0a:00:02:03": "53:4f:53:50:2b:72", - "52:55:0a:00:02:02": "53:4f:53:64:ec:34", - "52:56:00:00:00:02": "53:4f:53:3e:92:21" - }, - "keyword_map": {}, - "username_map": {} - + "hostname_map": { + "autopkgtest": "host0" + }, + "ip_map": { + "10.0.2.0/24": "100.0.0.0/24", + "10.0.2.15/24": "100.0.0.1/24", + "10.0.2.255": "100.0.0.255", + "10.0.2.3": "100.0.0.2/24", + "91.189.89.198": "33.43.50.21", + "5.4.0.102": "80.74.90.96", + "5.4.0.100": "69.87.15.65", + "5.4.0.26": "13.16.68.51", + "224.0.0.1": "92.20.91.63", + "91.189.94.4": "42.38.68.46", + "3.192.30.10": "93.87.22.28", + "5.4.0.99": "37.44.72.50", + "10.0.2.0/28": "101.0.0.1/28", + "10.0.2.0/30": "102.0.0.1/30", + "192.168.200.1": "37.72.13.85", + "192.168.200.4": "19.35.86.99", + "192.168.200.9": "39.80.73.13", + "192.168.201.0/24": "103.0.0.1/24", + "192.168.201.0/25": "104.0.0.1/25", + "224.0.0.251": "19.45.84.66", + "239.255.255.250": "93.52.70.42", + "123.45.67.89": "92.20.45.84", + "192.168.0.133": "29.81.60.51" + }, + "mac_map": { + "52:54:00:12:34:56": "53:4f:53:45:22:61", <----- never made it into the file collected + "33:33:00:00:00:16": "53:4f:53:63:ca:e1", + "33:33:00:00:00:02": "53:4f:53:46:bc:12", + "33:33:ff:12:34:56": "53:4f:53:9a:2e:fa", + "33:33:00:00:00:01": "53:4f:53:e6:29:f1", + "01:00:5e:00:00:01": "53:4f:53:67:6f:e7", + "01:80:c2:00:00:00": "53:4f:53:20:c4:71", + "01:80:c2:00:00:03": "53:4f:53:fb:50:f9", + "01:80:c2:00:00:0e": "53:4f:53:2d:57:8a", + "52:55:0a:00:02:03": "53:4f:53:50:2b:72", + "52:55:0a:00:02:02": "53:4f:53:64:ec:34", + "52:56:00:00:00:02": "53:4f:53:3e:92:21" + }, + "keyword_map": {}, + "username_map": {} ------------- - - Upstream issue filed - https://github.com/sosreport/sos/issues/2873
-- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1962733 Title: [sru] sosreport does not obfuscate a mac address when --mask is used To manage notifications about this bug go to: https://bugs.launchpad.net/sosreport/+bug/1962733/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
