[issue38507] Improve the documentation of the nested list initialization
New submission from JUN-WEI SONG : When I used the nested list, I need to initialize the nested list, so I used this expression: >>> nested_list = [[]] * 5 see also: https://stackoverflow.com/questions/12791501/python-initializing-a-list-of-lists So I later learned that such an expression would make the list inside the list have the same reference, which would cause the problem that you modified one element would lead to all elements changed in the nested list. For example: >>> nested_list[0].append(1) >>> nested_list [[1], [1], [1], [1], [1]] Therefore, maybe we could tell users how to initialize the list on the documentation like below: If you need to initialize the nested list, you could follow the below example, also, be aware of the expression like ``[[]] * 5``, this will cause the five lists in the nested list to have the same reference. >>> nested_list = [[] for _ in range(5)] -- assignee: docs@python components: Documentation messages: 354844 nosy: docs@python, krnick priority: normal severity: normal status: open title: Improve the documentation of the nested list initialization type: enhancement versions: Python 3.8 ___ Python tracker <https://bugs.python.org/issue38507> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue38507] Improve the documentation of the nested list initialization
JUN-WEI SONG added the comment: sorry that I did not notice it already documented -- resolution: -> duplicate stage: -> resolved status: open -> closed ___ Python tracker <https://bugs.python.org/issue38507> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue36260] Zip Bomb vulnerability
Change by JUN-WEI SONG : -- keywords: +patch pull_requests: +13288 stage: resolved -> patch review ___ Python tracker <https://bugs.python.org/issue36260> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue36260] Zip Bomb vulnerability
JUN-WEI SONG added the comment: Dear friends, We moved a little bit forward to improve the writing. :) -- ___ Python tracker <https://bugs.python.org/issue36260> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue36988] zipfile: string IndexError on extract
JUN-WEI SONG added the comment: The following output throws error when using unzip -t $ unzip -t file0.zip Output: Archive: file0.zip : mismatching "local" filename (zipfile_extract.pyUT^I), continuing with "central" filename version testing: error: invalid compressed data to inflate At least one error was detected in file0.zip. It looks like the zip file is corrupted. Maybe we could add some detection mechanisms before extract it like unzip, for example, unsupported characters or file corrupted check. -- nosy: +krnick ___ Python tracker <https://bugs.python.org/issue36988> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue36462] CVE-2019-9674 : zip bomb vulnerability in Lib/zipfile.py
New submission from JUN-WEI SONG :
Dear Python Community,
we found a python module vulnerability during these days and we got a CVE
number, CVE-2019-9674 after reported it to cve.mitre.org.
https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2019-9674
The reserved information of CVE-2019-9674 is shown below:
[Description]
Lib/zipfile.py in Python through 3.7.2 allows remote
attackers to cause a denial of service (resource consumption)
via a ZIP bomb.
[Additional Information]
The python zipfile library version 3.2, 3.3, 3.4, 3.5, 3.6,
3.7, 3.8. Allow attackers to cause a denial of service (disk
volume exhaustion) via a ZIP bomb.
We have found python standard library zipfile doesn't have
ZIP bomb detection and protection. If the user uses zipfile
library to unzip a ZIP bomb file, this might cause a denial
of service of the localhost.
[VulnerabilityType Other]
Denial-of-Service
Our proposed solutions:
1.The compression ratio:
Compression ratio = Uncompressed file size / Compressed file size
Since ZIP bomb file has a higher compression ratio (1028) than
normal ZIP file (1 to 3). Therefore, we calculate the compression
ratio and set a threshold for the detection.
2.Nested zip file
There is a high chance that it is zip bomb if it is a nested zip
file.
3.By limiting resources such as CPU, memory, disk usage.
Unsolved issue
However, we have not yet determined the compression ratio. We
temporarily set the compression ratio to 10, and if it exceeds, it
may be a ZIP bomb.
It is likely that detection may misjudge nested compressed files.
For example, under normal circumstances, compressed files are
included in the zip file.
Our solution code：
"""For ratio"""
def _exam_ratio(self, threshold=10):
"""If the ratio exceeds threshold, it may be a ZIP Bomb."""
sum_file_size = sum([data.file_size for data in self.filelist])
sum_compress_size = sum([data.compress_size for data in self.filelist])
ratio = sum_file_size / sum_compress_size
if (ratio > threshold):
raise BadZipFile("Zip Bomb Detected")
"""For Nested zip file"""
if(members.filename.endswith(".zip")):
raise BadZipFile("Nested Zip File Detected")
Thanks!
--
components: Library (Lib)
messages: 339053
nosy: krnick
priority: normal
severity: normal
status: open
title: CVE-2019-9674 : zip bomb vulnerability in Lib/zipfile.py
type: security
versions: Python 3.5, Python 3.6, Python 3.7, Python 3.8
___
Python tracker
<https://bugs.python.org/issue36462>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue36260] Cpython/Lib vulnerability found and request a patch submission
JUN-WEI SONG added the comment: Thank you python community, these two issues are indeed the same problem. I also think that it is good to make a related document to reduce such problems. -- stage: -> resolved status: -> closed ___ Python tracker <https://bugs.python.org/issue36260> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue36462] CVE-2019-9674 : zip bomb vulnerability in Lib/zipfile.py
JUN-WEI SONG added the comment: Thanks to the python community, both of these issues are the same. I also think it's a good thing to make related documentation to reduce this type of problem rather than implementing it on a low-level zipfile module. Perhaps we can customize such a requirement through a pip package. -- ___ Python tracker <https://bugs.python.org/issue36462> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue36260] Cpython/Lib vulnerability found and request a patch submission
JUN-WEI SONG added the comment: Hello Python community, With Christian Heimes’ suggestion, we manipulate appropriate warning to inform users that they may encounter zip bomb issues when using the zipfile module. The warning we would like to add in the zipfile documentation is shown below : https://github.com/python/cpython/blob/3.7/Doc/library/zipfile.rst .. warning:: Never extract files from untrusted sources without prior inspection. It is possible that the file may contain zip bomb issues such as 42.zip. The zip bomb will usually be a small file before decompression, but once it is decompressed, it will exhaust system resources. You can protect your system by limiting system resources, limiting compression ratio (zip bombs are usually quite high), and checking for nested zip files. We are also pleasure to provide a patch to enhance the zipfile module to provide basic information. In zipfile.py https://github.com/python/cpython/blob/master/Lib/zipfile.py Inside the ZipFile class : def filecount(self): """Return total count of files in the archive.""" return len(self.filelist) def total_compressed_size(self): """Return total compressed size in the archive.""" return sum([data.compress_size for data in self.filelist]) def total_uncompressed_size(self): """Return total uncompressed size in the archive.""" return sum([data.file_size for data in self.filelist]) -- resolution: -> remind status: closed -> open ___ Python tracker <https://bugs.python.org/issue36260> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue36260] Cpython/Lib vulnerability found and request a patch submission
JUN-WEI SONG added the comment: Thank you very much for your reply. Based on discussions above, consensuses are improving the zipfile documentation. And we (JUN-WEI SONG & KunYu Chen) would like to work on this. With opinions of Serhiy Storchaka, Christian Heimes and the ideas we have, possible pitfalls are listed below. 1. From file itself: Decompression may fail due to an incorrect password, an incorrect CRC checksum, an incorrect PKZIP format, an unsupported compression method, or an unsupported decryption. 2. File system: Each file system has different limitations such as allowable characters in directory entries, the max length of file name, the max length of path name, the max size of single file, the max number of files, the max number of files in a single directory, etc. Decompression will fail as long as these limitations are exceeded. 3. Operating system: The lack of memory or disk space would lead to decompression failed (see also Zip Bomb). 4. Interrupt: Users should be careful in interrupting the process of decompression, such as control-C or killing the process during decompression, which may result in incomplete decompression of the archive. 5. Different default behaviors: Users should figure out different default extraction behaviors, such as when extracting into the existing tree, it will overwriting an existing file without asking, or when in a case-insensitive file system, it keeps only one file when extracting an archive which contains many files that have the same name but different case. Please let us know if anything’s missing. -- ___ Python tracker <https://bugs.python.org/issue36260> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
