чт, 18 мая 2017 г. в 23:08, Antonio Diaz Diaz <[email protected]>:
> Maybe the fair thing to do is to compare hardware acceleration with > software acceleration. Here's the relevant link to comparison: https://www.strchr.com/crc32_popcnt Basically, for x86, hardware version is ~10 times faster than Sarwate and 3 times faster than Slice-By-8. And also, 8k of cache is freed that way, which can be another factor of speedup during compression/decompression. > Also, at a size of 4294967263 bits (537 MB), ethernet CRC32 has been > confirmed to feature the longest dataword at HD 3 for all possible CRC32 > polynomials. Twice that of CRC32C. > That's a choice between better protection of smaller files (90Kbit to 2Gbit) and better protection of larger files (2Gbit to 4 Gbit). If someone cares about safety, they can limit member size to 2Gbit, and get much better protection with CRC32c at a price of slightly less compression rate, which is not possible with Ethernet poly (member size of 92kbit is nothing). In my experience with LZMA compression, there are two kinds of errors > not caught by the decoder: > 1) Without error multiplication. The error affects a single byte and is > guaranteed to be detected by both CRCs (ethernet and CRC32C). > 2) With error multiplication. The total number of bits flipped is larger > than 4, which both CRCs can detect about equally well. > That sounds reasonable and calls for CRC64 (which is enabled by default in xz, but is done terribly wrong). Also, CRC64 is much less supported in hardware than CRC32c. I have performed many millions of trial decompressions and never found > any indication that the current CRC is not perfectly adequate. > It would be interesting to see undetected error rate for such trials, and compare it with crc32c. > > It does not work like that: > > $ lzip -t v2.lz > v2.lz: Version 2 member format not supported. Ah, that's a shame and calls for .lz2 file extension. :) Maybe you're right, and it's better for safety-related reasons to just use a homegrown container format with safety-oriented CRC instead of using existing container format like LZIP, since it really has a burden of forward compatibility. And yes, performance reasons do not outweight forward compatibility problems.
_______________________________________________ Lzip-bug mailing list [email protected] https://lists.nongnu.org/mailman/listinfo/lzip-bug
