Hi,
I finally got some time to run some benchmarks. I checked the results[1]
into the "hardening" svn tree, in case other people want to contribute
more stuff.
On Wed, Jan 30, 2008 at 08:46:55PM +0100, Moritz Muehlenhoff wrote:
> Video:
> mplayer with the -benchmark option in conjunction with -nosound and -vo.
mplayer doesn't compile with PIE due to the various ASM routines. (I've
noted this failure mode in the wiki[2] now.) However, with everything
else enabled (including FORTIFY_SOURCE), there was no measurable
difference (it was below the percentage difference between runs):
runtime in seconds
Mplayer Normal Hardened
1 10.87 10.807
2 10.873 10.824
3 10.854 10.963
4 10.809 10.84
5 10.877 10.838
avg 10.8566 10.8544 diff: -0.02%
error 0.19% 1.00%
> HTML rendering:
> Mike Hommey once blogged about benchmarking the ACID test:
> http://web.glandium.org/blog/?cat=17
I followed that to: http://celtickane.com/webdesign/jsspeed2007.php
The differences between runs were too high for me to use, so I skipped
this for now.
> Nexuiz:
> | To run the benchmark: start Nexuiz & open the console (`) issuing:
> | timedemo demos/demo1.dem The results will be stored in:
> | ~/.nexuiz/data/benchmark.log
This one showed a possible difference:
nexuiz Normal Hardened
1 66.68 68.113
2 66.802 66.93
3 66.758 67.03
4 66.728 67.051
5 66.859 67.037
avg 66.7654 67.2322 diff: 0.70%
error 0.14% 1.31%
So, for nexuiz, with all hardening enabled in i386, there was a
less-than-1-percent reduction in speed. Though the error margin for the
hardened runs were still larger than the measured slow-down.
> Not sure about XML benchmarks.
I did parse/render tests with inkscape on i386. Some of that is XML, but
I figured it was heavy CPU, which might be worth something. Note that
inkscape already compiles with all hardening options (excepting PIE),
so the "hardened" time differences are entirely due to PIE. This one
turned out similar to nexuiz, but with less error. Again, less than 1
percent slow-down was seen.
inkscape Normal Hardened
1 48.163 48.503
2 48.227 48.535
3 48.267 48.647
4 48.335 48.431
5 48.199 48.587
avg 48.2382 48.5406 diff: 0.63%
error 0.20% 0.22%
I also ran inkscape and nexuiz tests on x86_64, and there was no
measurable difference. I'm unclear if this was due to the extra
registers, or just that that CPU was much faster and the difference
vanished into the noise.
-Kees
[1] http://svn.debian.org/wsvn/hardening/benchmarks/
[2] http://wiki.debian.org/Hardening
--
Kees Cook @outflux.net
--
To UNSUBSCRIBE, email to [EMAIL PROTECTED]
with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]
