On 20/11/14 15:23, Daniel Hellstrom wrote:
On 11/20/2014 03:09 PM, Sebastian Huber wrote:
[...]
The image is a boxplot of all the MaxThreadDispatchDisabledTime samples.
I think this is a nice utility and you seem to be able to update the
results quickly too. Thanks. Since it is the test-suite you run how
can we be sure that a test is not badly written, I mean disabling
interrupts for too long time since the test itself is not testing
interrupt latencies etc.? Perhaps the mean is most interesting, or
that some of the max cases should be ignored? Have you analysed the
worst tests to see where the effects comes from?
Is is very easy to use the XMP reports produced by the test suite.
1. Run the test suite with --enable-profiling and gather all the output
in one big file.
2. Filter the big file with the filter-test-xml.py which deletes
everything that doesn't look like XML.
3. Use the XML X Path language to get what you want, for example boxplot.py.
The mean values are not that interesting since the main goal for a
real-time operating system is to provide a reasonable worst case thread
dispatch latency. Attached are the outliers for the two test runs. I
didn't analyse every outlier in detail, but most can be explained with
interrupt processing of the clock tick and some inherent problems of the
SMP support.
--
Sebastian Huber, embedded brains GmbH
Address : Dornierstr. 4, D-82178 Puchheim, Germany
Phone : +49 89 189 47 41-16
Fax : +49 89 189 47 41-09
E-Mail : sebastian.hu...@embedded-brains.de
PGP : Public key available on request.
Diese Nachricht ist keine geschäftliche Mitteilung im Sinne des EHUG.
#!/usr/bin/python
#
# Copyright (c) 2014 embedded brains GmbH. All rights reserved.
#
# The license and distribution terms for this file may be
# found in the file LICENSE in this distribution or at
# http://www.rtems.com/license/LICENSE.
#
import sys
import re
print "<ProfilingCollection>"
while True:
line = sys.stdin.readline()
if not line:
break
m = re.match('^\s*<pause.*>\s*$', line)
if m:
continue
m = re.match('^\s*<assocnamebad.*>\s*$', line)
if m:
continue
m = re.match('^\s*<.*throw this exception.*>\s*$', line)
if m:
continue
m = re.match('^(\s*<.*>)\s*$', line)
if m:
print m.group(1)
print "</ProfilingCollection>"
#!/usr/bin/python
#
# Copyright (c) 2014 embedded brains GmbH. All rights reserved.
#
# The license and distribution terms for this file may be
# found in the file LICENSE in this distribution or at
# http://www.rtems.com/license/LICENSE.
#
import libxml2
from libxml2 import xmlNode
import matplotlib.pyplot as plt
def m(n):
return [int(n.getContent()), n.get_parent().get_parent().prop("name")]
def getData(name):
doc = libxml2.parseFile(name)
ctx = doc.xpathNewContext()
return map(m, ctx.xpathEval('/ProfilingCollection/ProfilingReport/PerCPUProfilingReport/MaxThreadDispatchDisabledTime'))
def getDataUni(name):
return [x[0] for x in getData(name) if x[0] != 0 and x[1].find('SMP') == -1]
def getDataSMP(name):
return [x[0] for x in getData(name) if x[0] != 0 and x[1].find('SMP') != -1]
data = [ \
getDataUni('tests-ngmp-gr-cpci-xc4v-2014-10-24-d412e2f02596e834dbd5aa8b830dc303e91532c3/tests.xml'), \
getDataUni('tests-ngmp-gr-cpci-xc4v-next/tests.xml'), \
getDataSMP('tests-ngmp-gr-cpci-xc4v-2014-10-24-d412e2f02596e834dbd5aa8b830dc303e91532c3/tests.xml'), \
getDataSMP('tests-ngmp-gr-cpci-xc4v-next/tests.xml'), \
]
labels = ( \
'45MHz\nNGMP\nStd\nNon-SMP', \
'45MHz\nNGMP\nOpt\nNon-SMP', \
'45MHz\nNGMP\nStd\nSMP', \
'45MHz\nNGMP\nOpt\nSMP', \
)
cmPerInch = 2.54
plt.figure(figsize = (18.0 / cmPerInch, 18.0 / 1.618 / cmPerInch))
plt.boxplot(data, 0, 'rs');
plt.xticks(range(1, len(labels) + 1), labels, fontsize = 5)
plt.ylabel('max thread dispatch disabled time [ns]')
plt.yscale('log')
plt.savefig('max-thread-dispatch-disabled-times-log.png')
plt.savefig('max-thread-dispatch-disabled-times-log.eps')
.Tests with a maximum thread dispatch disabled time greater than 65921ns
(outliers).
[cols="<,>",frame="topbot",options="header"]
|==============================================================
| Test | Maximum thread dispatch disable time [ns]
| SPTLS 2 | 66133
| SMPLOAD 1 | 67155
| PSXMSGQ 1 | 67377
| SPINTRCRITICAL 8 | 68555
| SMPSCHEDAFFINITY 5 | 69311
| BLOCK 5 | 69866
| FSDOSFSFORMAT 1 | 70644
| CPUUSE | 73400
| SPINTRCRITICAL 15 | 73511
| SMPWAKEAFTER 1 | 73822
| SMPWAKEAFTER 1 | 74133
| SPINTRCRITICAL 7 | 75911
| SP 11 | 77933
| SP 4 | 79066
| SPINTRCRITICAL 6 | 81111
| SPINTRCRITICAL 10 | 81333
| SMPWAKEAFTER 1 | 83977
| SP 69 | 84200
| SMPMIGRATION 2 | 84977
| RTMONUSE | 87177
| SMPLOAD 1 | 88088
| SMPSCHEDULER 1 | 93866
| SMPWAKEAFTER 1 | 95000
| PSXINTRCRITICAL 1 | 99933
| SMPLOAD 1 | 100577
| SMPMRSP 1 | 104266
| PSXMSGQ 4 | 112022
| SPINTRCRITICAL 20 | 114533
| SMPSCHEDULER 3 | 122777
| SMPMRSP 1 | 123800
| SMPMRSP 1 | 129422
| SMPMRSP 1 | 131488
| SMPSCHEDULER 3 | 153222
| SPINTRCRITICAL 17 | 155688
| SMPSCHEDULER 3 | 190044
|==============================================================
.Tests with a maximum thread dispatch disabled time greater than 62187ns
(outliers).
[cols="<,>",frame="topbot",options="header"]
|==============================================================
| Test | Maximum thread dispatch disable time [ns]
| CPUUSE | 62577
| SPTLS 2 | 62644
| SP 4 | 63311
| FSDOSFSFORMAT 1 | 64622
| FSFPATHCONF RFS | 64822
| FSTIME DOSFS | 65444
| FSSYMLINK RFS | 65977
| SPINTRCRITICAL 1 | 66955
| PSXMSGQ 1 | 67111
| SPINTRCRITICAL 5 | 67355
| SPINTRCRITICAL 3 | 67511
| SPINTRCRITICAL 12 | 69311
| SPINTRCRITICAL 11 | 69400
| SPINTRCRITICAL 22 | 69622
| SMPMIGRATION 2 | 70111
| RTMONUSE | 70155
| SPINTRCRITICAL 2 | 71400
| SMPWAKEAFTER 1 | 71755
| SPINTRCRITICAL 4 | 71955
| SPINTRCRITICAL 21 | 72688
| SPINTRCRITICAL 16 | 73311
| SP 11 | 73555
| SPINTRCRITICAL 14 | 74000
| SPINTRCRITICAL 9 | 74933
| SPINTRCRITICAL 8 | 76488
| SMPLOAD 1 | 77355
| SMPMIGRATION 2 | 79000
| BLOCK 5 | 80222
| FLASHDISK 1 | 82644
| SPINTRCRITICAL 7 | 85155
| SMPWAKEAFTER 1 | 86022
| SPINTRCRITICAL 15 | 86355
| SPINTRCRITICAL 6 | 88488
| PSXINTRCRITICAL 1 | 90000
| SPINTRCRITICAL 10 | 91666
| SMPWAKEAFTER 1 | 92244
| SMPLOAD 1 | 97222
| SMPSCHEDULER 1 | 102133
| SMPMRSP 1 | 105244
| PSXMSGQ 4 | 108066
| SMPMRSP 1 | 124555
| SPINTRCRITICAL 20 | 126444
| SMPMRSP 1 | 127244
| SMPSCHEDULER 3 | 131755
| SMPMRSP 1 | 139444
| SPINTRCRITICAL 17 | 152400
| SMPSCHEDULER 3 | 163311
| SMPLOAD 1 | 170422
| SMPWAKEAFTER 1 | 179688
| SMPSCHEDULER 3 | 199400
|==============================================================
_______________________________________________
devel mailing list
devel@rtems.org
http://lists.rtems.org/mailman/listinfo/devel
