I waited to reply because Chris got most of my questions in his first response. :)
On Tue, Dec 14, 2021 at 11:46 PM Chris Johns <chr...@rtems.org> wrote: > > On 14/12/21 6:24 pm, Sebastian Huber wrote: > > Hello Chris, > > > > On 13/12/2021 22:01, Chris Johns wrote: > >> On 14/12/21 1:53 am, Sebastian Huber wrote: > >>> Hello, > >>> > >>> the ESA activity to pre-qualify parts of RTEMS according to ECSS > >>> requirements is > >>> nearly complete. There is a short presentation available here: > >>> > >>> https://indico.esa.int/event/374/timetable/ > >> > >> Was the change in memory usage for 4.8 of 23812 bytes to 68896 explained? > > > > The foot print increase has mainly five reasons: > > > > * General GCC code generation > > > > * Chip errata workarounds done by GCC > > > > * More features used from RTEMS (for example uniprocessor synchronization > > done > > via task priorities vs. mutex synchronization) > > > > * SMP support of RTEMS > > > > * New RTEMS features such as transitive priority inheritance > > > > Thanks. If the 4.8 size was based on the Edisoft version, it was also stripped of functionality even within methods in RTEMS itself. Combine that with uniprocessor vs SMP and more robust algorithms in the current RTEMS and it creeps up. It would be interesting to see a straight uniprocessor comparison but even that would not have feature parity. > > >>> We finished the specification of the pre-qualified RTEMS feature set. The > >>> specification is available in an RTEMS Project repository: > >>> > >>> https://git.rtems.org/rtems-central/tree/spec > >> > >> I had a quick look. Is there a more user friendly view of this data? > >> > >> I think the term "specification" is a little bit misleading because the > >> data > >> files are not easily read by a person. I understand this is the > >> specification > >> data set however it is not what I am traditionally use to seeing. > > > > You can use the "./specview.py" script to get views of the specification. > > For > > example, this command displays the transition map for the > > rtems_signal_send() > > directive: > > Is specview.py part of rtems.git? > > If not part of rtems.git how much data is there for all the output? That is it > is generated and held in the repo with the tests? > > > > > ./specview.py --filter=action-table /rtems/signal/req/send > > .. table:: > > :class: longtable > > > > ===== ========== ===== ======= ======= ======== ====== ====== > > ============= > > ========= > > Entry Descriptor Task Set Handler ASR Nested Status Handler > > Recursive > > ===== ========== ===== ======= ======= ======== ====== ====== > > ============= > > ========= > > 0 0 NoObj Zero Invalid Enabled Yes InvNum NoCall > > No > > 1 0 NoObj Zero Invalid Enabled No InvNum NoCall > > No > > 2 0 NoObj Zero Invalid Disabled Yes InvNum NoCall > > No > > 3 0 NoObj Zero Invalid Disabled No InvNum NoCall > > No > > 4 0 NoObj Zero Valid Enabled Yes InvNum NoCall > > No > > 5 0 NoObj Zero Valid Enabled No InvNum NoCall > > No > > 6 0 NoObj Zero Valid Disabled Yes InvNum NoCall > > No > > 7 0 NoObj Zero Valid Disabled No InvNum NoCall > > No > > 8 1 NoObj NonZero Invalid Enabled Yes InvId NoCall > > No > > 9 1 NoObj NonZero Invalid Enabled No InvId NoCall > > No > > 10 1 NoObj NonZero Invalid Disabled Yes InvId NoCall > > No > > 11 1 NoObj NonZero Invalid Disabled No InvId NoCall > > No > > 12 1 NoObj NonZero Valid Enabled Yes InvId NoCall > > No > > 13 1 NoObj NonZero Valid Enabled No InvId NoCall > > No > > 14 1 NoObj NonZero Valid Disabled Yes InvId NoCall > > No > > 15 1 NoObj NonZero Valid Disabled No InvId NoCall > > No > > 16 0 Self Zero Invalid Enabled Yes InvNum NoCall > > No > > 17 0 Self Zero Invalid Enabled No InvNum NoCall > > No > > 18 0 Self Zero Invalid Disabled Yes InvNum NoCall > > No > > 19 0 Self Zero Invalid Disabled No InvNum NoCall > > No > > 20 0 Self Zero Valid Enabled Yes InvNum NoCall > > No > > 21 0 Self Zero Valid Enabled No InvNum NoCall > > No > > 22 0 Self Zero Valid Disabled Yes InvNum NoCall > > No > > 23 0 Self Zero Valid Disabled No InvNum NoCall > > No > > 24 2 Self NonZero Invalid Enabled Yes NotDef NoCall > > No > > 25 2 Self NonZero Invalid Enabled No NotDef NoCall > > No > > 26 2 Self NonZero Invalid Disabled Yes NotDef NoCall > > No > > 27 2 Self NonZero Invalid Disabled No NotDef NoCall > > No > > 28 6 Self NonZero Valid Enabled Yes Ok DuringSend > > Yes > > 29 4 Self NonZero Valid Enabled No Ok DuringSend > > No > > 30 3 Self NonZero Valid Disabled Yes Ok AfterEnable > > No > > 31 3 Self NonZero Valid Disabled No Ok AfterEnable > > No > > 32 0 Other Zero Invalid Enabled Yes InvNum NoCall > > No > > 33 0 Other Zero Invalid Enabled No InvNum NoCall > > No > > 34 0 Other Zero Invalid Disabled Yes InvNum NoCall > > No > > 35 0 Other Zero Invalid Disabled No InvNum NoCall > > No > > 36 0 Other Zero Valid Enabled Yes InvNum NoCall > > No > > 37 0 Other Zero Valid Enabled No InvNum NoCall > > No > > 38 0 Other Zero Valid Disabled Yes InvNum NoCall > > No > > 39 0 Other Zero Valid Disabled No InvNum NoCall > > No > > 40 2 Other NonZero Invalid Enabled Yes NotDef NoCall > > No > > 41 2 Other NonZero Invalid Enabled No NotDef NoCall > > No > > 42 2 Other NonZero Invalid Disabled Yes NotDef NoCall > > No > > 43 2 Other NonZero Invalid Disabled No NotDef NoCall > > No > > 44 7 Other NonZero Valid Enabled Yes Ok AfterDispatch > > Yes > > 45 5 Other NonZero Valid Enabled No Ok AfterDispatch > > No > > 46 3 Other NonZero Valid Disabled Yes Ok AfterEnable > > No > > 47 3 Other NonZero Valid Disabled No Ok AfterEnable > > No > > ===== ========== ===== ======= ======= ======== ====== ====== > > ============= > > ========= > > Is the > > > > Here the same information in a different view, for each post-condition set > > the > > pre-condition sets are displayed: > > > > ./specview.py --filter=action-list /rtems/signal/req/send > > > > Status = Ok, Handler = DuringSend, Recursive = Yes > > > > * Task = Self, Set = NonZero, Handler = Valid, ASR = Enabled, Nested = > > Yes > > > > Status = Ok, Handler = DuringSend, Recursive = No > > > > * Task = Self, Set = NonZero, Handler = Valid, ASR = Enabled, Nested = > > No > > > > Status = Ok, Handler = AfterDispatch, Recursive = Yes > > > > * Task = Other, Set = NonZero, Handler = Valid, ASR = Enabled, Nested = > > Yes > > > > Status = Ok, Handler = AfterDispatch, Recursive = No > > > > * Task = Other, Set = NonZero, Handler = Valid, ASR = Enabled, Nested = > > No > > > > Status = Ok, Handler = AfterEnable, Recursive = No > > > > * Task = { Self, Other }, Set = NonZero, Handler = Valid, ASR = > > Disabled, > > Nested = { Yes, No } > > > > Status = InvId, Handler = NoCall, Recursive = No > > > > * Task = NoObj, Set = NonZero, Handler = { Invalid, Valid }, ASR = { > > Enabled, Disabled }, Nested = { Yes, No } > > > > Status = NotDef, Handler = NoCall, Recursive = No > > > > * Task = { Self, Other }, Set = NonZero, Handler = Invalid, ASR = { > > Enabled, > > Disabled }, Nested = { Yes, No } > > > > Status = InvNum, Handler = NoCall, Recursive = No > > > > * Task = { NoObj, Self, Other }, Set = Zero, Handler = { Invalid, Valid > > }, > > ASR = { Enabled, Disabled }, Nested = { Yes, No } > > > > > >> > >>> The validation tests are generated from the specification using the > >>> "./spec2modules.py" script and end up in the RTEMS sources of a Git > >>> submodule. I > >>> think the specification and the generation tool is now sufficiently > >>> stable so > >>> that the validation test code can be integrated in the RTEMS master > >>> branch. The > >>> patch set is too big for the mailing list, so you can review it here: > >>> > >>> https://git.rtems.org/sebh/rtems.git/log/?h=validation > >> > >> The link failed. > > > > Yes, viewing my personal repository no longer works. I am not sure if this > > is a > > temporary issue. This is why I added the github link. > > It seems to have been temporary. It is back again. > > > > >> > >>> https://github.com/sebhub/rtems/tree/validation > >> > >> The header in a test says the regeneration instructions are in the > >> engineering > >> manual but I could not quickly find them? > > > > https://docs.rtems.org/branches/master/eng/req/howto.html#generate-content-after-changes > > > > > > In an earlier version of the header, we had a link which you didn't like: > > If I need to look at the formatting rules the heading "Software Development > Management" is easy to see and then a click on "Coding Standards" gives me > what > I am looking for. > > To generate these headers I click on "Software Requirements Engineering" and > then do I just guess until I find it in the "How To" section? I am actually > asking this be sorted out so it is not left hanging and we are not left > guessing > what to do. If it can be rearrange into something meaningful it would help. :) Have you had anyone not directly involved in the generation process try this from the instructions? That always seems to be when the holes show up. > > commit a6689fb147649a29ff5533cec8a53635e2c95ec6 > > Author: Sebastian Huber <sebastian.hu...@embedded-brains.de> > > Date: Fri Jan 22 16:01:46 2021 +0100 > > > > Improve file header comment in generated files > > > > diff --git a/cpukit/include/rtems/rtems/types.h > > b/cpukit/include/rtems/rtems/types.h > > index 32b45113c8..a058eedea1 100644 > > --- a/cpukit/include/rtems/rtems/types.h > > +++ b/cpukit/include/rtems/rtems/types.h > > @@ -38,11 +38,15 @@ > > * worded better please post a report or patch to an RTEMS mailing list > > * or raise a bug report: > > * > > - * https://docs.rtems.org/branches/master/user/support/bugs.html > > + * https://www.rtems.org/bugs.html > > * > > - * For information on updating and regenerating please refer to: > > + * For information on updating and regenerating please refer to the How-To > > + * section in the Software Requirements Engineering chapter of the > > + * RTEMS Software Engineering manual. The manual is provided as a part of > > + * a release. For development sources please refer to the online > > + * documentation at: > > * > > - * https://docs.rtems.org/branches/master/eng/req/howto.html > > + * https://docs.rtems.org > > */ > > > > /* Generated from spec:/rtems/type/if/header */ > > > > I am fine with the header, as we know links to the documentation is really > hard > long term. Early on, we discussed some annotation maybe like SPDX to note that a file had higher rigor with requirements and tests. Is this Generated note implying this? Is it sufficient to note that if you touch this, we would really like you to address the full chain on this? > >>> The patch set is organized so that general support code for the > >>> validation tests > >>> is added first and then there are commits for each pre-qualified Classic > >>> API > >>> Manager or subsystem. > >>> > >>> I did build all BSPs with the patch set. The validation tests use only > >>> statically allocated resources. > >> > >> Are the validation tests compatible with rtems-test? > > > > Yes. > > Great. > > >> How many test executables does this add to the testsuite? > > > > If RTEMS_SMP is disabled, then there are 19 new executables in the patch > > set. If > > RTEMS_SMP is enable, then there are 28 new executables. > > Nice. > > >> What hardware have the validation tests been run on? Any tier 1 archs? > > > > I tested with the sparc/leon3 BSPs and the arm/realview_pbx_a9_qemu. > > Is the leon3 tested on hardware or simulation? > > > You need a > > full implementation of the new Interrupt Manager directives and a working > > Cache > > Manager implementation. > > Is this documented? > > I am sorry I do not know the list of archs and bsps that support the new > interrupt manager directives. Maybe it would be good to list them? Yes. And since you don't know the bsps which don't have the support, what will the tests do? > > I noticed an issue with the thread restart on aarch64/a53_lp64_qemu. I passed this along to Kinsey but it would be helpful if you could start a thread/ticket about this so he knows what to look at. > > > > On powerpc/psim there is an issue in one test case, due to: > > > > #define CPU_ALL_TASKS_ARE_FP CPU_HARDWARE_FP > > Sorry, I am not following what the issue is? Does this effect all PPC BSPS? > > > Another issue is that the tm27 interrupt must be independent of the clock > > driver > > interrupt. This is not the case for powerpc/psim. This is a change in the design from the original assumptions over 30 years ago. It will break more than psim. Many boards do not have a second source for a counter/timer and it was assumed that the clock hardware could be used since the tmtests do not configure a clock driver. What exactly broke this? > > > > There is definitely some work left to cover all edge cases. Some tests are > > quite > > complicated. > > Sure. I would like to understand the effects this has? > > >> Is there anything that interprets the new test output format? It looks > >> like lots > >> of great info but a little difficult to read. > > > > EDISOFT worked on a test report generator, however, it is not yet in a > > reviewable state. > > OK. I think something that handles this data would be good to have. > > >>> Some low memory targets are not able to link all test suites. > >> > >> Are these excluded in the normal way? > > > > Yes: > > > > https://github.com/sebhub/rtems/commit/2feedb9e7805f483e35b7914b9bd7c808e31a8b4#diff-bf7b325198d4133c9e52f49d77447905ecd3c6ac5159d5cd85e7efeffa6da47e > > Nice. > > This looks really good and I like what it provides. It is a fantastic baseline > of the interfaces we have however to be useful long term it needs to generate > regression notices and so that means we need to know and understand the > baselines this brings us now and for which arches. Somewhere in this discussion was mention of coverage results. It was mentioned that branches were covered but I would appreciate that being clarified. Is this at the source level or assembly level? How is this done? Is the procedure documented? The coverage reports from rtems-test/covoar are at the assembly language level and report on branch statements being taken and not taken. This is intended to help us exceed MCDC testing. I know you posted some gcov support patches earlier so am guessing that you used that. Some insight would be great. Thanks. --joel > > Chris > _______________________________________________ > devel mailing list > devel@rtems.org > http://lists.rtems.org/mailman/listinfo/devel _______________________________________________ devel mailing list devel@rtems.org http://lists.rtems.org/mailman/listinfo/devel
