On 13/12/19 12:56 am, Sebastian Huber wrote:
> I would like to write new test cases and programs using the RTEMS Test
> Framework:
>
> https://docs.rtems.org/branches/master/eng/test-framework.html
I have no specific issue with the test framework as a way forward however I am
not yet comfortable about the integration of these new types of tests into the
eco-system plus what happens to the existing tests?
Currently the `rtems-test` command is the only supported way we have to run the
tests on simulators and hardware and I am busy promoting and encouraging users
in our community to integrate to this tool and to use it to validate their
systems. I feel it is important we maintain this command and what it does.
I am concerned I am expected to integrate these new tests into the `rtems-test`
command. It would be nice if you could explain what needs to be done in the
eco-system to integrate these type of tests and how you think this work will be
done?
> There will be two sets of tests. One set are unit tests. The other set are
> validation tests. The validation tests check that the implementation meets its
> requirements. The unit tests don't need a link to requirements. They can be
> used
> to test internal APIs like the chains, red-black trees, etc. independent of
> requirements.
>
> I suggest to add "testsuites/unit" for unit tests and "testsuites/validation"
> for validation tests.
I am not sure about these names but I cannot think of anything better. I see
below you will provide another layer of directories in these paths. That is
good.
> For validation tests we need a link from test cases to requirements. This
> needs
> to be discussed separately.
Sure. It would be good if the eco-system aspects are discussed as well.
> With the RTEMS Test Framework test cases can be written independently from the
> test runner program defining the test suite. This could be used for example to
> run a test case in different contexts. Another option is to group test cases
> depending on the available RAM in the target system, e.g. bigger systems can
> execute more test cases at once. This gives raise to a different organization
> compared to the existing tests programs.
Can you please explain how rtems-test is to handle integrated tests?
How do you manage possible cross-talk between tests?
I think the approach is sound as the time to load a test is often much more than
the time the test runs.
> I suggest to add topic directories to "testsuites/unit" and
> "testsuites/validation", e.g. "testsuites/unit/chains". Test case files use a
> naming scheme of "tc-${topic}-${name}.c" and are placed in the corresponding
> topic directory. Test suite files use a naming scheme of "ts-${name}.c" and
> are
> placed in "testsuites/unit" or "testsuites/validation".
Is ${name} a name of a C file selected by the developer? Can there be more than
one ${name}s ?
Chris
_______________________________________________
devel mailing list
devel@rtems.org
http://lists.rtems.org/mailman/listinfo/devel
