I think this will be possible. I just tried a few bare metal assembly demos on
the Pi A+ and Pi2 and it looks like even without the published CPU datasheet,
they have things figured out.
For example: An assembly SMP NEON fractal demo:
https://github.com/PeterLemon/RaspberryPi/tree/master/SMP/N
Sounds like a good approach.
There is a wealth of code out there, but much of it uses a license that is
incompatible with RTEMS.
If we did not consider the license, I think we could have graphics, USB,
network, etc working pretty quickly.
Good resources here, but not directly useable:
Gra