On 04/10/2019 09:20, Chris Johns wrote:
On 4/10/19 4:21 pm, Sebastian Huber wrote:
On 03/10/2019 04:32, Chris Johns wrote:
On 3/10/19 3:30 am, Gedare Bloom wrote:
On Wed, Oct 2, 2019 at 5:12 AM Sebastian Huber
<sebastian.hu...@embedded-brains.de> wrote:
On 30/09/2019 15:14, Sebastian Huber wrote:
Hello,
I would like to work on a new build system prototype. The idea is to use
specification items maintained by Doorstop (YAML files), a Python
configuration script and waf to build RTEMS and the tests. This is
similar to the libbsd build. The difference is that in libbsd the build
data is maintained directly in Python code (libbsd.py).
A key design aspect is how the configuration of RTEMS is handled and maintained.
Amar's solution provides a specific model for managing settings and specifically
BSP options. We need a way to add options for a BSP that lets us collect,
document and validate these settings. We need a way to query and review option
defaults. The solution needs to be able to add and remove BSPs with minimal
impact and this leads to being able to build a BSP that is external to the RTEMS
source tree. There are BSPs that are private or export restricted. Another issue
to consider is how we deprecate, update or alter options.
I suspect we will have an iterating design around the internal design and
implementation complexity and suitable external user work-flows.
Now is a good time to create a wish list for the new build system. I cannot
implement everything, but I can try to make a substantial initial step forward.
I agree.
Doorstop is a tool to manage a directed, acyclic graph of items. Each item has a
set of attributes (key-value pairs). Some attributes are pre-defined by Doorstop
with requirements management in mind. You can also add an arbitrary number of
custom attributes. With this data structure you can tackle a lot of problems.
The items are stored in YAML format files.
Note YAML is not part of the standard Python, it is add-on package. This means
using it would mean we would need to provide support on a number of hosts.
The libyaml package seems to be used in a lot of other packages and can
be installed with pip. If the use of YAML for the build would be a show
stopper for you, it would be good to know this early.
My approach would be to store all the data that defines the build in build item
files. The wscript file reads the build items, configures the build and performs
the build as specified also by command line options, e.g. which BSP, with tests,
etc. The build item files can be also used by other tools.
A BSP is represented by a BSP build item which links to BSP option items, object
items (start.o) and file groups for librtemsbsp.a.
A BSP is just a node in the graph, you can ask for example: if I remove this BSP
item, does it create isolated items? The isolated items can be removed if the
BSP is removed. If you change an option, you can figure out the impact, e.g.
which BSP uses this option.
I had not considered going to level but what you say seems consistent with my
thinking on how configuration should happen. With out the detail it is hard so I
will make a general statement that a change to the build system has to serve the
general RTEMS community first and this means easy to access and manage
configurations generated by RTEMS. I ask RTEMS to configure a BSP and a
configuration file is generated. I can then edit the file to alter specific
settings I need. The is what Amar did.
My approach would be to configure the build via command line options and
then the configuration is read-only for the build. Being able to alter
an existing configuration seems a bit complex to me.
And I do not see Doorstop as a part of the general community :)
You would not need Doorstop to build a BSP, just to maintain a BSP.
Since the build is driven by item files which just reside in a directory, you
can also point the build script to another directory with items defined
externally. The question is if waf is capable to build files outside the top
directory. Building externally defined stuff has a very low priority for me. I
just want to point out that there are no fundamental obstacles with this
approach.
Yes this is an issue and something I would need to discuss with Thomas.
The details of BSP options can be easily managed with specialized attributes,
e.g. type, min, max, default value, etc. We can utilize the full power of
https://docs.python.org/3.6/library/optparse.html
which is a very nice Python module. We may use
https://docs.python.org/3.6/library/optparse.html#optparse-option-callbacks
Sure ...
to provide services we need for this BSP (and others):
https://git.rtems.org/rtems/tree/c/src/lib/libbsp/arm/atsam/configure.ac
The callbacks can be stored in the items files as Python code and imported with
exec(). The callbacks need an ability to install a post-processing handler which
is invoked during the waf configure(), e.g. to generate the linker command file
based on options.
Amar implemented this back in 2015 ....
https://git.rtems.org/amar/waf-old.git/tree/py/config?h=fix
... and it is worth reviewing. It is simple and specific.
Yes, I have this check out. However, I was not able to configure or
build anything due to Python/waf errors.
The process needs to be wrapped so Python programming skill are not needed to
update and add new options or BSPs. Adding complex data structures in config
files can be hard to debug and fix if the provided support is not strong. A user
adding or working on a BSP has to be able to navigate and fix something they
break. The more complex the structuring of the data the more complex the support
needs to be.
Yes, but I think we can require some Python skills from a BSP developer.
--
Sebastian Huber, embedded brains GmbH
Address : Dornierstr. 4, D-82178 Puchheim, Germany
Phone : +49 89 189 47 41-16
Fax : +49 89 189 47 41-09
E-Mail : sebastian.hu...@embedded-brains.de
PGP : Public key available on request.
Diese Nachricht ist keine geschäftliche Mitteilung im Sinne des EHUG.
_______________________________________________
devel mailing list
devel@rtems.org
http://lists.rtems.org/mailman/listinfo/devel
