Re: For GSoC: RTEMS release notes new generator

[Chris Johns]

On 1/9/2022 4:43 am, Mahmoud Abumandour wrote:
> Hello all,
> 
> Here, I present the current output of the new release notes generator that 
> will
> be used in the release generation process. 

Thank you.

> Currently, the generator is capable
> is generating markdown and reStructuredText. Both have minor flaws that, in my
> opinion, are somewhat acceptable.
> 
> You can find a sample Markdown PDF for the 4.11.3 milestone
> here: 
> https://drive.google.com/file/d/1S_aKZcJi6c2DX7bCXgt6R9gvVdXgn5Sm/view?usp=sharing
>  
> 

This looks really good. Nice work and thanks for all the effort you have put in.

> And its reStructuredText counterpart
> here: https://drive.google.com/drive/folders/1mntfWm6gnHPUpliwHS_ClJYjjnMOhZZV
> 
> 
> The generator has been tested on generating files for the following dot 
> releases
> 4.11.3, 5.1, 5.2, and 6.1 and it produces decent documents for all of them.

Awesome.

> You can access the code here and test it
> yourself: https://github.com/i3abghany/release-notes-generator
> 
> 
> For integrating the generator in the release workflow, do you recommend a
> certain way to go by in doing this? 

I am not sure but it is a fantastic problem to have :)

> Maybe use the code as a submodule in the
> `rtems-release` repository and call it from the `rtems-release-notes` script 
> in it?

It could which means we have a top level directory for it? Or do we just place
the code in rtems-release?

If there is a general use for the tool it could be placed in rtems-tools.

What you think?

> Also, the generator was initially designed to produce notes for *one* dot
> release at a time, but I found out the current generator goes up from the
> supplied /version /to the specified /revision/. Is this the desired behaviour 
> in
> the new generator as well? I am asking as I took over development in the new
> generator and it did not conform to this behaviour, and I didn't know that it
> was followed in the `rtems-release` scripts.

The intent is the release notes tell about all the changes on a release branch.
If you have .1 and you move to .4 the notes should have all the changes so you
do not need to go and fine then.

It is not a big deal as things stand and we could solve this in a few ways. For
example collecting the dot point release notes into the next version.

I am about to start looking at releases with 5.2 before the end of the year. Are
you about and would you like to help?

Chris
___
devel mailing list
devel@rtems.org
http://lists.rtems.org/mailman/listinfo/devel

Re: [PATCH rtems-lwip] lwip.py: Allow lwIP to build against stale install

[Kinsey Moore]

On 9/10/2022 20:22, Chris Johns wrote:

On 10/9/2022 12:07 pm, Kinsey Moore wrote:

This removes the default BSP include path from environment variables so
that rtems-lwip can build even when there's a stale version with
outdated headers installed in the BSP.


I am not following what this does and what problem you are attempting 
to solve?


I am not comfortable seeing rtems_waf being overridden like this.

Chris

This is approximately the same change that you did for libbsd that 
shuffles the include paths so that lwip builds against local headers 
first and isn't broken by a stale install of rtems-lwip in the BSP 
install directory.


Kinsey


---
  lwip.py | 22 --
  1 file changed, 16 insertions(+), 6 deletions(-)

diff --git a/lwip.py b/lwip.py
index 84eef2c..3d9cb29 100644
--- a/lwip.py
+++ b/lwip.py
@@ -99,6 +99,9 @@ def build(bld):
  drv_incl = []
  arch_lib_path = rtems.arch_bsp_lib_path(bld.env.RTEMS_VERSION,
bld.env.RTEMS_ARCH_BSP)
+    bsp_incl = os.path.relpath(
+    os.path.join(bld.env.PREFIX, arch_lib_path, 'include')
+    )
  with open('file-import.json', 'r') as cf:
  files = json.load(cf)
  for f in files['files-to-import']:
@@ -155,6 +158,7 @@ def build(bld):
  lwip_obj_incl.extend(drv_incl)
  lwip_obj_incl.extend(bsd_compat_incl)
  lwip_obj_incl.extend(common_includes)
+    lwip_obj_incl.append(bsp_incl)
    bld(features='c',
  target='lwip_obj',
@@ -166,9 +170,7 @@ def build(bld):
  drv_obj_incl = []
  drv_obj_incl.extend(drv_incl)
  drv_obj_incl.extend(common_includes)
-    drv_obj_incl.append(os.path.relpath(
-    os.path.join(bld.env.PREFIX, arch_lib_path, 'include')
-    ))
+    drv_obj_incl.append(bsp_incl)
    bld(features='c',
  target='driver_obj',
@@ -203,9 +205,7 @@ def build(bld):
  test_app_incl.extend(drv_incl)
  test_app_incl.extend(common_includes)
  test_app_incl.append('rtemslwip/test/')
-    test_app_incl.append(
-    os.path.relpath(os.path.join(arch_lib_path, 'include'))
-    )
+    test_app_incl.append(bsp_incl)
  bld.program(features='c',
  target='networking01.exe',
source='rtemslwip/test/networking01/sample_app.c',
@@ -235,8 +235,18 @@ def add_flags(flags, new_flags):
  flags.append(flag)
    +def strip_bsp_include(bsp_include_path, current_flags):
+    # this does not handle quted strings; maybe needed
+    for bsp_path in bsp_include_path:
+    current_flags = [flag for flag in current_flags if flag != 
bsp_path]

+    return current_flags
+
+
  def bsp_configure(conf, arch_bsp):
  conf.env.LIB += ['m']
  section_flags = ["-fdata-sections", "-ffunction-sections"]
  add_flags(conf.env.CFLAGS, section_flags)
  add_flags(conf.env.CXXFLAGS, section_flags)
+    conf.env.CFLAGS = strip_bsp_include(conf.env.IFLAGS, 
conf.env.CFLAGS)
+    conf.env.CXXFLAGS = strip_bsp_include(conf.env.IFLAGS, 
conf.env.CXXFLAGS)
+    conf.env.LINKFLAGS = strip_bsp_include(conf.env.IFLAGS, 
conf.env.LINKFLAGS)

___
devel mailing list
devel@rtems.org
http://lists.rtems.org/mailman/listinfo/devel

Re: [PATCH rtems-lwip] lwip.py: Allow lwIP to build against stale install

[Chris Johns]

On 11/9/22 11:30 pm, Kinsey Moore wrote:
> On 9/10/2022 20:22, Chris Johns wrote:
>> On 10/9/2022 12:07 pm, Kinsey Moore wrote:
>>> This removes the default BSP include path from environment variables so
>>> that rtems-lwip can build even when there's a stale version with
>>> outdated headers installed in the BSP.
>>
>> I am not following what this does and what problem you are attempting to 
>> solve?
>>
>> I am not comfortable seeing rtems_waf being overridden like this.
>>
>> Chris
>>
> This is approximately the same change that you did for libbsd that shuffles 
> the
> include paths so that lwip builds against local headers first and isn't broken
> by a stale install of rtems-lwip in the BSP install directory.

Yes it is the workaround to the problem. It is not 100% fool proof but it is
good enough for most cases (eg does not handle deleting a header that is
installed). The correct method is sandboxing the prefix into a separate
directory so you can delete the package you are working on.

I am wondering if rtems_waf handle this so we avoid coping the solution to
multiple places? Would having rtems_waf always append the BSP include work?

Chris
___
devel mailing list
devel@rtems.org
http://lists.rtems.org/mailman/listinfo/devel

[PATCH] user: Add deployment section

[chrisj]

From: Chris Johns 

---
 user/deployment/index.rst | 426 ++
 user/index.rst|   3 +-
 2 files changed, 428 insertions(+), 1 deletion(-)
 create mode 100644 user/deployment/index.rst

diff --git a/user/deployment/index.rst b/user/deployment/index.rst
new file mode 100644
index 000..d1315af
--- /dev/null
+++ b/user/deployment/index.rst
@@ -0,0 +1,426 @@
+.. SPDX-License-Identifier: CC-BY-SA-4.0
+
+.. Copyright (C) 2022 Chris Johns
+
+.. _BSPs:
+
+Deployment
+**
+.. index:: Deployment
+.. index:: packages
+
+Deployment is a process companies, organizations or teams use to
+control and manage delivery of RTEMS tools, kernels and third party
+libraries. Deployed tools, kernels and libraries are packaged and
+controlled so the same tools and libraries are used in all phases of a
+project.
+
+The Quick Start guide details how tools are built using the RSB. The
+tools are installed on your development computer and available for you
+to build applications. That build can be viewed as the simplest form
+of deployment because it is simple and easy however it does not
+scale. Building the tools and kernel on each development machine in a
+project or company is time consuming, difficult to get right and
+costly to audit.
+
+This section covers the building of tools, kernels and third party
+libraries using the RSB for deployment. Custom RSB buildset files are
+supported across releases giving you an easy update path. The RSB can
+generate a single tarfile for any prefix without needing to install
+the pieces built helping ease integration with packaging systems and
+continuous integration (CI) for automated workflows.
+
+RSB Deployment
+--
+
+The RSB provides support for deployment using custom buildset files. A
+custom buildset file resides outside the RSB and can build tools for a
+number of architectures and kernels for BSPs. Deployment can include
+third party libraries if a single BSP is being built.
+
+The RSB ``--no-install`` option builds the tools and kernel without
+the final installation phase. A prefix that is not accessible when
+running the RSB can be used. This is important if a CI flow is being
+used.
+
+The buildset tar file option ``--bset-tar-file`` packages the build's
+staging directory tree into a single tar file. The tar file can be
+used as the input source to a packaging system.
+
+Buildset configuration files can be tested by adding the ``--dry-run``
+option to the ``sb-set-builder`` command line.
+
+The buildset examples that follow assume the prefix path used does not
+exist or is not writable and the environment path does not include any
+RTEMS tools.
+
+Deployment Repository
+^
+
+Create a repository to hold a project's buildset configuration
+files:
+
+.. code-block:: none
+
+$ mkdir a-project
+$ cd a-project
+$ git init
+
+Add the RSB as a sub-module:
+
+.. code-block:: none
+
+$ git submodule add git://git.rtems.org/rtems-source-builder.git
+
+Create a configuration directory:
+
+.. code-block:: none
+
+$ mkdir config
+$ git add config
+
+Tools Configuration
+^^^
+
+This example will build a single tool set with a local configuration
+file.
+
+Create a configuration file for the ``project``:
+
+.. code-block:: none
+
+$ vi config/project-tools.bset
+
+Add the following to the buildset configuration file:
+
+.. code-block:: none
+
+#
+# Project Tools
+#
+@rtems-ver-major@/rtems-aarch64
+
+Commit the changes to the repository:
+
+.. code-block:: none
+
+   $ git add config/project-tools.bset
+   $ git commit -m "Add project aarch64 tools buildset"
+
+Build a tarfile containing the tools using the RSB submodule:
+
+.. code-block:: none
+
+   $ ./rtems-source-builder/source-builder/sb-set-builder \
+   --prefix=/opt/project --log=project.txt \
+   --bset-tar-file --no-install \
+   project-tools
+
+Once the build has finished the ``tar`` directory will contain the
+``project`` tools in a tarfile:
+
+.. code-block:: none
+
+   $ ls tar
+   project-tools.tar.bz2
+
+Inspect the tarfile to check the path matches the prefix used to build
+the tools (sizes may vary):
+
+.. code-block:: none
+
+   $ tar Jtvf tar/project-tools.tar.bz2 | less
+   drwxr-xr-x  0 chris  eng0 Sep  6 14:27 opt/project/bin/
+   -rwxr-xr-x  0 chris  eng  1320888 Sep  6 14:20 
opt/project/bin/aarch64-rtems@rtems-ver-major@-addr2line
+   -rwxr-xr-x  0 chris  eng  1358688 Sep  6 14:20 
opt/project/bin/aarch64-rtems@rtems-ver-major@-ar
+   -rwxr-xr-x  0 chris  eng  2381976 Sep  6 14:20 
opt/project/bin/aarch64-rtems@rtems-ver-major@-as
+   -rwxr-xr-x  0 chris  eng  1328440 Sep  6 14:27 
opt/project/bin/aarch64-rtems@rtems-ver-major@-c++
+   -rwxr-xr-x  0 chris  eng  1316240 Sep  6 14:20 
opt/project/bin/aarch64-rtems@rtems-ver-major@-c++filt
+   -rwxr-xr-x  0 chris  eng  1328440 Sep  6 14:27 
opt/project/bin/aarch64-rtems@rtems-ver-major@-cpp
+   -rwxr-xr-x