Re: Memory Protection project interface details (GSoC 2020)

[Utkarsh Rai]

On Wed, May 13, 2020 at 3:56 AM Joel Sherrill  wrote:

> I hate to top post but I'm not sure where to insert this.
>
> As a first step, I would ensure that paging catches stack overflow errors.
>

I understand that any thread-stack protection mechanism has to ensure
against a stack overflow error, adding this to my plan would mean that I
have to re-adjust my goals, one of them being support for typed-memory
objects. I would request your opinion as to what features are more valuable
to the community so I can focus on them.

Then move on to an optional capability where threads cannot access the
> stacks of other threads. POSIX does not say anything about whether that
> should work or not but there are cases (especially in RTEMS) where if a
> blocking thread has a message queue buffer on a stack and another thread
> does a send, then it will write to another thread's stack.
>

One of the most important parts of this project will be proper
documentation to make the user understand the possible cases where enabling
this feature will cause problems and proper handling of these issues(For
all the cases the users need stack sharing, they will have to explicitly
make certain calls to share stacks )

> This is accepted programming practice.
>


>
> I'm not opposed to an option where per-thread stack protection is
> available. But making it mandatory is a bad step. Using the MMU to detect
> stack overflow is good.
>

Yes, the thread-stack protection will be user-configurable.

>
> As Hesham mentioned, this is hard on some architectures if you don't have
> a nice page management system. Can you make sure the minimum processor
> architectural requirements are documented. Not just in an email. This will
> ultimately be information in the CPU Supplement.
>
Noted.

>
> --joel
>
> On Tue, May 12, 2020 at 5:02 PM Hesham Almatary <
> hesham.almat...@cl.cam.ac.uk> wrote:
>
>> On Tue, 12 May 2020 at 04:57, Gedare Bloom  wrote:
>> >
>> > On Thu, May 7, 2020 at 9:59 PM Hesham Almatary
>> >  wrote:
>> > >
>> > > Hello Utkarsh,
>> > >
>> > > I'd suggest you don't spend too much efforts on setting up BBB
>> > > hardware if you haven't already. Debugging on QEMU with GDB is way
>> > > easier, and you can consider either qemu-xilinx-zynq-a9 or rpi2 BSPs.
>> > > Later, you can move your code to BBB if you want, since both are based
>> > > on ARMv7.
>> > +1
>> >
>> > Past work has also used psim successfully I thought? Or am I mistaken
>> there.
>> >
>> Before my 2012 project (and part of it, yes), we used psim. The use of
>> software TLBs wasn't very ideal/easy though, so we moved to ARM in
>> 2013. The development/testing was mainly on a RPi board. I don't
>> remember there was a QEMU model for it yet.
>>
>>
>> > >
>> > > On Thu, 7 May 2020 at 18:26, Utkarsh Rai 
>> wrote:
>> > > >
>> > > > Hello,
>> > > > This is to ensure that all the interested parties are on the same
>> page before I start my project and can give their invaluable feedback.
>> > Excellent, thank you for getting the initiative.
>> >
>> > I'll be taking on the primary mentorship for your project, with
>> > support from the co-mentors (Peter, Hesham, Sebastian). For now, I
>> > prefer you to continue to make your presence on the mailing list. We
>> > will establish other forms of communication as needed and will take on
>> > IRC meetings once coding begins in earnest.
>> >
>> > > > My GSoC project, providing user-configurable thread stack
>> protection, requires adding architecture-specific low-level support as well
>> as high-level API support. I will be starting my project with ARMv7-A (on
>> BBB) based MMU since RTEMS already has quite mature support for it. As
>> already mentioned in my proposal I will be focusing more on the High-level
>> interface and let it drive whatever further low-level support is needed.
>> > > > Once the application uses MMU for thread stack address generation
>> each thread will be automatically protected as the page tables other than
>> that of the executing thread would be made dormant. When the user has to
>> share thread stacks they will have to obtain the stack attributes of the
>> threads to be shared by pthread_attr_getstack() and then get a file
>> descriptor of the memory to be mapped by a call to shm_open() and finally
>> map this to the stack of the other thread through
>> > > > mmap(), this is the POSIX compliant way I could think of. Now at
>> the low level, it means mapping the page table of the thread to be shared
>> into the address space of the executing thread. This is an area where the
>> low-level support has to be provided. At the high-level, this means
>> providing support to mmap and shared-memory interface as mmap provides
>> support for a file by simply
>> > > > copying the memory from the file to the destination. For shared
>> memory objects it can
>> > > > provide read/write access but cannot provide restriction of
>> write/read access. One of the areas that I have to look into more detail is
>> thread context-

Help on how to configure for user-defined memory protection support (GSoC 2020)

[Utkarsh Rai]

Hello,
My GSoC project,  providing thread stack protection support, has to be a
user-configurable feature.
My question is,  what would be the best way to implement this, my idea was
to model it based on the existing system configuration
, but Dr.
Gedare pointed out that configuration is undergoing heavy changes and may
look completely different in future releases. Kindly advise me as to what
would be the best way to proceed.
___
devel mailing list
devel@rtems.org
http://lists.rtems.org/mailman/listinfo/devel

Re: Adding fenv support for ARM

[Joel Sherrill]

National free hint day!!!

Look above the line that is reported. You will see __BEGIN_DECL. It isn't
defined. It is intended to expand to the __cplusplus wrapper.

After you fix this, you will sequentially find 2 other things to change in
this file.

Then we need to make a decision whether the static inline versions of the
methods in sys/fenv.h are appropriate for newlib or not.

--joel

On Tue, May 12, 2020 at 5:20 PM Joel Sherrill  wrote:

> Gedare is right but one more piece of information is needed. When newlib
> is being built, it lays target dependent files on top of generic ones. So
> the default sys/fenv.h is replaced with something like
> libc/machine/arm/sys/fenv.h. Make sure that
> /home/eshan/development/newlib/newlib-cygwin/newlib/libc/include/fenv.h is
> in fact the file you think it should be.
>
> Also look at what is supposed to be in the generic fenv.h and
> . You will probably have to tweak it some.
>
> Check out what is in x86_64/fenv.h and riscv/fenv.h with respect to the
> default environment variable. It looks to be declared differently in the
> other ports so you may have to make it match.
>
> But more output does help.
>
> --joel
>
>
>
> On Tue, May 12, 2020 at 4:53 PM Gedare Bloom  wrote:
>
>> Take the first error reported, and show what is being compiled. You've
>> trimmed off too much preceding this to know what is being compiled and
>> how this might be influencing your error.
>>
>> The specific error you're getting is often due to some syntax errors
>> preceding the line shown.
>>
>> On Tue, May 12, 2020 at 2:59 PM Eshan Dhawan 
>> wrote:
>> >
>> > Hello everyone,
>> > After adding the files and compiling. I am
>> > getting a few errors which I am not able to rectify.
>> > If you could take a look and tell how to sort them.
>> >
>> > link to newlib's git repo:
>> https://github.com/eshandhawan51/newlib-cygwin/tree/add_fenv_support
>> >
>> > errors :
>> >
>> > In file included from
>> /home/eshan/development/newlib/newlib-cygwin/newlib/libc/include/fenv.h:15:0,
>> >  from
>> ../../../../../../newlib-cygwin/newlib/libm/fenv/feclearexcept.c:57:
>> >
>> /home/eshan/development/newlib/c-arm-rtems5-newlib/arm-rtems5/thumb/newlib/targ-include/sys/fenv.h:82:1:
>> error: expected '=', ',', ';', 'asm' or '__attribute__' before 'extern'
>> >  extern const fenv_t *_fe_dfl_env;
>> >  ^~
>> > In file included from
>> ../../../../../../newlib-cygwin/newlib/libm/fenv/feclearexcept.c:57:0:
>> >
>> /home/eshan/development/newlib/newlib-cygwin/newlib/libc/include/fenv.h:22:1:
>> error: expected '=', ',', ';', 'asm' or '__attribute__' before 'int'
>> >  int feclearexcept(int excepts);
>> >  ^~~
>> >
>> > thanks
>> > -Eshan
>> > On Sat, May 9, 2020 at 2:53 AM Joel Sherrill  wrote:
>> >>
>> >>
>> >>
>> >> On Thu, May 7, 2020 at 1:33 PM Eshan Dhawan 
>> wrote:
>> >>>
>> >>> Hello everyone,
>> >>> This the tread to discuss adding fenv support for ARM architecture in
>> newlib.
>> >>>
>> >>> Link to previous thread:
>> https://lists.rtems.org/pipermail/devel/2020-March/058473.html
>> >>>
>> >>> ---FreeBSD Source
>> >>> # https://github.com/freebsd/freebsd/tree/master/lib/msun/arm
>> >>> #
>> https://github.com/freebsd/freebsd/blob/master/sys/arm/include/ieeefp.h
>> >>> ---NetBSD Source
>> >>> # https://github.com/NetBSD/src/tree/trunk/lib/libm/arch/arm
>> >>>
>> >>> --- Musl has no implementation for fenv support for ARM  in it
>> >>>
>> >>> After going through the implementations both FreeBSD and NetBSD have
>> a stub implementation.
>> >>> But then FreeBSD implementation seems to be more portable compared to
>> that in NetBSD.
>> >>>
>> >>> Should I start porting it from FreeBSD?
>> >>> And which ARM BSP should I use for testing the changes?
>> >>> Currently, I have xilinx_zynq_a9_qemu build
>> >>
>> >>
>> >> I think the FreeBSD implementation is the one to use. It looks to
>> support hard and soft floating point implementations.
>> >>
>> >> I think the fenv.h file there will have to be worked on so some of the
>> contents go into sys or machine/fenv.h. There is a portable fenv.h.
>> >>
>> >>>
>> >>>
>> >>> There would also be a requirement to discuss where to place the files
>> in Newlib.
>> >>
>> >>
>> >> Not much to discuss. It should go into
>> newlib-cygwin/newlib/libm/machine/arm.
>> >>>
>> >>>
>> >>> Thanks
>> >>> -Eshan
>> >>>
>> > ___
>> > devel mailing list
>> > devel@rtems.org
>> > http://lists.rtems.org/mailman/listinfo/devel
>>
>
___
devel mailing list
devel@rtems.org
http://lists.rtems.org/mailman/listinfo/devel

Re: Memory Protection project interface details (GSoC 2020)

[Gedare Bloom]

On Wed, May 13, 2020 at 6:12 AM Utkarsh Rai  wrote:
>
>
>
> On Wed, May 13, 2020 at 3:56 AM Joel Sherrill  wrote:
>>
>> I hate to top post but I'm not sure where to insert this.
>>
>> As a first step, I would ensure that paging catches stack overflow errors.
>
>
> I understand that any thread-stack protection mechanism has to ensure against 
> a stack overflow error, adding this to my plan would mean that I have to 
> re-adjust my goals, one of them being support for typed-memory objects. I 
> would request your opinion as to what features are more valuable to the 
> community so I can focus on them.
>

When Joel says stack overflow he means a thread writing beyond the
thread's stack area. I don't think this would require TYM? Usually the
way this is done is to add a gap between stacks when you allocate
them, which is a bit wasteful of physical memory.

>> Then move on to an optional capability where threads cannot access the 
>> stacks of other threads. POSIX does not say anything about whether that 
>> should work or not but there are cases (especially in RTEMS) where if a 
>> blocking thread has a message queue buffer on a stack and another thread 
>> does a send, then it will write to another thread's stack.
>
>
> One of the most important parts of this project will be proper documentation 
> to make the user understand the possible cases where enabling this feature 
> will cause problems and proper handling of these issues(For all the cases the 
> users need stack sharing, they will have to explicitly make certain calls to 
> share stacks )
>>
>> This is accepted programming practice.
>
>
>>
>>
>> I'm not opposed to an option where per-thread stack protection is available. 
>> But making it mandatory is a bad step. Using the MMU to detect stack 
>> overflow is good.
>
>
> Yes, the thread-stack protection will be user-configurable.
>>
>>
>> As Hesham mentioned, this is hard on some architectures if you don't have a 
>> nice page management system. Can you make sure the minimum processor 
>> architectural requirements are documented. Not just in an email. This will 
>> ultimately be information in the CPU Supplement.
>
> Noted.
>>
>>
>> --joel
>>
>> On Tue, May 12, 2020 at 5:02 PM Hesham Almatary 
>>  wrote:
>>>
>>> On Tue, 12 May 2020 at 04:57, Gedare Bloom  wrote:
>>> >
>>> > On Thu, May 7, 2020 at 9:59 PM Hesham Almatary
>>> >  wrote:
>>> > >
>>> > > Hello Utkarsh,
>>> > >
>>> > > I'd suggest you don't spend too much efforts on setting up BBB
>>> > > hardware if you haven't already. Debugging on QEMU with GDB is way
>>> > > easier, and you can consider either qemu-xilinx-zynq-a9 or rpi2 BSPs.
>>> > > Later, you can move your code to BBB if you want, since both are based
>>> > > on ARMv7.
>>> > +1
>>> >
>>> > Past work has also used psim successfully I thought? Or am I mistaken 
>>> > there.
>>> >
>>> Before my 2012 project (and part of it, yes), we used psim. The use of
>>> software TLBs wasn't very ideal/easy though, so we moved to ARM in
>>> 2013. The development/testing was mainly on a RPi board. I don't
>>> remember there was a QEMU model for it yet.
>>>
>>>
>>> > >
>>> > > On Thu, 7 May 2020 at 18:26, Utkarsh Rai  
>>> > > wrote:
>>> > > >
>>> > > > Hello,
>>> > > > This is to ensure that all the interested parties are on the same 
>>> > > > page before I start my project and can give their invaluable feedback.
>>> > Excellent, thank you for getting the initiative.
>>> >
>>> > I'll be taking on the primary mentorship for your project, with
>>> > support from the co-mentors (Peter, Hesham, Sebastian). For now, I
>>> > prefer you to continue to make your presence on the mailing list. We
>>> > will establish other forms of communication as needed and will take on
>>> > IRC meetings once coding begins in earnest.
>>> >
>>> > > > My GSoC project, providing user-configurable thread stack protection, 
>>> > > > requires adding architecture-specific low-level support as well as 
>>> > > > high-level API support. I will be starting my project with ARMv7-A 
>>> > > > (on BBB) based MMU since RTEMS already has quite mature support for 
>>> > > > it. As already mentioned in my proposal I will be focusing more on 
>>> > > > the High-level interface and let it drive whatever further low-level 
>>> > > > support is needed.
>>> > > > Once the application uses MMU for thread stack address generation 
>>> > > > each thread will be automatically protected as the page tables other 
>>> > > > than that of the executing thread would be made dormant. When the 
>>> > > > user has to share thread stacks they will have to obtain the stack 
>>> > > > attributes of the threads to be shared by pthread_attr_getstack() and 
>>> > > > then get a file descriptor of the memory to be mapped by a call to 
>>> > > > shm_open() and finally map this to the stack of the other thread 
>>> > > > through
>>> > > > mmap(), this is the POSIX compliant way I could think of. Now at the 
>>> > > > low level, it means mapping the page table of

Re: [PATCH] posix: Get real priority in pthread_getattr_np()

[Heinz Junkes]

Thanks for the good ideas.
I have now replaced the accesses to the internal thread structure 
(real_priority, current_priority) with API - calls. 
One of the calls is from the pthread-library and therefore needs --enable-posix 
when creating the BSP.
Calls used
pthread_getschedparam
rtems_task_set_priority

Heinz


> On 12. May 2020, at 07:28, Sebastian Huber 
>  wrote:
> 
> This is in line with pthread_setschedparam() and
> pthread_getschedparam().
> 
> Update #2514.
> ---
> cpukit/posix/src/pthreadgetattrnp.c | 10 ++
> 1 file changed, 6 insertions(+), 4 deletions(-)
> 
> diff --git a/cpukit/posix/src/pthreadgetattrnp.c 
> b/cpukit/posix/src/pthreadgetattrnp.c
> index a449ad48eb..eae29d1de7 100644
> --- a/cpukit/posix/src/pthreadgetattrnp.c
> +++ b/cpukit/posix/src/pthreadgetattrnp.c
> @@ -40,6 +40,7 @@ int pthread_getattr_np(
>   ISR_lock_Context  lock_context;
>   Thread_CPU_budget_algorithms  budget_algorithm;
>   const Scheduler_Control  *scheduler;
> +  Priority_Control  priority;
>   bool  ok;
> 
>   if ( attr == NULL ) {
> @@ -66,15 +67,12 @@ int pthread_getattr_np(
>   }
> 
>   scheduler = _Thread_Scheduler_get_home( the_thread );
> -  attr->schedparam.sched_priority = _POSIX_Priority_From_core(
> -scheduler,
> -_Thread_Get_priority( the_thread )
> -  );
>   _POSIX_Threads_Get_sched_param_sporadic(
> the_thread,
> scheduler,
> &attr->schedparam
>   );
> +  priority = the_thread->Real_priority.priority;
> 
>   if ( _Thread_Is_joinable( the_thread ) ) {
> attr->detachstate = PTHREAD_CREATE_JOINABLE;
> @@ -97,6 +95,10 @@ int pthread_getattr_np(
>   attr->is_initialized = true;
>   attr->contentionscope = PTHREAD_SCOPE_PROCESS;
>   attr->cputime_clock_allowed = 1;
> +  attr->schedparam.sched_priority = _POSIX_Priority_From_core(
> +scheduler,
> +priority
> +  );
>   attr->schedpolicy =
> _POSIX_Thread_Translate_to_sched_policy( budget_algorithm );
> 
> -- 
> 2.26.1
> 
> ___
> devel mailing list
> devel@rtems.org
> http://lists.rtems.org/mailman/listinfo/devel



smime.p7s
Description: S/MIME cryptographic signature
___
devel mailing list
devel@rtems.org
http://lists.rtems.org/mailman/listinfo/devel

Re: Memory Protection project interface details (GSoC 2020)

[dufault]

> On May 12, 2020, at 16:51 , Gedare Bloom  wrote:
> 
> On Tue, May 12, 2020 at 10:58 AM Utkarsh Rai  wrote:
>> 
>> 
>> 
>> On Tue, May 12, 2020 at 9:27 AM Gedare Bloom  wrote:
>>> 
>>> On Thu, May 7, 2020 at 9:59 PM Hesham Almatary
>>>  wrote:
 
 Hello Utkarsh,
 
 I'd suggest you don't spend too much efforts on setting up BBB
 hardware if you haven't already. Debugging on QEMU with GDB is way
 easier, and you can consider either qemu-xilinx-zynq-a9 or rpi2 BSPs.
 Later, you can move your code to BBB if you want, since both are based
 on ARMv7.
>>> +1
>>> 
>>> Past work has also used psim successfully I thought? Or am I mistaken there.
>>> 
 
 On Thu, 7 May 2020 at 18:26, Utkarsh Rai  wrote:
> 
> Hello,
> This is to ensure that all the interested parties are on the same page 
> before I start my project and can give their invaluable feedback.
>>> Excellent, thank you for getting the initiative.
>>> 
>>> I'll be taking on the primary mentorship for your project, with
>>> support from the co-mentors (Peter, Hesham, Sebastian). For now, I
>>> prefer you to continue to make your presence on the mailing list. We
>>> will establish other forms of communication as needed and will take on
>>> IRC meetings once coding begins in earnest.
>>> 
> My GSoC project, providing user-configurable thread stack protection, 
> requires adding architecture-specific low-level support as well as 
> high-level API support. I will be starting my project with ARMv7-A (on 
> BBB) based MMU since RTEMS already has quite mature support for it. As 
> already mentioned in my proposal I will be focusing more on the 
> High-level interface and let it drive whatever further low-level support 
> is needed.
> Once the application uses MMU for thread stack address generation each 
> thread will be automatically protected as the page tables other than that 
> of the executing thread would be made dormant. When the user has to share 
> thread stacks they will have to obtain the stack attributes of the 
> threads to be shared by pthread_attr_getstack() and then get a file 
> descriptor of the memory to be mapped by a call to shm_open() and finally 
> map this to the stack of the other thread through
> mmap(), this is the POSIX compliant way I could think of. Now at the low 
> level, it means mapping the page table of the thread to be shared into 
> the address space of the executing thread. This is an area where the 
> low-level support has to be provided. At the high-level, this means 
> providing support to mmap and shared-memory interface as mmap provides 
> support for a file by simply
> copying the memory from the file to the destination. For shared memory 
> objects it can
> provide read/write access but cannot provide restriction of write/read 
> access. One of the areas that I have to look into more detail is thread 
> context-switch, as after every context switch the TLBs need to be flushed 
> and reinitialized lest we get an invalid address for the executing 
> thread. Since context-switch is low-level architecture-specific, this 
> also has to be provided with more support.
>>> 
>>> This is really dense text. Try to break apart your writing a little
>>> bit to help clarify your thoughts.  You should also translate some of
>>> your proposal into a wiki page if you haven't started that yet, and a
>>> blog post. Both of those will help to focus your thoughts into words.
>>> 
>>> "mapping the page table" is not meaningful to me. I think you mean
>>> something like "mapping a page from the page table"?
>>> 
>>> Will the design
>>> support sharing task stacks using MPUs with 4 regions? 8?  (It seems
>>> challenging to me, but might be possible in some limited
>>> configurations. Having support for those kinds of targets might still
>>> be useful, with the caveat that sharing stacks is not possible.)
>> 
>> 
>> I will have to look into this in a bit more detail before I can give you a 
>> comprehensive answer.
>> 
>>> 
>>> The first step is to get a BSP running that has sufficient
>>> capabilities for you to test out memory protection with. Do a little
>>> bit of digging, but definitely simulation is the way to go.
>> 
>> 
>> As suggested by Hesham, I have been able to run the qemu-Xilinx-zynq-a9 BSP 
>> on qemu and I have learned how to debug it through GDB.
>> The BSP supports memory protection and as pointed out once I get it done on 
>> this, I can move my code for other BSP with ARMv7 (RPI, BBB ).
>> 
>>> 
>>> The second step from my perspective is to determine how to introduce
>>> strict isolation between task stacks. Don't worry about sharing at
>>> this stage, but rather can you completely isolate tasks? Then you can
>>> start to poke holes in the isolation.
>> 
>> 
>> My understanding of this is that to completely isolate the tasks and page 
>> tables of the task wi

Re: Memory Protection project interface details (GSoC 2020)

[Utkarsh Rai]

On Wed, May 13, 2020 at 8:11 PM Gedare Bloom  wrote:

> On Wed, May 13, 2020 at 6:12 AM Utkarsh Rai 
> wrote:
> >
> >
> >
> > On Wed, May 13, 2020 at 3:56 AM Joel Sherrill  wrote:
> >>
> >> I hate to top post but I'm not sure where to insert this.
> >>
> >> As a first step, I would ensure that paging catches stack overflow
> errors.
> >
> >
> > I understand that any thread-stack protection mechanism has to ensure
> against a stack overflow error, adding this to my plan would mean that I
> have to re-adjust my goals, one of them being support for typed-memory
> objects. I would request your opinion as to what features are more valuable
> to the community so I can focus on them.
> >
>
> When Joel says stack overflow he means a thread writing beyond the
> thread's stack area. I don't think this would require TYM? Usually the
> way this is done is to add a gap between stacks when you allocate
> them, which is a bit wasteful of physical memory.
>
> Yes, it will not require TYM, sorry I did not make my thoughts clear.
Protection against stack overflow is necessary and independent of the TYM
interface, my concern is I do not have an estimate of the time it would
take to implement this. This means I will have to re-adjust my goals such
as the implementation of the TYM interface. If we plan on providing
protection against stack overflow I would request your opinion on how
should I re-adjust my goals so that it remains feasible for me to complete
them.

> >> Then move on to an optional capability where threads cannot access the
> stacks of other threads. POSIX does not say anything about whether that
> should work or not but there are cases (especially in RTEMS) where if a
> blocking thread has a message queue buffer on a stack and another thread
> does a send, then it will write to another thread's stack.
> >
> >
> > One of the most important parts of this project will be proper
> documentation to make the user understand the possible cases where enabling
> this feature will cause problems and proper handling of these issues(For
> all the cases the users need stack sharing, they will have to explicitly
> make certain calls to share stacks )
> >>
> >> This is accepted programming practice.
> >
> >
> >>
> >>
> >> I'm not opposed to an option where per-thread stack protection is
> available. But making it mandatory is a bad step. Using the MMU to detect
> stack overflow is good.
> >
> >
> > Yes, the thread-stack protection will be user-configurable.
> >>
> >>
> >> As Hesham mentioned, this is hard on some architectures if you don't
> have a nice page management system. Can you make sure the minimum processor
> architectural requirements are documented. Not just in an email. This will
> ultimately be information in the CPU Supplement.
> >
> > Noted.
> >>
> >>
> >> --joel
> >>
> >> On Tue, May 12, 2020 at 5:02 PM Hesham Almatary <
> hesham.almat...@cl.cam.ac.uk> wrote:
> >>>
> >>> On Tue, 12 May 2020 at 04:57, Gedare Bloom  wrote:
> >>> >
> >>> > On Thu, May 7, 2020 at 9:59 PM Hesham Almatary
> >>> >  wrote:
> >>> > >
> >>> > > Hello Utkarsh,
> >>> > >
> >>> > > I'd suggest you don't spend too much efforts on setting up BBB
> >>> > > hardware if you haven't already. Debugging on QEMU with GDB is way
> >>> > > easier, and you can consider either qemu-xilinx-zynq-a9 or rpi2
> BSPs.
> >>> > > Later, you can move your code to BBB if you want, since both are
> based
> >>> > > on ARMv7.
> >>> > +1
> >>> >
> >>> > Past work has also used psim successfully I thought? Or am I
> mistaken there.
> >>> >
> >>> Before my 2012 project (and part of it, yes), we used psim. The use of
> >>> software TLBs wasn't very ideal/easy though, so we moved to ARM in
> >>> 2013. The development/testing was mainly on a RPi board. I don't
> >>> remember there was a QEMU model for it yet.
> >>>
> >>>
> >>> > >
> >>> > > On Thu, 7 May 2020 at 18:26, Utkarsh Rai 
> wrote:
> >>> > > >
> >>> > > > Hello,
> >>> > > > This is to ensure that all the interested parties are on the
> same page before I start my project and can give their invaluable feedback.
> >>> > Excellent, thank you for getting the initiative.
> >>> >
> >>> > I'll be taking on the primary mentorship for your project, with
> >>> > support from the co-mentors (Peter, Hesham, Sebastian). For now, I
> >>> > prefer you to continue to make your presence on the mailing list. We
> >>> > will establish other forms of communication as needed and will take
> on
> >>> > IRC meetings once coding begins in earnest.
> >>> >
> >>> > > > My GSoC project, providing user-configurable thread stack
> protection, requires adding architecture-specific low-level support as well
> as high-level API support. I will be starting my project with ARMv7-A (on
> BBB) based MMU since RTEMS already has quite mature support for it. As
> already mentioned in my proposal I will be focusing more on the High-level
> interface and let it drive whatever further low-level support is needed.
> >>> > > > Once the application uses MM

Re: Memory Protection project interface details (GSoC 2020)

[Joel Sherrill]

On Wed, May 13, 2020 at 9:41 AM Gedare Bloom  wrote:

> On Wed, May 13, 2020 at 6:12 AM Utkarsh Rai 
> wrote:
> >
> >
> >
> > On Wed, May 13, 2020 at 3:56 AM Joel Sherrill  wrote:
> >>
> >> I hate to top post but I'm not sure where to insert this.
> >>
> >> As a first step, I would ensure that paging catches stack overflow
> errors.
> >
> >
> > I understand that any thread-stack protection mechanism has to ensure
> against a stack overflow error, adding this to my plan would mean that I
> have to re-adjust my goals, one of them being support for typed-memory
> objects. I would request your opinion as to what features are more valuable
> to the community so I can focus on them.
> >
>
> When Joel says stack overflow he means a thread writing beyond the
> thread's stack area. I don't think this would require TYM? Usually the
> way this is done is to add a gap between stacks when you allocate
> them, which is a bit wasteful of physical memory.
>

A gap in the virtual address?  I guess if you map virtual:physical 1:1,
then it is a gap in both. But technically, it doesn't have to be 1:1.

Independent of being pedantic on that, enlighten me how using the MMU
for stack overflow protection isn't a logical step on the path of this
project?
It is just a mapping thread stacks and letting every thread have access to
all threads memory but leaving gaps.

I have seen the basic benefits of an MMU as follows (increasing in
complexity and dynamicism):

+ mark region of memory as read/write, execute, read only, no cache,
   and for stack use only. What you can do depends on the architecture.
  This is a gross form of memory protection using large areas of memory.
   This is usually done at BSP startup and left along.

+ Provide each thread a protected area where overflow/underflow is
  detected by the MMU. This requires finer grain control but if still simple
   since it is only changed at run-time by the thread stack
allocator/deallocator.

+ Finer grain control of which threads can access which other threads'
stack.
   This will require more management and something happens at context
switch.

+ Very fine grain control of objects like CHERI and other similar
approaches.

This project is aiming for the third. Wouldn't it need to move through the
first
two capabilities?


>
> >> Then move on to an optional capability where threads cannot access the
> stacks of other threads. POSIX does not say anything about whether that
> should work or not but there are cases (especially in RTEMS) where if a
> blocking thread has a message queue buffer on a stack and another thread
> does a send, then it will write to another thread's stack.
> >
> >
> > One of the most important parts of this project will be proper
> documentation to make the user understand the possible cases where enabling
> this feature will cause problems and proper handling of these issues(For
> all the cases the users need stack sharing, they will have to explicitly
> make certain calls to share stacks )
> >>
> >> This is accepted programming practice.
> >
> >
> >>
> >>
> >> I'm not opposed to an option where per-thread stack protection is
> available. But making it mandatory is a bad step. Using the MMU to detect
> stack overflow is good.
> >
> >
> > Yes, the thread-stack protection will be user-configurable.
> >>
> >>
> >> As Hesham mentioned, this is hard on some architectures if you don't
> have a nice page management system. Can you make sure the minimum processor
> architectural requirements are documented. Not just in an email. This will
> ultimately be information in the CPU Supplement.
> >
> > Noted.
> >>
> >>
> >> --joel
> >>
> >> On Tue, May 12, 2020 at 5:02 PM Hesham Almatary <
> hesham.almat...@cl.cam.ac.uk> wrote:
> >>>
> >>> On Tue, 12 May 2020 at 04:57, Gedare Bloom  wrote:
> >>> >
> >>> > On Thu, May 7, 2020 at 9:59 PM Hesham Almatary
> >>> >  wrote:
> >>> > >
> >>> > > Hello Utkarsh,
> >>> > >
> >>> > > I'd suggest you don't spend too much efforts on setting up BBB
> >>> > > hardware if you haven't already. Debugging on QEMU with GDB is way
> >>> > > easier, and you can consider either qemu-xilinx-zynq-a9 or rpi2
> BSPs.
> >>> > > Later, you can move your code to BBB if you want, since both are
> based
> >>> > > on ARMv7.
> >>> > +1
> >>> >
> >>> > Past work has also used psim successfully I thought? Or am I
> mistaken there.
> >>> >
> >>> Before my 2012 project (and part of it, yes), we used psim. The use of
> >>> software TLBs wasn't very ideal/easy though, so we moved to ARM in
> >>> 2013. The development/testing was mainly on a RPi board. I don't
> >>> remember there was a QEMU model for it yet.
> >>>
> >>>
> >>> > >
> >>> > > On Thu, 7 May 2020 at 18:26, Utkarsh Rai 
> wrote:
> >>> > > >
> >>> > > > Hello,
> >>> > > > This is to ensure that all the interested parties are on the
> same page before I start my project and can give their invaluable feedback.
> >>> > Excellent, thank you for getting the initiative.
> >>> >
> >>> > I'll be taking on 

Re: GSoC: RTEMS directory for FreeBSD imports

[Niteesh G. S.]

Hello,

This mail is to regain attention for this topic and also to discuss a few
details
regarding the porting process.

In the previous thread, Sebastian mentioned that we will be hard wiring the
OF
functions with the FDT implementation. For reasons please have a look at
previous thread.
https://lists.rtems.org/pipermail/devel/2020-May/059762.html
This will be achieved by inlining the functions in openfirm.h with functions
defined in ofw_fdt.c. This approach is valid for most functions but not all.
Since not all functions have a one to one mapping.

For example,
   The OF_peer can be directly mapped to ofw_fdt_peer.
   But for OF_getencprop it calls ofw_fdt_getprop after few manipulations.
Inlining these functions doesn't seem like a good practice for me.

One way to approach this would be to add the implementation for these
functions in
ofw_fdt.c but this would cause code redundancy if we plan to import
openfirm.c
in future since these functions are already defined in openfirm.c.

Another approach will be to import openfirm.c also and redefine the OFW_*
macro to directly call the respective functions.

I don't really know if there is any other better way to approach this. Any
suggestion on
how to proceed.

Once this is resolved I will proceed with the porting even if we haven't
finalized the
directory since it is just a matter of moving files once we are finalized.

If you think this is too early to start with coding for GSoC please
understand that my
university exams haven't been conducted yet. And due to the COVID pandemic,
the
dates are quite uncertain. But it is mostly expected to happen during the
coding period (july/aug)
and this would eat up quite a lot of time. So just to be one the safe side
I started
quite early.

Thank,
Niteesh.

On Mon, May 11, 2020 at 12:48 PM Christian Mauderer <
christian.maude...@embedded-brains.de> wrote:

> On 11/05/2020 09:11, Chris Johns wrote:
> > On 11/5/20 4:55 pm, Christian Mauderer wrote:
> >> On 11/05/2020 06:57, Chris Johns wrote:
> >>>
> >>>
> >>> On 11/5/20 2:03 pm, Niteesh G. S. wrote:
>  On Mon, May 11, 2020 at 4:34 AM Chris Johns   > wrote:
> 
>   On 10/5/20 6:17 pm, Niteesh G. S. wrote:
>    > This thread is a continuation of "GSoC 2020: Implementation
>  of OFW
>    > functions".
>    >
>    > A summary of points discussed in that thread is given below.
>    >
>    > Below is a short description of my GSoC project. For more
>   information please
>    > refer to the wiki.
>    >
>  https://devel.rtems.org/wiki/GSoC/2020/Beagle_FDT_initialization
>    > My GSoC project deals with refactoring the Beagle BSP to add
>   support for FDT
>    > based initialization. As part of this process, I will have to
>   import the
>    > pin mux driver
>    > into RTEMS which currently is present in libBSD.
>    > This would require having support for OFW functions which are
>   currently
>    > not implemented
>    > in RTEMS. Some drivers(eg: imx_iomux.c) which require these
>   functions
>    > provide
>    > a local implementation using libFDT.
> 
>   I hope you do not mind if I wind back a couple of steps...
> 
>   OFW? Is this http://wiki.laptop.org/go/Open_Firmware?
>   How does OFW related to FDT?
> 
> 
>  We are only interested in the device tree interface provided by the
> OF.
>  Functions like OF_getprop, OF_parent, etc.
> 
> >>>
> >>> Why not call libfdt functions? I am wondering what there is in FreeBSD
> >>> that is calling these functions? I am not questioning the need, it is a
> >>> case of not understanding the dependency.
> >>
> >> The use case for the OF_... and ofw_... functions is more or less a
> >> simple import from FreeBSD drivers. Beneath that there are some quite
> >> nice shortcuts in the OF_... and ofw_... functions that would have to be
> >> re-implemented in each driver (like ofw_bus_node_status_okay()).
> >>
> >> Some drivers already use hacked versions of the functions. For example:
> >>
> >> bsps/sparc64/shared/clock/ckinit.c
> >> bsps/arm/imx/start/imx_iomux.c
> >>
> >> A use case where the OF_... stuff would have been handy:
> >>
> >> For the imx pin initialization I would have loved to just use the
> >> fdt_pinctrl_configure_tree() from FreeBSD. But that one had a lot of
> >> OF_.. stuff. Therefore I had to reimplement that function in a
> >> imx_pinctrl_configure_children(). My implementation basically does
> >> exactly the same thing but uses fdt_... functions instead of the OF_...
> >> functions.
> >
> > Thanks. I think I understand. The scope seems to be the low level SoC
> > type initialisation. This makes sense.
>
> And maybe some low level drivers like serial or I2C. I don't think that
> we should go much further in complexity. Basically everything th

Re: Memory Protection project interface details (GSoC 2020)

[Utkarsh Rai]

On Wed, May 13, 2020 at 10:27 PM Joel Sherrill  wrote:

>
>
> On Wed, May 13, 2020 at 9:41 AM Gedare Bloom  wrote:
>
>> On Wed, May 13, 2020 at 6:12 AM Utkarsh Rai 
>> wrote:
>> >
>> >
>> >
>> > On Wed, May 13, 2020 at 3:56 AM Joel Sherrill  wrote:
>> >>
>> >> I hate to top post but I'm not sure where to insert this.
>> >>
>> >> As a first step, I would ensure that paging catches stack overflow
>> errors.
>> >
>> >
>> > I understand that any thread-stack protection mechanism has to ensure
>> against a stack overflow error, adding this to my plan would mean that I
>> have to re-adjust my goals, one of them being support for typed-memory
>> objects. I would request your opinion as to what features are more valuable
>> to the community so I can focus on them.
>> >
>>
>> When Joel says stack overflow he means a thread writing beyond the
>> thread's stack area. I don't think this would require TYM? Usually the
>> way this is done is to add a gap between stacks when you allocate
>> them, which is a bit wasteful of physical memory.
>>
>
> A gap in the virtual address?  I guess if you map virtual:physical 1:1,
> then it is a gap in both. But technically, it doesn't have to be 1:1.
>
> Independent of being pedantic on that, enlighten me how using the MMU
> for stack overflow protection isn't a logical step on the path of this
> project?
> It is just a mapping thread stacks and letting every thread have access to
> all threads memory but leaving gaps.
>
> I have seen the basic benefits of an MMU as follows (increasing in
> complexity and dynamicism):
>
> + mark region of memory as read/write, execute, read only, no cache,
>and for stack use only. What you can do depends on the architecture.
>   This is a gross form of memory protection using large areas of memory.
>This is usually done at BSP startup and left along.
>
> + Provide each thread a protected area where overflow/underflow is
>   detected by the MMU. This requires finer grain control but if still
> simple
>since it is only changed at run-time by the thread stack
> allocator/deallocator.
>
> + Finer grain control of which threads can access which other threads'
> stack.
>This will require more management and something happens at context
> switch.
>
> + Very fine grain control of objects like CHERI and other similar
> approaches.
>
> This project is aiming for the third. Wouldn't it need to move through the
> first
> two capabilities?
>

Thank you, the stack-overflow protection should be the first obvious step
when implementing this feature, your analysis has helped me clear my
doubts.

>
>
>>
>> >> Then move on to an optional capability where threads cannot access the
>> stacks of other threads. POSIX does not say anything about whether that
>> should work or not but there are cases (especially in RTEMS) where if a
>> blocking thread has a message queue buffer on a stack and another thread
>> does a send, then it will write to another thread's stack.
>> >
>> >
>> > One of the most important parts of this project will be proper
>> documentation to make the user understand the possible cases where enabling
>> this feature will cause problems and proper handling of these issues(For
>> all the cases the users need stack sharing, they will have to explicitly
>> make certain calls to share stacks )
>> >>
>> >> This is accepted programming practice.
>> >
>> >
>> >>
>> >>
>> >> I'm not opposed to an option where per-thread stack protection is
>> available. But making it mandatory is a bad step. Using the MMU to detect
>> stack overflow is good.
>> >
>> >
>> > Yes, the thread-stack protection will be user-configurable.
>> >>
>> >>
>> >> As Hesham mentioned, this is hard on some architectures if you don't
>> have a nice page management system. Can you make sure the minimum processor
>> architectural requirements are documented. Not just in an email. This will
>> ultimately be information in the CPU Supplement.
>> >
>> > Noted.
>> >>
>> >>
>> >> --joel
>> >>
>> >> On Tue, May 12, 2020 at 5:02 PM Hesham Almatary <
>> hesham.almat...@cl.cam.ac.uk> wrote:
>> >>>
>> >>> On Tue, 12 May 2020 at 04:57, Gedare Bloom  wrote:
>> >>> >
>> >>> > On Thu, May 7, 2020 at 9:59 PM Hesham Almatary
>> >>> >  wrote:
>> >>> > >
>> >>> > > Hello Utkarsh,
>> >>> > >
>> >>> > > I'd suggest you don't spend too much efforts on setting up BBB
>> >>> > > hardware if you haven't already. Debugging on QEMU with GDB is way
>> >>> > > easier, and you can consider either qemu-xilinx-zynq-a9 or rpi2
>> BSPs.
>> >>> > > Later, you can move your code to BBB if you want, since both are
>> based
>> >>> > > on ARMv7.
>> >>> > +1
>> >>> >
>> >>> > Past work has also used psim successfully I thought? Or am I
>> mistaken there.
>> >>> >
>> >>> Before my 2012 project (and part of it, yes), we used psim. The use of
>> >>> software TLBs wasn't very ideal/easy though, so we moved to ARM in
>> >>> 2013. The development/testing was mainly on a RPi board. I don't
>> >>> remember there was a QEMU mode

Re: Memory Protection project interface details (GSoC 2020)

[Gedare Bloom]

On Wed, May 13, 2020 at 10:53 AM Utkarsh Rai  wrote:
>
>
>
> On Wed, May 13, 2020 at 8:11 PM Gedare Bloom  wrote:
>>
>> On Wed, May 13, 2020 at 6:12 AM Utkarsh Rai  wrote:
>> >
>> >
>> >
>> > On Wed, May 13, 2020 at 3:56 AM Joel Sherrill  wrote:
>> >>
>> >> I hate to top post but I'm not sure where to insert this.
>> >>
>> >> As a first step, I would ensure that paging catches stack overflow errors.
>> >
>> >
>> > I understand that any thread-stack protection mechanism has to ensure 
>> > against a stack overflow error, adding this to my plan would mean that I 
>> > have to re-adjust my goals, one of them being support for typed-memory 
>> > objects. I would request your opinion as to what features are more 
>> > valuable to the community so I can focus on them.
>> >
>>
>> When Joel says stack overflow he means a thread writing beyond the
>> thread's stack area. I don't think this would require TYM? Usually the
>> way this is done is to add a gap between stacks when you allocate
>> them, which is a bit wasteful of physical memory.
>>
> Yes, it will not require TYM, sorry I did not make my thoughts clear. 
> Protection against stack overflow is necessary and independent of the TYM 
> interface, my concern is I do not have an estimate of the time it would take 
> to implement this. This means I will have to re-adjust my goals such as the 
> implementation of the TYM interface. If we plan on providing protection 
> against stack overflow I would request your opinion on how should I re-adjust 
> my goals so that it remains feasible for me to complete them.
>>

I think the discussion is leading you to the right progression for
your goals. It's normal for goals to shift slightly during the
project. Your focus in each phase of the project needs to be on what
are the code deliverables you can produce in that phase, and how to
prepare for the next.  It seems to me that implementing a simple
scheme is the best place to start.

>> >> Then move on to an optional capability where threads cannot access the 
>> >> stacks of other threads. POSIX does not say anything about whether that 
>> >> should work or not but there are cases (especially in RTEMS) where if a 
>> >> blocking thread has a message queue buffer on a stack and another thread 
>> >> does a send, then it will write to another thread's stack.
>> >
>> >
>> > One of the most important parts of this project will be proper 
>> > documentation to make the user understand the possible cases where 
>> > enabling this feature will cause problems and proper handling of these 
>> > issues(For all the cases the users need stack sharing, they will have to 
>> > explicitly make certain calls to share stacks )
>> >>
>> >> This is accepted programming practice.
>> >
>> >
>> >>
>> >>
>> >> I'm not opposed to an option where per-thread stack protection is 
>> >> available. But making it mandatory is a bad step. Using the MMU to detect 
>> >> stack overflow is good.
>> >
>> >
>> > Yes, the thread-stack protection will be user-configurable.
>> >>
>> >>
>> >> As Hesham mentioned, this is hard on some architectures if you don't have 
>> >> a nice page management system. Can you make sure the minimum processor 
>> >> architectural requirements are documented. Not just in an email. This 
>> >> will ultimately be information in the CPU Supplement.
>> >
>> > Noted.
>> >>
>> >>
>> >> --joel
>> >>
>> >> On Tue, May 12, 2020 at 5:02 PM Hesham Almatary 
>> >>  wrote:
>> >>>
>> >>> On Tue, 12 May 2020 at 04:57, Gedare Bloom  wrote:
>> >>> >
>> >>> > On Thu, May 7, 2020 at 9:59 PM Hesham Almatary
>> >>> >  wrote:
>> >>> > >
>> >>> > > Hello Utkarsh,
>> >>> > >
>> >>> > > I'd suggest you don't spend too much efforts on setting up BBB
>> >>> > > hardware if you haven't already. Debugging on QEMU with GDB is way
>> >>> > > easier, and you can consider either qemu-xilinx-zynq-a9 or rpi2 BSPs.
>> >>> > > Later, you can move your code to BBB if you want, since both are 
>> >>> > > based
>> >>> > > on ARMv7.
>> >>> > +1
>> >>> >
>> >>> > Past work has also used psim successfully I thought? Or am I mistaken 
>> >>> > there.
>> >>> >
>> >>> Before my 2012 project (and part of it, yes), we used psim. The use of
>> >>> software TLBs wasn't very ideal/easy though, so we moved to ARM in
>> >>> 2013. The development/testing was mainly on a RPi board. I don't
>> >>> remember there was a QEMU model for it yet.
>> >>>
>> >>>
>> >>> > >
>> >>> > > On Thu, 7 May 2020 at 18:26, Utkarsh Rai  
>> >>> > > wrote:
>> >>> > > >
>> >>> > > > Hello,
>> >>> > > > This is to ensure that all the interested parties are on the same 
>> >>> > > > page before I start my project and can give their invaluable 
>> >>> > > > feedback.
>> >>> > Excellent, thank you for getting the initiative.
>> >>> >
>> >>> > I'll be taking on the primary mentorship for your project, with
>> >>> > support from the co-mentors (Peter, Hesham, Sebastian). For now, I
>> >>> > prefer you to continue to make your presence on the mailing list. We
>>