On Tue, Sep 6, 2016 at 8:40 AM, Sebastian Huber
<sebastian.hu...@embedded-brains.de> wrote:
> Add priority nodes which contribute to the overall thread priority.
>
> The actual priority of a thread is now an aggregation of priority nodes.
> The thread priority aggregation for the home scheduler instance of a
> thread consists of at least one priority node, which is normally the
> real priority of the thread. The locking protocols (e.g. priority
> ceiling and priority inheritance), rate-monotonic period objects and the
> POSIX sporadic server add, change and remove priority nodes.
>
> A thread changes its priority now immediately, e.g. priority changes are
> not deferred until the thread releases its last resource.
>
> Replace the _Thread_Change_priority() function with
>
> * _Thread_Priority_perform_actions(),
> * _Thread_Priority_add(),
> * _Thread_Priority_remove(),
> * _Thread_Priority_change(), and
> * _Thread_Priority_update().
>
> Update #2412.
> Update #2556.
> ---
> cpukit/libmisc/capture/capture.h | 2 +-
> cpukit/libmisc/cpuuse/cpuusagetop.c | 6 +-
> cpukit/posix/include/rtems/posix/pthreadimpl.h | 2 +
> cpukit/posix/include/rtems/posix/threadsup.h | 8 +-
> cpukit/posix/src/killinfo.c | 2 +-
> cpukit/posix/src/mutexsetprioceiling.c | 13 +-
> cpukit/posix/src/pthread.c | 103 ++-
> cpukit/posix/src/pthreadcreate.c | 32 +-
> cpukit/posix/src/pthreadgetschedparam.c | 2 +-
> cpukit/posix/src/pthreadsetschedparam.c | 143 ++--
> cpukit/posix/src/pthreadsetschedprio.c | 92 +--
> cpukit/rtems/include/rtems/rtems/ratemon.h | 6 +
> cpukit/rtems/src/ratemoncancel.c | 12 +-
> cpukit/rtems/src/ratemoncreate.c | 2 +
> cpukit/rtems/src/ratemonperiod.c | 15 +-
> cpukit/rtems/src/semsetpriority.c | 18 +-
> cpukit/rtems/src/tasksetpriority.c | 95 +--
> cpukit/score/Makefile.am | 3 +-
> cpukit/score/include/rtems/score/coremutex.h | 4 +-
> cpukit/score/include/rtems/score/coremuteximpl.h | 148 ++--
> cpukit/score/include/rtems/score/mrsp.h | 12 +-
> cpukit/score/include/rtems/score/mrspimpl.h | 255 ++++---
> cpukit/score/include/rtems/score/priority.h | 159 +++-
> cpukit/score/include/rtems/score/priorityimpl.h | 417 ++++++++++
> cpukit/score/include/rtems/score/scheduler.h | 28 +-
> cpukit/score/include/rtems/score/schedulercbs.h | 17 +-
> cpukit/score/include/rtems/score/scheduleredf.h | 26 +-
> .../score/include/rtems/score/scheduleredfimpl.h | 14 +-
> cpukit/score/include/rtems/score/schedulerimpl.h | 86 ++-
> cpukit/score/include/rtems/score/schedulernode.h | 19 +-
> .../score/include/rtems/score/schedulernodeimpl.h | 18 +-
> .../score/include/rtems/score/schedulersmpimpl.h | 11 +-
> cpukit/score/include/rtems/score/thread.h | 53 +-
> cpukit/score/include/rtems/score/threadimpl.h | 261 ++++---
> cpukit/score/include/rtems/score/threadq.h | 98 ++-
> cpukit/score/include/rtems/score/threadqimpl.h | 145 ++--
> cpukit/score/preinstall.am | 4 +
> cpukit/score/src/mutex.c | 21 +-
> cpukit/score/src/schedulercbs.c | 22 +-
> cpukit/score/src/schedulercbsnodeinit.c | 1 +
> cpukit/score/src/schedulercbsreleasejob.c | 40 +-
> cpukit/score/src/schedulercbsunblock.c | 51 +-
> cpukit/score/src/schedulerdefaultnodeinit.c | 4 +-
> cpukit/score/src/schedulerdefaultreleasejob.c | 20 +-
> cpukit/score/src/scheduleredfchangepriority.c | 8 +-
> cpukit/score/src/scheduleredfnodeinit.c | 6 +-
> cpukit/score/src/scheduleredfreleasejob.c | 84 +--
> cpukit/score/src/scheduleredfunblock.c | 2 +-
> cpukit/score/src/scheduleredfyield.c | 2 +-
> cpukit/score/src/schedulerpriority.c | 2 +-
> cpukit/score/src/schedulerprioritysmp.c | 7 +-
> cpukit/score/src/schedulersimplesmp.c | 2 +-
> cpukit/score/src/schedulerstrongapa.c | 7 +-
> cpukit/score/src/thread.c | 4 +-
> cpukit/score/src/threadchangepriority.c | 362 ++++++---
> cpukit/score/src/threadinitialize.c | 7 +-
> cpukit/score/src/threadmp.c | 3 +-
> cpukit/score/src/threadqenqueue.c | 120 +--
> cpukit/score/src/threadqflush.c | 9 +-
> cpukit/score/src/threadqops.c | 839
> ++++++++++++++++-----
> cpukit/score/src/threadrestart.c | 61 +-
> cpukit/score/src/threadsetpriority.c | 59 --
> cpukit/score/src/threadtimeout.c | 3 +
> testsuites/smptests/smpmutex01/init.c | 3 +-
> testsuites/smptests/smpscheduler03/init.c | 43 +-
> testsuites/sptests/spmutex01/init.c | 141 +++-
> testsuites/sptests/spsem03/init.c | 30 +-
> testsuites/sptests/spsem03/spsem03.doc | 4 +-
> 68 files changed, 2790 insertions(+), 1508 deletions(-)
> create mode 100644 cpukit/score/include/rtems/score/priorityimpl.h
> delete mode 100644 cpukit/score/src/threadsetpriority.c
>
...
> diff --git a/cpukit/score/include/rtems/score/priority.h
> b/cpukit/score/include/rtems/score/priority.h
> index 842f017..f53c321 100644
> --- a/cpukit/score/include/rtems/score/priority.h
> +++ b/cpukit/score/include/rtems/score/priority.h
> @@ -1,17 +1,15 @@
> /**
> - * @file rtems/score/priority.h
> + * @file
> *
> - * @brief Thread Priority Manipulation Routines
> - *
> - * This include file contains all thread priority manipulation routines.
> - * This Handler provides mechanisms which can be used to
> - * initialize and manipulate thread priorities.
> + * @brief Priority Handler API
> */
>
> /*
> * COPYRIGHT (c) 1989-2011.
> * On-Line Applications Research Corporation (OAR).
> *
> + * Copyright (c) 2016 embedded brains GmbH.
> + *
> * The license and distribution terms for this file may be
> * found in the file LICENSE in this distribution or at
> * http://www.rtems.org/license/LICENSE.
> @@ -20,58 +18,153 @@
> #ifndef _RTEMS_SCORE_PRIORITY_H
> #define _RTEMS_SCORE_PRIORITY_H
>
> -/**
> - * @defgroup ScorePriority Priority Handler
> - *
> - * @ingroup Score
> - *
> - * This handler encapsulates functionality which is used to manage
> - * thread priorities. At the SuperCore level 256 priority levels
> - * are supported with lower numbers representing logically more important
> - * threads. The priority level 0 is reserved for internal RTEMS use.
> - * Typically it is assigned to threads which defer internal RTEMS
> - * actions from an interrupt to thread level to improve interrupt response.
> - * Priority level 255 is assigned to the IDLE thread and really should not
> - * be used by application threads. The default IDLE thread implementation
> - * is an infinite "branch to self" loop which never yields to other threads
> - * at the same priority.
> - */
> -/**@{*/
> -
> -/*
> - * Processor specific information.
> - */
> +#include <rtems/score/chain.h>
> #include <rtems/score/cpu.h>
> +#include <rtems/score/rbtree.h>
> +
> +struct Scheduler_Control;
>
> #ifdef __cplusplus
> extern "C" {
> #endif
>
> /**
> - * The following type defines the control block used to manage
> - * thread priorities.
> + * @defgroup ScorePriority Priority Handler
> + *
> + * @ingroup Score
> *
> - * @note Priority 0 is reserved for internal threads only.
> + * This handler encapsulates functionality which is used to manage thread
> + * priorities. The actual priority of a thread is an aggregation of priority
> + * nodes. The thread priority aggregation for the home scheduler instance
> of a
> + * thread consists of at least one priority node, which is normally the real
> + * priority of the thread. The locking protocols (e.g. priority ceiling and
> + * priority inheritance), rate-monotonic period objects and the POSIX
> sporadic
> + * server add, change and remove priority nodes.
> + *
> + * @{
> + */
> +
> +/**
> + * @brief A plain thread priority value.
> + *
> + * Lower values represent higher priorities. So, a priority value of zero
> + * represents the highest priority thread. This value is reserved for
> internal
> + * threads and the priority ceiling protocol.
> */
> typedef uint64_t Priority_Control;
>
> -/** This defines the highest (most important) thread priority. */
> +/**
> + * @brief The highest (most important) thread priority value.
> + */
> #define PRIORITY_MINIMUM 0
>
> /**
> - * @brief This defines the priority of pseudo-ISR threads.
> + * @brief The priority value of pseudo-ISR threads.
> *
> * Examples are the MPCI and timer server threads.
> */
> #define PRIORITY_PSEUDO_ISR PRIORITY_MINIMUM
>
> -/** This defines the default lowest (least important) thread priority. */
> +/**
> + * @brief The default lowest (least important) thread priority value.
> + *
> + * This value is CPU port dependent.
> + */
> #if defined (CPU_PRIORITY_MAXIMUM)
> #define PRIORITY_DEFAULT_MAXIMUM CPU_PRIORITY_MAXIMUM
> #else
> #define PRIORITY_DEFAULT_MAXIMUM 255
> #endif
>
> +/**
> + * @brief The priority node to build up a priority aggregation.
> + */
> +typedef struct {
> + /**
> + * @brief Node component for a chain or red-black tree.
> + */
> + union {
> + Chain_Node Chain;
> + RBTree_Node RBTree;
> + } Node;
> +
> + /**
> + * @brief The priority value of this node.
> + */
> + Priority_Control priority;
> +} Priority_Node;
> +
> +/**
> + * @brief The priority action type.
> + */
> +typedef enum {
> + PRIORITY_ACTION_ADD,
> + PRIORITY_ACTION_CHANGE,
> + PRIORITY_ACTION_REMOVE,
> + PRIORITY_ACTION_INVALID
> +} Priority_Action_type;
> +
> +typedef struct Priority_Aggregation Priority_Aggregation;
> +
> +/**
> + * @brief The priority aggregation.
> + */
> +struct Priority_Aggregation {
> + /**
> + * @brief A priority node reflection the overall priority of the
> aggregation.
> + *
reflection -> "reflection of" or maybe "reflects"? Or, "represents"
> + * May be used to add this aggregation to another aggregation to build up a
> + * recursive priority scheme.
> + */
> + Priority_Node Node;
> +
> + /**
> + * @brief A red-black tree to contain priority nodes contributing to the
> + * overall priority of the this priority aggregation.
"the this" -> "this"
> + */
> + RBTree_Control Contributors;
> +
Is it the case that a priority node is only ever on one of these
"Contributors" trees at a time?
> +#if defined(RTEMS_SMP)
> + /**
> + * @brief The scheduler instance of this priority aggregation.
> + */
> + const struct Scheduler_Control *scheduler;
> +#endif
> +
> + /**
> + * @brief A priority action block to manage priority node additions,
> changes
> + * and removals.
> + */
> + struct {
> +#if defined(RTEMS_SMP)
> + /**
> + * @brief The next priority aggregation in the action chain.
> + */
> + Priority_Aggregation *next;
> +#endif
> +
> + /**
> + * @brief The priority node of the action.
> + */
> + Priority_Node *node;
> +
> + /**
> + * @brief The type of the action.
> + */
> + Priority_Action_type type;
> + } Action;
> +};
> +
> +/**
> + * @brief A set of priority actions.
> + */
> +typedef struct {
> + /**
> + * @brief The first action of a priority action chain.
I would prefer consistently to refer to either actions in the priority
action chain, or priority aggregations in the action chain. It should
be clarified somewhere in the doxygen that a Priority_Aggregation can
be on a singly-linked list of aggregations for (pending?) priority
actions, and also on a red-black tree for recursive priority
dependencies.
It may be a bit confusing to refer to something as a chain that is not
making use of the "RTEMS Chains"
> + */
> + Priority_Aggregation *actions;
> +} Priority_Actions;
> +
> #ifdef __cplusplus
> }
> #endif
> diff --git a/cpukit/score/include/rtems/score/priorityimpl.h
> b/cpukit/score/include/rtems/score/priorityimpl.h
> new file mode 100644
> index 0000000..d022c1f
> --- /dev/null
> +++ b/cpukit/score/include/rtems/score/priorityimpl.h
> @@ -0,0 +1,417 @@
> +/*
> + * Copyright (c) 2016 embedded brains GmbH. All rights reserved.
> + *
> + * embedded brains GmbH
> + * Dornierstr. 4
> + * 82178 Puchheim
> + * Germany
> + * <rt...@embedded-brains.de>
> + *
> + * The license and distribution terms for this file may be
> + * found in the file LICENSE in this distribution or at
> + * http://www.rtems.org/license/LICENSE.
> + */
> +
> +#ifndef _RTEMS_SCORE_PRIORITYIMPL_H
> +#define _RTEMS_SCORE_PRIORITYIMPL_H
> +
> +#include <rtems/score/priority.h>
> +#include <rtems/score/scheduler.h>
> +
> +#ifdef __cplusplus
> +extern "C" {
> +#endif /* __cplusplus */
> +
> +RTEMS_INLINE_ROUTINE void _Priority_Actions_initialize_empty(
> + Priority_Actions *actions
> +)
> +{
> + actions->actions = NULL;
> +}
> +
> +RTEMS_INLINE_ROUTINE void _Priority_Actions_initialize_one(
> + Priority_Actions *actions,
> + Priority_Aggregation *aggregation,
> + Priority_Node *node,
> + Priority_Action_type type
> +)
> +{
> +#if defined(RTEMS_SMP)
> + aggregation->Action.next = NULL;
> +#endif
> + aggregation->Action.node = node;
> + aggregation->Action.type = type;
> +
> + actions->actions = aggregation;
> +}
> +
> +RTEMS_INLINE_ROUTINE bool _Priority_Actions_is_empty(
> + const Priority_Actions *actions
> +)
> +{
> + return actions->actions == NULL;
> +}
> +
> +RTEMS_INLINE_ROUTINE bool _Priority_Actions_is_valid(
> + const Priority_Aggregation *aggregation
> +)
> +{
> +#if defined(RTEMS_SMP)
> + return aggregation != NULL;
> +#else
> + (void) aggregation;
> + return false;
> +#endif
> +}
> +
> +RTEMS_INLINE_ROUTINE void _Priority_Actions_move(
> + Priority_Actions *target_actions,
> + Priority_Actions *source_actions
> +)
> +{
> + target_actions->actions = source_actions->actions;
> + source_actions->actions = NULL;
> +}
> +
> +RTEMS_INLINE_ROUTINE Priority_Aggregation *_Priority_Actions_get_first(
> + const Priority_Actions *actions
> +)
> +{
> + return actions->actions;
> +}
> +
> +RTEMS_INLINE_ROUTINE void _Priority_Actions_add(
> + Priority_Actions *actions,
> + Priority_Aggregation *aggregation
> +)
> +{
> +#if defined(RTEMS_SMP)
> + aggregation->Action.next = actions->actions;
Do we not care what was previously in the .next field?
[...]
> diff --git a/cpukit/score/include/rtems/score/threadq.h
> b/cpukit/score/include/rtems/score/threadq.h
> index a4ad082..058603b 100644
> --- a/cpukit/score/include/rtems/score/threadq.h
> +++ b/cpukit/score/include/rtems/score/threadq.h
> @@ -47,8 +47,6 @@ typedef struct Thread_queue_Queue Thread_queue_Queue;
>
> typedef struct Thread_queue_Operations Thread_queue_Operations;
>
> -typedef struct Thread_queue_Path Thread_queue_Path;
> -
> /**
> * @brief Thread queue deadlock callout.
> *
> @@ -195,6 +193,45 @@ typedef struct {
> */
> uint64_t timeout;
>
> +#if defined(RTEMS_SMP)
> + /**
> + * @brief Representation of a thread queue path from a start thread queue
> to
> + * the terminal thread queue.
> + *
> + * The start thread queue is determined by the object on which a thread
> intends
> + * to block. The terminal thread queue is the thread queue reachable via
> + * thread queue links those owner is not blocked on a thread queue. The
> thread
those -> whose ?
> + * queue links are determined by the thread queue owner and thread wait
> queue
> + * relationships.
> + */
> + struct {
> + /**
> + * @brief The chain of thread queue links defining the thread queue path.
> + */
> + Chain_Control Links;
> +
> + /**
> + * @brief The start of a thread queue path.
> + */
> + Thread_queue_Link Start;
> + } Path;
> +#endif
> +
> + struct {
> + Priority_Actions Actions;
> +
> + /**
> + * @brief Count of threads to update the priority via
> + * _Thread_Priority_update().
> + */
> + size_t update_count;
I'd prefer uint32_t for unsigned counters since usually size_t denotes
an object's (byte) size.
> +
> + /**
> + * @brief Threads to update the priority via _Thread_Priority_update().
> + */
> + Thread_Control *update[ 2 ];
Why 2?
> + } Priority;
> +
> /**
> * @brief Invoked in case of a detected deadlock.
> *
> @@ -237,7 +274,7 @@ typedef struct {
> /**
> * @brief The actual thread priority queue.
> */
> - RBTree_Control Queue;
> + Priority_Aggregation Queue;
> } Thread_queue_Priority_queue;
>
> /**
> @@ -289,6 +326,11 @@ typedef struct _Thread_queue_Heads {
>
> #if defined(RTEMS_SMP)
> /**
> + * @brief Boost priority.
This note could be expanded (e.g. boosted by xxx). Thanks for
improving the doxy along the way!
> + */
> + Priority_Node Boost_priority;
> +
> + /**
> * @brief One priority queue per scheduler instance.
> */
> Thread_queue_Priority_queue Priority[ RTEMS_ZERO_LENGTH_ARRAY ];
> @@ -337,34 +379,33 @@ struct Thread_queue_Queue {
> };
>
> /**
> - * @brief Thread queue priority change operation.
> + * @brief Thread queue action operation.
> *
> * @param[in] queue The actual thread queue.
> * @param[in] the_thread The thread.
> - * @param[in] new_priority The new priority value.
> - *
> - * @see Thread_queue_Operations.
> + * @param[in] queue_context The thread queue context providing the thread
> queue
> + * action set to perform. Returns the thread queue action set to perform
> on
> + * the thread queue owner or the empty set in case there is nothing to do.
> */
> -typedef void ( *Thread_queue_Priority_change_operation )(
> - Thread_queue_Queue *queue,
> - Thread_Control *the_thread,
> - Priority_Control new_priority
> +typedef void ( *Thread_queue_Priority_actions_operation )(
> + Thread_queue_Queue *queue,
> + Priority_Actions *priority_actions
> );
>
> /**
> * @brief Thread queue enqueue operation.
> *
> * A potential thread to update the priority due to priority inheritance is
> - * returned via the thread queue path. This thread is handed over to
> - * _Thread_Update_priority().
> + * returned via the thread queue context. This thread is handed over to
> + * _Thread_Priority_update().
> *
> * @param[in] queue The actual thread queue.
> * @param[in] the_thread The thread to enqueue on the queue.
> */
> typedef void ( *Thread_queue_Enqueue_operation )(
> - Thread_queue_Queue *queue,
> - Thread_Control *the_thread,
> - Thread_queue_Path *path
> + Thread_queue_Queue *queue,
> + Thread_Control *the_thread,
> + Thread_queue_Context *queue_context
> );
>
> /**
> @@ -374,8 +415,9 @@ typedef void ( *Thread_queue_Enqueue_operation )(
> * @param[in] the_thread The thread to extract from the thread queue.
> */
> typedef void ( *Thread_queue_Extract_operation )(
> - Thread_queue_Queue *queue,
> - Thread_Control *the_thread
> + Thread_queue_Queue *queue,
> + Thread_Control *the_thread,
> + Thread_queue_Context *queue_context
> );
>
> /**
> @@ -390,9 +432,10 @@ typedef void ( *Thread_queue_Extract_operation )(
> * @return The previous first thread on the queue.
> */
> typedef Thread_Control *( *Thread_queue_Surrender_operation )(
> - Thread_queue_Queue *queue,
> - Thread_queue_Heads *heads,
> - Thread_Control *previous_owner
> + Thread_queue_Queue *queue,
> + Thread_queue_Heads *heads,
> + Thread_Control *previous_owner,
> + Thread_queue_Context *queue_context
> );
>
> /**
> @@ -415,16 +458,9 @@ typedef Thread_Control *( *Thread_queue_First_operation
> )(
> */
> struct Thread_queue_Operations {
> /**
> - * @brief Thread queue priority change operation.
> - *
> - * Called by _Thread_Change_priority() to notify a thread about a priority
> - * change. In case this thread waits currently for a resource the handler
> - * may adjust its data structures according to the new priority value.
> This
> - * handler must not be NULL, instead the default handler
> - * _Thread_Do_nothing_priority_change() should be used in case nothing
> needs
> - * to be done during a priority change.
> - */
> - Thread_queue_Priority_change_operation priority_change;
> + * @brief Thread queue priority actions operation.
This note also should be expanded a bit.
> + */
> + Thread_queue_Priority_actions_operation priority_actions;
>
> /**
> * @brief Thread queue enqueue operation.
> diff --git a/cpukit/score/include/rtems/score/threadqimpl.h
> b/cpukit/score/include/rtems/score/threadqimpl.h
> index 977b0ce..3555b7f 100644
> --- a/cpukit/score/include/rtems/score/threadqimpl.h
> +++ b/cpukit/score/include/rtems/score/threadqimpl.h
> @@ -21,7 +21,7 @@
>
> #include <rtems/score/threadq.h>
> #include <rtems/score/chainimpl.h>
> -#include <rtems/score/rbtreeimpl.h>
> +#include <rtems/score/priorityimpl.h>
> #include <rtems/score/scheduler.h>
> #include <rtems/score/smp.h>
> #include <rtems/score/thread.h>
> @@ -39,38 +39,8 @@ extern "C" {
> */
> /**@{*/
>
> -/**
> - * @brief Representation of a thread queue path from a start thread queue to
> - * the terminal thread queue.
> - *
> - * The start thread queue is determined by the object on which a thread
> intends
> - * to block. The terminal thread queue is the thread queue reachable via
> - * thread queue links those owner is not blocked on a thread queue. The
> thread
> - * queue links are determined by the thread queue owner and thread wait queue
> - * relationships.
> - */
> -struct Thread_queue_Path {
> -#if defined(RTEMS_SMP)
> - /**
> - * @brief The chain of thread queue links defining the thread queue path.
> - */
> - Chain_Control Links;
> -
> - /**
> - * @brief The start of a thread queue path.
> - */
> - Thread_queue_Link Start;
> -#endif
> -
> - /**
> - * @brief A potential thread to update the priority via
> - * _Thread_Update_priority().
> - *
> - * This thread is determined by thread queues which support priority
> - * inheritance.
> - */
> - Thread_Control *update_priority;
> -};
> +#define THREAD_QUEUE_LINK_OF_PATH_NODE( node ) \
> + RTEMS_CONTAINER_OF( node, Thread_queue_Link, Path_node );
>
> /**
> * @brief Thread queue with a layout compatible to struct _Thread_queue_Queue
> @@ -210,6 +180,42 @@ RTEMS_INLINE_ROUTINE void
> _Thread_queue_Context_set_deadlock_callout(
> queue_context->deadlock_callout = deadlock_callout;
> }
>
> +RTEMS_INLINE_ROUTINE void _Thread_queue_Context_clear_priority_updates(
> + Thread_queue_Context *queue_context
> +)
> +{
> + queue_context->Priority.update_count = 0;
> +}
> +
> +RTEMS_INLINE_ROUTINE size_t _Thread_queue_Context_get_priority_updates(
> + Thread_queue_Context *queue_context
> +)
> +{
> + return queue_context->Priority.update_count;
> +}
> +
> +RTEMS_INLINE_ROUTINE void _Thread_queue_Context_set_priority_updates(
> + Thread_queue_Context *queue_context,
> + size_t update_count
> +)
> +{
> + queue_context->Priority.update_count = update_count;
> +}
Are there some rules for what can/should set the priority updates? Is
this a monotonic counter until clear()?
> +
> +RTEMS_INLINE_ROUTINE void _Thread_queue_Context_add_priority_update(
> + Thread_queue_Context *queue_context,
> + Thread_Control *the_thread
> +)
> +{
> + size_t n;
> +
> + n = queue_context->Priority.update_count;
> + _Assert( n < RTEMS_ARRAY_SIZE( queue_context->Priority.update ) );
> +
> + queue_context->Priority.update_count = n + 1;
These two assignments could make use of the above _get/set_priority_updates().
> + queue_context->Priority.update[ n ] = the_thread;
> +}
> +
> /**
> * @brief Sets the MP callout in the thread queue context.
> *
> @@ -274,9 +280,12 @@ RTEMS_INLINE_ROUTINE void _Thread_queue_Heads_initialize(
> #if defined(RTEMS_SMP)
> size_t i;
>
> + _Priority_Node_initialize( &heads->Boost_priority, 0 );
> + _Priority_Node_set_inactive( &heads->Boost_priority );
> +
> for ( i = 0; i < _Scheduler_Count; ++i ) {
> _Chain_Initialize_node( &heads->Priority[ i ].Node );
> - _RBTree_Initialize_empty( &heads->Priority[ i ].Queue );
> + _Priority_Initialize_empty( &heads->Priority[ i ].Queue );
> }
> #endif
>
> @@ -579,16 +588,6 @@ RTEMS_INLINE_ROUTINE void _Thread_queue_Enqueue(
> );
> }
>
> -bool _Thread_queue_Do_extract_locked(
> - Thread_queue_Queue *queue,
> - const Thread_queue_Operations *operations,
> - Thread_Control *the_thread
> -#if defined(RTEMS_MULTIPROCESSING)
> - ,
> - const Thread_queue_Context *queue_context
> -#endif
> -);
> -
> /**
> * @brief Extracts the thread from the thread queue, restores the default
> wait
> * operations and restores the default thread lock.
> @@ -599,8 +598,7 @@ bool _Thread_queue_Do_extract_locked(
> * @param[in] queue The actual thread queue.
> * @param[in] operations The thread queue operations.
> * @param[in] the_thread The thread to extract.
> - * @param[in] queue_context The thread queue context. This parameter is only
> - * used on multiprocessing configurations.
> + * @param[in] queue_context The thread queue context.
> *
> * @return Returns the unblock indicator for
> _Thread_queue_Unblock_critical().
> * True indicates, that this thread must be unblocked by the scheduler later
> in
> @@ -610,32 +608,12 @@ bool _Thread_queue_Do_extract_locked(
> * since this thread may already block on another resource in an SMP
> * configuration.
> */
> -#if defined(RTEMS_MULTIPROCESSING)
> - #define _Thread_queue_Extract_locked( \
> - unblock, \
> - queue, \
> - the_thread, \
> - queue_context \
> - ) \
> - _Thread_queue_Do_extract_locked( \
> - unblock, \
> - queue, \
> - the_thread, \
> - queue_context \
> - )
> -#else
> - #define _Thread_queue_Extract_locked( \
> - unblock, \
> - queue, \
> - the_thread, \
> - queue_context \
> - ) \
> - _Thread_queue_Do_extract_locked( \
> - unblock, \
> - queue, \
> - the_thread \
> - )
> -#endif
> +bool _Thread_queue_Extract_locked(
> + Thread_queue_Queue *queue,
> + const Thread_queue_Operations *operations,
> + Thread_Control *the_thread,
> + const Thread_queue_Context *queue_context
> +);
>
> /**
> * @brief Unblocks the thread which was on the thread queue before.
> @@ -735,7 +713,7 @@ void _Thread_queue_Extract_with_proxy(
>
> /**
> * @brief Surrenders the thread queue previously owned by the thread to the
> - * first enqueued thread if it exists.
> + * first enqueued thread.
> *
> * The owner of the thread queue must be set to NULL by the caller.
> *
> @@ -743,21 +721,18 @@ void _Thread_queue_Extract_with_proxy(
> * thread dispatch if necessary.
> *
> * @param[in] queue The actual thread queue.
> - * @param[in] operations The thread queue operations.
> - * @param[in] heads The thread queue heads.
> + * @param[in] heads The thread queue heads. It must not be NULL.
> * @param[in] previous_owner The previous owner thread surrendering the
> thread
> * queue.
> - * @param[in] keep_priority Indicates if the previous owner thread should
> keep
> - * its current priority.
> * @param[in] queue_context The thread queue context of the lock acquire.
> + * @param[in] operations The thread queue operations.
> */
> void _Thread_queue_Surrender(
> Thread_queue_Queue *queue,
> - const Thread_queue_Operations *operations,
> Thread_queue_Heads *heads,
> Thread_Control *previous_owner,
> - bool keep_priority,
> - Thread_queue_Context *queue_context
> + Thread_queue_Context *queue_context,
> + const Thread_queue_Operations *operations
> );
>
> RTEMS_INLINE_ROUTINE bool _Thread_queue_Is_empty(
> @@ -980,6 +955,16 @@ void _Thread_queue_Unblock_proxy(
> );
> #endif
>
> +bool _Thread_queue_Path_acquire_critical(
> + Thread_queue_Queue *queue,
> + Thread_Control *the_thread,
> + Thread_queue_Context *queue_context
> +);
> +
> +void _Thread_queue_Path_release_critical(
> + Thread_queue_Context *queue_context
> +);
> +
> /**
> * @brief Helper structure to ensure that all objects containing a thread
> queue
> * have the right layout.
> diff --git a/cpukit/score/preinstall.am b/cpukit/score/preinstall.am
> index 61d44b8..94865a0 100644
> --- a/cpukit/score/preinstall.am
> +++ b/cpukit/score/preinstall.am
> @@ -200,6 +200,10 @@ $(PROJECT_INCLUDE)/rtems/score/priority.h:
> include/rtems/score/priority.h $(PROJ
> $(INSTALL_DATA) $< $(PROJECT_INCLUDE)/rtems/score/priority.h
> PREINSTALL_FILES += $(PROJECT_INCLUDE)/rtems/score/priority.h
>
> +$(PROJECT_INCLUDE)/rtems/score/priorityimpl.h:
> include/rtems/score/priorityimpl.h $(PROJECT_INCLUDE)/rtems/score/$(dirstamp)
> + $(INSTALL_DATA) $< $(PROJECT_INCLUDE)/rtems/score/priorityimpl.h
> +PREINSTALL_FILES += $(PROJECT_INCLUDE)/rtems/score/priorityimpl.h
> +
> $(PROJECT_INCLUDE)/rtems/score/prioritybitmap.h:
> include/rtems/score/prioritybitmap.h
> $(PROJECT_INCLUDE)/rtems/score/$(dirstamp)
> $(INSTALL_DATA) $< $(PROJECT_INCLUDE)/rtems/score/prioritybitmap.h
> PREINSTALL_FILES += $(PROJECT_INCLUDE)/rtems/score/prioritybitmap.h
> diff --git a/cpukit/score/src/mutex.c b/cpukit/score/src/mutex.c
> index bfa36ff..daa90a5 100644
> --- a/cpukit/score/src/mutex.c
> +++ b/cpukit/score/src/mutex.c
> @@ -128,33 +128,20 @@ static void _Mutex_Release_critical(
> )
> {
> Thread_queue_Heads *heads;
> - bool keep_priority;
>
> + heads = mutex->Queue.Queue.heads;
> mutex->Queue.Queue.owner = NULL;
> -
> --executing->resource_count;
>
> - /*
> - * Ensure that the owner resource count is visible to all other
> - * processors and that we read the latest priority restore
> - * hint.
> - */
> - _Atomic_Fence( ATOMIC_ORDER_ACQ_REL );
> -
> - heads = mutex->Queue.Queue.heads;
> - keep_priority = _Thread_Owns_resources( executing )
> - || !executing->priority_restore_hint;
> -
> - if ( __predict_true( heads == NULL && keep_priority ) ) {
> + if ( __predict_true( heads == NULL ) ) {
> _Mutex_Queue_release( mutex, queue_context );
> } else {
> _Thread_queue_Surrender(
> &mutex->Queue.Queue,
> - MUTEX_TQ_OPERATIONS,
> heads,
> executing,
> - keep_priority,
> - queue_context
> + queue_context,
> + MUTEX_TQ_OPERATIONS
> );
> }
> }
> diff --git a/cpukit/score/src/schedulercbs.c b/cpukit/score/src/schedulercbs.c
> index 98ec0eb..f114d4f 100644
> --- a/cpukit/score/src/schedulercbs.c
> +++ b/cpukit/score/src/schedulercbs.c
> @@ -19,24 +19,28 @@
> #endif
>
> #include <rtems/score/schedulercbsimpl.h>
> -#include <rtems/score/threadimpl.h>
> -#include <rtems/score/wkspace.h>
>
> void _Scheduler_CBS_Budget_callout(
> Thread_Control *the_thread
> )
> {
> - Priority_Control new_priority;
> - Priority_Control unused;
> - Scheduler_CBS_Node *node;
> - Scheduler_CBS_Server_id server_id;
> + Scheduler_CBS_Node *node;
> + Scheduler_CBS_Server_id server_id;
> + Thread_queue_Context queue_context;
> +
> + node = _Scheduler_CBS_Thread_get_node( the_thread );
>
> /* Put violating task to background until the end of period. */
> - new_priority = the_thread->Start.initial_priority;
> - _Thread_Set_priority( the_thread, new_priority, &unused, true );
> + _Thread_queue_Context_clear_priority_updates( &queue_context );
> + _Scheduler_CBS_Cancel_job(
> + NULL,
> + the_thread,
> + node->deadline_node,
> + &queue_context
> + );
> + _Thread_Priority_update( &queue_context );
>
> /* Invoke callback function if any. */
> - node = _Scheduler_CBS_Thread_get_node( the_thread );
> if ( node->cbs_server->cbs_budget_overrun ) {
> _Scheduler_CBS_Get_server_id(
> node->cbs_server->task_id,
> diff --git a/cpukit/score/src/schedulercbsnodeinit.c
> b/cpukit/score/src/schedulercbsnodeinit.c
> index 5380069..89b6f8e 100644
> --- a/cpukit/score/src/schedulercbsnodeinit.c
> +++ b/cpukit/score/src/schedulercbsnodeinit.c
> @@ -33,4 +33,5 @@ void _Scheduler_CBS_Node_initialize(
>
> the_node = _Scheduler_CBS_Node_downcast( node );
> the_node->cbs_server = NULL;
> + the_node->deadline_node = NULL;
> }
> diff --git a/cpukit/score/src/schedulercbsreleasejob.c
> b/cpukit/score/src/schedulercbsreleasejob.c
> index d2169af..186f95c 100644
> --- a/cpukit/score/src/schedulercbsreleasejob.c
> +++ b/cpukit/score/src/schedulercbsreleasejob.c
> @@ -21,10 +21,12 @@
>
> #include <rtems/score/schedulercbsimpl.h>
>
> -Thread_Control *_Scheduler_CBS_Release_job(
> +void _Scheduler_CBS_Release_job(
> const Scheduler_Control *scheduler,
> Thread_Control *the_thread,
> - uint64_t deadline
> + Priority_Node *priority_node,
> + uint64_t deadline,
> + Thread_queue_Context *queue_context
> )
> {
> Scheduler_CBS_Node *node;
> @@ -38,5 +40,37 @@ Thread_Control *_Scheduler_CBS_Release_job(
> the_thread->cpu_time_budget = serv_info->parameters.budget;
> }
>
> - return _Scheduler_EDF_Release_job( scheduler, the_thread, deadline );
> + node->deadline_node = priority_node;
> +
> + _Scheduler_EDF_Release_job(
> + scheduler,
> + the_thread,
> + priority_node,
> + deadline,
> + queue_context
> + );
> +}
> +
> +void _Scheduler_CBS_Cancel_job(
> + const Scheduler_Control *scheduler,
> + Thread_Control *the_thread,
> + Priority_Node *priority_node,
> + Thread_queue_Context *queue_context
> +)
> +{
> + Scheduler_CBS_Node *node;
> +
> + node = _Scheduler_CBS_Thread_get_node( the_thread );
> +
> + if ( node->deadline_node != NULL ) {
> + _Assert( node->deadline_node == priority_node );
> + node->deadline_node = NULL;
> +
> + _Scheduler_EDF_Cancel_job(
> + scheduler,
> + the_thread,
> + priority_node,
> + queue_context
> + );
> + }
> }
> diff --git a/cpukit/score/src/schedulercbsunblock.c
> b/cpukit/score/src/schedulercbsunblock.c
> index 0c1e48e..c09f471 100644
> --- a/cpukit/score/src/schedulercbsunblock.c
> +++ b/cpukit/score/src/schedulercbsunblock.c
> @@ -30,13 +30,11 @@ Scheduler_Void_or_thread _Scheduler_CBS_Unblock(
> Thread_Control *the_thread
> )
> {
> - Scheduler_EDF_Context *context;
> - Scheduler_CBS_Node *node;
> - Scheduler_CBS_Server *serv_info;
> - Priority_Control priority;
> - bool prepend_it;
> + Scheduler_CBS_Node *node;
> + Scheduler_CBS_Server *serv_info;
> + Priority_Control priority;
> + bool prepend_it;
>
> - context = _Scheduler_EDF_Get_context( scheduler );
> node = _Scheduler_CBS_Thread_get_node( the_thread );
> serv_info = node->cbs_server;
> priority = _Scheduler_Node_get_priority( &node->Base.Base, &prepend_it );
> @@ -55,40 +53,19 @@ Scheduler_Void_or_thread _Scheduler_CBS_Unblock(
> Priority_Control budget_left = priority - _Watchdog_Ticks_since_boot;
>
> if ( deadline * budget_left > budget * deadline_left ) {
> - /* Put late unblocked task to background until the end of period. */
> -
> - priority = node->Base.background_priority;
> - the_thread->real_priority = priority;
> + Thread_queue_Context queue_context;
>
> - if (
> - _Thread_Priority_less_than(
> - _Thread_Get_priority( the_thread ),
> - priority
> - ) || !_Thread_Owns_resources( the_thread )
> - ) {
> - the_thread->current_priority = priority;
> - }
> + /* Put late unblocked task to background until the end of period. */
> + _Thread_queue_Context_clear_priority_updates( &queue_context );
> + _Scheduler_CBS_Cancel_job(
> + scheduler,
> + the_thread,
> + node->deadline_node,
> + &queue_context
> + );
> }
> }
>
> - node->Base.current_priority = priority;
> - _Scheduler_EDF_Enqueue( context, &node->Base, priority );
> -
> - /*
> - * If the thread that was unblocked is more important than the heir,
> - * then we have a new heir. This may or may not result in a
> - * context switch.
> - *
> - * Normal case:
> - * If the current thread is preemptible, then we need to do
> - * a context switch.
> - * Pseudo-ISR case:
> - * Even if the thread isn't preemptible, if the new heir is
> - * a pseudo-ISR system task, we need to do a context switch.
> - */
> - if ( priority < _Thread_Get_priority( _Thread_Heir ) ) {
> - _Scheduler_Update_heir( the_thread, priority == PRIORITY_PSEUDO_ISR );
> - }
> -
> + _Scheduler_EDF_Unblock( scheduler, the_thread );
> SCHEDULER_RETURN_VOID_OR_NULL;
> }
> diff --git a/cpukit/score/src/schedulerdefaultnodeinit.c
> b/cpukit/score/src/schedulerdefaultnodeinit.c
> index 10e71f8..53aed52 100644
> --- a/cpukit/score/src/schedulerdefaultnodeinit.c
> +++ b/cpukit/score/src/schedulerdefaultnodeinit.c
> @@ -28,7 +28,5 @@ void _Scheduler_default_Node_initialize(
> Priority_Control priority
> )
> {
> - (void) scheduler;
> -
> - _Scheduler_Node_do_initialize( node, the_thread, priority );
> + _Scheduler_Node_do_initialize( scheduler, node, the_thread, priority );
> }
> diff --git a/cpukit/score/src/schedulerdefaultreleasejob.c
> b/cpukit/score/src/schedulerdefaultreleasejob.c
> index 7272fc1..490d58b 100644
> --- a/cpukit/score/src/schedulerdefaultreleasejob.c
> +++ b/cpukit/score/src/schedulerdefaultreleasejob.c
> @@ -21,26 +21,30 @@
>
> #include <rtems/score/scheduler.h>
>
> -Thread_Control *_Scheduler_default_Release_job(
> +void _Scheduler_default_Release_job(
> const Scheduler_Control *scheduler,
> Thread_Control *the_thread,
> - uint64_t deadline
> + Priority_Node *priority_node,
> + uint64_t deadline,
> + Thread_queue_Context *queue_context
> )
> {
> (void) scheduler;
> (void) the_thread;
> + (void) priority_node;
> (void) deadline;
> -
> - return NULL;
> + (void) queue_context;
> }
>
> -Thread_Control *_Scheduler_default_Cancel_job(
> +void _Scheduler_default_Cancel_job(
> const Scheduler_Control *scheduler,
> - Thread_Control *the_thread
> + Thread_Control *the_thread,
> + Priority_Node *priority_node,
> + Thread_queue_Context *queue_context
> )
> {
> (void) scheduler;
> (void) the_thread;
> -
> - return NULL;
> + (void) priority_node;
> + (void) queue_context;
> }
> diff --git a/cpukit/score/src/scheduleredfchangepriority.c
> b/cpukit/score/src/scheduleredfchangepriority.c
> index 9a73128..8940b1d 100644
> --- a/cpukit/score/src/scheduleredfchangepriority.c
> +++ b/cpukit/score/src/scheduleredfchangepriority.c
> @@ -54,16 +54,12 @@ Scheduler_Void_or_thread _Scheduler_EDF_Update_priority(
> node = _Scheduler_EDF_Thread_get_node( the_thread );
> priority = _Scheduler_Node_get_priority( &node->Base, &prepend_it );
>
> - if ( priority == node->current_priority ) {
> + if ( priority == node->priority ) {
> /* Nothing to do */
> SCHEDULER_RETURN_VOID_OR_NULL;
> }
>
> - if ( ( priority & SCHEDULER_EDF_PRIO_MSB ) != 0 ) {
> - node->background_priority = priority;
> - }
> -
> - node->current_priority = priority;
> + node->priority = priority;
> context = _Scheduler_EDF_Get_context( scheduler );
>
> _Scheduler_EDF_Extract( context, node );
> diff --git a/cpukit/score/src/scheduleredfnodeinit.c
> b/cpukit/score/src/scheduleredfnodeinit.c
> index d290bd7..94f8fac 100644
> --- a/cpukit/score/src/scheduleredfnodeinit.c
> +++ b/cpukit/score/src/scheduleredfnodeinit.c
> @@ -29,11 +29,9 @@ void _Scheduler_EDF_Node_initialize(
> {
> Scheduler_EDF_Node *the_node;
>
> - (void) scheduler;
> -
> - _Scheduler_Node_do_initialize( node, the_thread, priority );
> + _Scheduler_Node_do_initialize( scheduler, node, the_thread, priority );
>
> the_node = _Scheduler_EDF_Node_downcast( node );
> - the_node->thread = the_thread;
> _RBTree_Initialize_node( &the_node->Node );
> + the_node->priority = priority;
> }
> diff --git a/cpukit/score/src/scheduleredfreleasejob.c
> b/cpukit/score/src/scheduleredfreleasejob.c
> index 4c74c48..c19d9b9 100644
> --- a/cpukit/score/src/scheduleredfreleasejob.c
> +++ b/cpukit/score/src/scheduleredfreleasejob.c
> @@ -20,75 +20,47 @@
>
> #include <rtems/score/scheduleredfimpl.h>
>
> -static bool _Scheduler_EDF_Release_job_filter(
> - Thread_Control *the_thread,
> - Priority_Control *new_priority_p,
> - void *arg
> +void _Scheduler_EDF_Release_job(
> + const Scheduler_Control *scheduler,
> + Thread_Control *the_thread,
> + Priority_Node *priority_node,
> + uint64_t deadline,
> + Thread_queue_Context *queue_context
> )
> {
> - Scheduler_EDF_Node *node;
> - Priority_Control current_priority;
> - Priority_Control new_priority;
> + (void) scheduler;
>
> - node = _Scheduler_EDF_Thread_get_node( the_thread );
> + _Thread_Wait_acquire_critical( the_thread, queue_context );
>
> - current_priority = _Thread_Get_priority( the_thread );
> - new_priority = *new_priority_p;
> + _Priority_Node_set_priority( priority_node, deadline );
>
> - node->current_priority = new_priority;
> - the_thread->real_priority = new_priority;
> + if ( _Priority_Node_is_active( priority_node ) ) {
> + _Thread_Priority_changed(
> + the_thread,
> + priority_node,
> + false,
> + queue_context
> + );
> + } else {
> + _Thread_Priority_add( the_thread, priority_node, queue_context );
> + }
>
> - return _Thread_Priority_less_than( current_priority, new_priority )
> - || !_Thread_Owns_resources( the_thread );
> + _Thread_Wait_release_critical( the_thread, queue_context );
> }
>
> -Thread_Control *_Scheduler_EDF_Release_job(
> +void _Scheduler_EDF_Cancel_job(
> const Scheduler_Control *scheduler,
> Thread_Control *the_thread,
> - uint64_t deadline
> -)
> -{
> - return _Thread_Apply_priority(
> - the_thread,
> - deadline,
> - NULL,
> - _Scheduler_EDF_Release_job_filter,
> - true
> - );
> -}
> -
> -static bool _Scheduler_EDF_Cancel_job_filter(
> - Thread_Control *the_thread,
> - Priority_Control *new_priority_p,
> - void *arg
> + Priority_Node *priority_node,
> + Thread_queue_Context *queue_context
> )
> {
> - Scheduler_EDF_Node *node;
> - Priority_Control current_priority;
> - Priority_Control new_priority;
> + (void) scheduler;
>
> - node = _Scheduler_EDF_Thread_get_node( the_thread );
> + _Thread_Wait_acquire_critical( the_thread, queue_context );
>
> - current_priority = _Thread_Get_priority( the_thread );
> - new_priority = node->background_priority;
> + _Thread_Priority_remove( the_thread, priority_node, queue_context );
> + _Priority_Node_set_inactive( priority_node );
>
> - node->current_priority = new_priority;
> - the_thread->real_priority = new_priority;
> -
> - return _Thread_Priority_less_than( current_priority, new_priority )
> - || !_Thread_Owns_resources( the_thread );
> -}
> -
> -Thread_Control *_Scheduler_EDF_Cancel_job(
> - const Scheduler_Control *scheduler,
> - Thread_Control *the_thread
> -)
> -{
> - return _Thread_Apply_priority(
> - the_thread,
> - 0,
> - NULL,
> - _Scheduler_EDF_Cancel_job_filter,
> - true
> - );
> + _Thread_Wait_release_critical( the_thread, queue_context );
> }
> diff --git a/cpukit/score/src/scheduleredfunblock.c
> b/cpukit/score/src/scheduleredfunblock.c
> index 9b156ec..a5cc4b6 100644
> --- a/cpukit/score/src/scheduleredfunblock.c
> +++ b/cpukit/score/src/scheduleredfunblock.c
> @@ -37,7 +37,7 @@ Scheduler_Void_or_thread _Scheduler_EDF_Unblock(
> priority = _Scheduler_Node_get_priority( &node->Base, &prepend_it );
> (void) prepend_it;
>
> - node->current_priority = priority;
> + node->priority = priority;
> _Scheduler_EDF_Enqueue( context, node, priority );
>
> /*
> diff --git a/cpukit/score/src/scheduleredfyield.c
> b/cpukit/score/src/scheduleredfyield.c
> index 06c1b46..3e64e5c 100644
> --- a/cpukit/score/src/scheduleredfyield.c
> +++ b/cpukit/score/src/scheduleredfyield.c
> @@ -33,7 +33,7 @@ Scheduler_Void_or_thread _Scheduler_EDF_Yield(
> node = _Scheduler_EDF_Thread_get_node( the_thread );
>
> _Scheduler_EDF_Extract( context, node );
> - _Scheduler_EDF_Enqueue( context, node, node->current_priority );
> + _Scheduler_EDF_Enqueue( context, node, node->priority );
> _Scheduler_EDF_Schedule_body( scheduler, the_thread, true );
>
> SCHEDULER_RETURN_VOID_OR_NULL;
> diff --git a/cpukit/score/src/schedulerpriority.c
> b/cpukit/score/src/schedulerpriority.c
> index 11cee92..ddfd973 100644
> --- a/cpukit/score/src/schedulerpriority.c
> +++ b/cpukit/score/src/schedulerpriority.c
> @@ -43,7 +43,7 @@ void _Scheduler_priority_Node_initialize(
> Scheduler_priority_Context *context;
> Scheduler_priority_Node *the_node;
>
> - _Scheduler_Node_do_initialize( node, the_thread, priority );
> + _Scheduler_Node_do_initialize( scheduler, node, the_thread, priority );
>
> context = _Scheduler_priority_Get_context( scheduler );
> the_node = _Scheduler_priority_Node_downcast( node );
> diff --git a/cpukit/score/src/schedulerprioritysmp.c
> b/cpukit/score/src/schedulerprioritysmp.c
> index 07e7af4..e624a6a 100644
> --- a/cpukit/score/src/schedulerprioritysmp.c
> +++ b/cpukit/score/src/schedulerprioritysmp.c
> @@ -57,7 +57,12 @@ void _Scheduler_priority_SMP_Node_initialize(
> Scheduler_priority_SMP_Node *the_node;
>
> the_node = _Scheduler_priority_SMP_Node_downcast( node );
> - _Scheduler_SMP_Node_initialize( &the_node->Base, the_thread, priority );
> + _Scheduler_SMP_Node_initialize(
> + scheduler,
> + &the_node->Base,
> + the_thread,
> + priority
> + );
>
> context = _Scheduler_Get_context( scheduler );
> self = _Scheduler_priority_SMP_Get_self( context );
> diff --git a/cpukit/score/src/schedulersimplesmp.c
> b/cpukit/score/src/schedulersimplesmp.c
> index 8f86ea8..9606896 100644
> --- a/cpukit/score/src/schedulersimplesmp.c
> +++ b/cpukit/score/src/schedulersimplesmp.c
> @@ -52,7 +52,7 @@ void _Scheduler_simple_SMP_Node_initialize(
> Scheduler_SMP_Node *the_node;
>
> the_node = _Scheduler_SMP_Node_downcast( node );
> - _Scheduler_SMP_Node_initialize( the_node, the_thread, priority );
> + _Scheduler_SMP_Node_initialize( scheduler, the_node, the_thread, priority
> );
> }
>
> static void _Scheduler_simple_SMP_Do_update(
> diff --git a/cpukit/score/src/schedulerstrongapa.c
> b/cpukit/score/src/schedulerstrongapa.c
> index 5d7c7f7..fc6d012 100644
> --- a/cpukit/score/src/schedulerstrongapa.c
> +++ b/cpukit/score/src/schedulerstrongapa.c
> @@ -183,7 +183,12 @@ void _Scheduler_strong_APA_Node_initialize(
> Scheduler_strong_APA_Node *the_node;
>
> the_node = _Scheduler_strong_APA_Node_downcast( node );
> - _Scheduler_SMP_Node_initialize( &the_node->Base, the_thread, priority );
> + _Scheduler_SMP_Node_initialize(
> + scheduler,
> + &the_node->Base,
> + the_thread,
> + priority
> + );
>
> context = _Scheduler_Get_context( scheduler );
> self = _Scheduler_strong_APA_Get_self( context );
> diff --git a/cpukit/score/src/thread.c b/cpukit/score/src/thread.c
> index c569ae5..864b21b 100644
> --- a/cpukit/score/src/thread.c
> +++ b/cpukit/score/src/thread.c
> @@ -32,9 +32,7 @@
> THREAD_OFFSET_ASSERT( Object );
> THREAD_OFFSET_ASSERT( Join_queue );
> THREAD_OFFSET_ASSERT( current_state );
> -THREAD_OFFSET_ASSERT( current_priority );
> -THREAD_OFFSET_ASSERT( real_priority );
> -THREAD_OFFSET_ASSERT( priority_restore_hint );
> +THREAD_OFFSET_ASSERT( Real_priority );
> THREAD_OFFSET_ASSERT( resource_count );
> THREAD_OFFSET_ASSERT( Scheduler );
> THREAD_OFFSET_ASSERT( Wait );
> diff --git a/cpukit/score/src/threadchangepriority.c
> b/cpukit/score/src/threadchangepriority.c
> index 3429e1a..42172b2 100644
> --- a/cpukit/score/src/threadchangepriority.c
> +++ b/cpukit/score/src/threadchangepriority.c
> @@ -10,6 +10,8 @@
> * COPYRIGHT (c) 1989-2014.
> * On-Line Applications Research Corporation (OAR).
> *
> + * Copyright (c) 2013, 2016 embedded brains GmbH
> + *
> * The license and distribution terms for this file may be
> * found in the file LICENSE in this distribution or at
> * http://www.rtems.org/license/LICENSE.
> @@ -20,145 +22,313 @@
> #endif
>
> #include <rtems/score/threadimpl.h>
> +#include <rtems/score/assert.h>
> #include <rtems/score/schedulerimpl.h>
>
> -static Thread_Control *_Thread_Apply_priority_locked(
> +static void _Thread_Set_scheduler_node_priority(
> + Priority_Aggregation *priority_aggregation,
> + bool prepend_it
> +)
> +{
> + _Scheduler_Node_set_priority(
> + SCHEDULER_NODE_OF_WAIT_PRIORITY_NODE( priority_aggregation ),
> + _Priority_Get_priority( priority_aggregation ),
> + prepend_it
> + );
> +}
> +
> +#if defined(RTEMS_SMP)
> +static void _Thread_Priority_action_add(
> + Priority_Aggregation *priority_aggregation,
> + Priority_Actions *priority_actions,
> + void *arg
> +)
> +{
> + _Thread_Set_scheduler_node_priority( priority_aggregation, false );
> + _Priority_Set_action_type( priority_aggregation, PRIORITY_ACTION_ADD );
> + _Priority_Actions_add( priority_actions, priority_aggregation );
> +}
> +
> +static void _Thread_Priority_action_remove(
> + Priority_Aggregation *priority_aggregation,
> + Priority_Actions *priority_actions,
> + void *arg
> +)
> +{
> + _Thread_Set_scheduler_node_priority( priority_aggregation, true );
> + _Priority_Set_action_type( priority_aggregation, PRIORITY_ACTION_REMOVE );
> + _Priority_Actions_add( priority_actions, priority_aggregation );
> +}
> +#endif
> +
> +static void _Thread_Priority_action_change(
> + Priority_Aggregation *priority_aggregation,
> + bool prepend_it,
> + Priority_Actions *priority_actions,
> + void *arg
> +)
> +{
> + _Thread_Set_scheduler_node_priority( priority_aggregation, prepend_it );
> +#if defined(RTEMS_SMP)
> + _Priority_Set_action_type( priority_aggregation, PRIORITY_ACTION_CHANGE );
> +#elif defined(RTEMS_DEBUG)
> + _Priority_Set_action_type( priority_aggregation, PRIORITY_ACTION_INVALID );
> +#endif
> + _Priority_Actions_add( priority_actions, priority_aggregation );
> +}
> +
> +static void _Thread_Priority_do_perform_actions(
> Thread_Control *the_thread,
> - Priority_Control new_priority,
> - void *arg,
> - Thread_Change_priority_filter filter,
> + Thread_queue_Queue *queue,
> + const Thread_queue_Operations *operations,
> bool prepend_it,
> Thread_queue_Context *queue_context
> )
> {
> - /*
> - * For simplicity set the priority restore hint unconditionally since this
> is
> - * an average case optimization. Otherwise complicated atomic operations
> - * would be necessary. Synchronize with a potential read of the resource
> - * count in the filter function. See also _CORE_mutex_Surrender(),
> - * _Thread_Set_priority_filter() and _Thread_Restore_priority_filter().
> - */
> - the_thread->priority_restore_hint = true;
> - _Atomic_Fence( ATOMIC_ORDER_ACQ_REL );
> -
> - /*
> - * Do not bother recomputing all the priority related information if
> - * we are not REALLY changing priority.
> - */
> - if ( ( *filter )( the_thread, &new_priority, arg ) ) {
> - _Scheduler_Thread_set_priority( the_thread, new_priority, prepend_it );
> -
> - ( *the_thread->Wait.operations->priority_change )(
> - the_thread->Wait.queue,
> - the_thread,
> - new_priority
> + Priority_Actions actions_now;
> + Priority_Aggregation *priority_aggregation;
> +
> + _Assert( !_Priority_Actions_is_empty( &queue_context->Priority.Actions ) );
> +
> + _Priority_Actions_move( &actions_now, &queue_context->Priority.Actions );
> + priority_aggregation = _Priority_Actions_get_first( &actions_now );
> +
> + do {
> + Priority_Aggregation *priority_aggregation_next;
> + Priority_Node *priority_action_node;
> + Priority_Action_type priority_action_type;
> +
> + priority_aggregation_next = _Priority_Get_next_action(
> + priority_aggregation
> + );
> +
> + priority_action_node = priority_aggregation->Action.node;
> + priority_action_type = priority_aggregation->Action.type;
> +
> + switch ( priority_action_type ) {
> + case PRIORITY_ACTION_ADD:
> +#if defined(RTEMS_SMP)
> + _Priority_Insert(
> + priority_aggregation,
> + priority_action_node,
> + &queue_context->Priority.Actions,
> + _Thread_Priority_action_add,
> + _Thread_Priority_action_change,
> + NULL
> + );
> +#else
> + _Priority_Non_empty_insert(
> + priority_aggregation,
> + priority_action_node,
> + &queue_context->Priority.Actions,
> + _Thread_Priority_action_change,
> + NULL
> + );
> +#endif
> + break;
> + case PRIORITY_ACTION_REMOVE:
> +#if defined(RTEMS_SMP)
> + _Priority_Extract(
> + priority_aggregation,
> + priority_action_node,
> + &queue_context->Priority.Actions,
> + _Thread_Priority_action_remove,
> + _Thread_Priority_action_change,
> + NULL
> + );
> +#else
> + _Priority_Extract_non_empty(
> + priority_aggregation,
> + priority_action_node,
> + &queue_context->Priority.Actions,
> + _Thread_Priority_action_change,
> + NULL
> + );
> +#endif
> + break;
> + default:
> + _Assert( priority_action_type == PRIORITY_ACTION_CHANGE );
> + _Priority_Change(
> + priority_aggregation,
> + priority_action_node,
> + prepend_it,
> + &queue_context->Priority.Actions,
> + _Thread_Priority_action_change,
> + NULL
> + );
> + break;
> + }
> +
> + priority_aggregation = priority_aggregation_next;
> + } while ( _Priority_Actions_is_valid( priority_aggregation ) );
> +
> + if ( !_Priority_Actions_is_empty( &queue_context->Priority.Actions ) ) {
> + _Thread_queue_Context_add_priority_update( queue_context, the_thread );
> + ( *operations->priority_actions )(
> + queue,
> + &queue_context->Priority.Actions
> );
> - } else {
> - the_thread = NULL;
> }
> +}
>
> - return the_thread;
> +void _Thread_Priority_perform_actions(
> + Thread_Control *start_of_path,
> + Thread_queue_Context *queue_context
> +)
> +{
> +#if defined(RTEMS_SMP)
> + Thread_queue_Link *link;
> +#endif
> + Thread_Control *the_thread;
> + size_t update_count;
> +
> + _Assert( start_of_path != NULL );
> +
> +#if defined(RTEMS_SMP)
> + link = &queue_context->Path.Start;
> +#endif
> + the_thread = start_of_path;
> + update_count = _Thread_queue_Context_get_priority_updates( queue_context );
> +
> + while ( true ) {
> + Thread_queue_Queue *queue;
> +
> +#if defined(RTEMS_SMP)
> + _Assert( link->owner == the_thread );
> + queue = link->Lock_context.Wait.queue;
> +#else
> + queue = the_thread->Wait.queue;
> +#endif
> +
> + _Thread_Priority_do_perform_actions(
> + the_thread,
> + queue,
> + the_thread->Wait.operations,
> + false,
> + queue_context
> + );
> +
> + if ( _Priority_Actions_is_empty( &queue_context->Priority.Actions ) ) {
> + return;
> + }
> +
> + _Assert( queue != NULL );
> + the_thread = queue->owner;
> + _Assert( the_thread != NULL );
> +
> +#if defined(RTEMS_SMP)
> + link = THREAD_QUEUE_LINK_OF_PATH_NODE( _Chain_Next( &link->Path_node ) );
> +#endif
> + _Thread_queue_Context_set_priority_updates( queue_context, update_count
> );
Has the queue_context->update_count changed since it was earlier read?
If yes, why is it correct to restore it?
> + }
> }
>
> -Thread_Control *_Thread_Apply_priority(
> - Thread_Control *the_thread,
> - Priority_Control new_priority,
> - void *arg,
> - Thread_Change_priority_filter filter,
> - bool prepend_it
> +static void _Thread_Priority_apply(
> + Thread_Control *the_thread,
> + Priority_Node *priority_action_node,
> + Thread_queue_Context *queue_context,
> + bool prepend_it,
> + Priority_Action_type priority_action_type
> )
> {
> - Thread_queue_Context queue_context;
> - Thread_Control *the_thread_to_update;
> + Scheduler_Node *own_node;
> + Thread_queue_Queue *queue;
>
> - _Thread_Wait_acquire( the_thread, &queue_context );
> - the_thread_to_update = _Thread_Apply_priority_locked(
> + own_node = _Thread_Scheduler_get_own_node( the_thread );
> + _Priority_Actions_initialize_one(
> + &queue_context->Priority.Actions,
> + &own_node->Wait.Priority,
> + priority_action_node,
> + priority_action_type
> + );
> + queue = the_thread->Wait.queue;
> + _Thread_Priority_do_perform_actions(
> the_thread,
> - new_priority,
> - arg,
> - filter,
> + queue,
> + the_thread->Wait.operations,
> prepend_it,
> - &queue_context
> + queue_context
> );
> - _Thread_Wait_release( the_thread, &queue_context );
> - return the_thread_to_update;
> -}
> -
> -void _Thread_Update_priority( Thread_Control *the_thread )
> -{
> - if ( the_thread != NULL ) {
> - ISR_lock_Context lock_context;
>
> - _Thread_State_acquire( the_thread, &lock_context );
> - _Scheduler_Update_priority( the_thread );
> - _Thread_State_release( the_thread, &lock_context );
> + if ( !_Priority_Actions_is_empty( &queue_context->Priority.Actions ) ) {
> + _Thread_queue_Path_acquire_critical( queue, the_thread, queue_context );
> + _Thread_Priority_perform_actions( queue->owner, queue_context );
> + _Thread_queue_Path_release_critical( queue_context );
> }
> }
>
> -void _Thread_Change_priority(
> - Thread_Control *the_thread,
> - Priority_Control new_priority,
> - void *arg,
> - Thread_Change_priority_filter filter,
> - bool prepend_it
> +void _Thread_Priority_add(
> + Thread_Control *the_thread,
> + Priority_Node *priority_node,
> + Thread_queue_Context *queue_context
> )
> {
> - the_thread = _Thread_Apply_priority(
> + _Thread_Priority_apply(
> the_thread,
> - new_priority,
> - arg,
> - filter,
> - prepend_it
> + priority_node,
> + queue_context,
> + false,
> + PRIORITY_ACTION_ADD
> );
> - _Thread_Update_priority( the_thread );
> }
>
> -static bool _Thread_Raise_priority_filter(
> - Thread_Control *the_thread,
> - Priority_Control *new_priority,
> - void *arg
> +void _Thread_Priority_remove(
> + Thread_Control *the_thread,
> + Priority_Node *priority_node,
> + Thread_queue_Context *queue_context
> )
> {
> - return _Thread_Priority_less_than(
> - _Thread_Get_priority( the_thread ),
> - *new_priority
> + _Thread_Priority_apply(
> + the_thread,
> + priority_node,
> + queue_context,
> + true,
> + PRIORITY_ACTION_REMOVE
> );
> }
>
> -void _Thread_Raise_priority(
> - Thread_Control *the_thread,
> - Priority_Control new_priority
> +void _Thread_Priority_changed(
> + Thread_Control *the_thread,
> + Priority_Node *priority_node,
> + bool prepend_it,
> + Thread_queue_Context *queue_context
> )
> {
> - _Thread_Change_priority(
> + _Thread_Priority_apply(
> the_thread,
> - new_priority,
> - NULL,
> - _Thread_Raise_priority_filter,
> - false
> + priority_node,
> + queue_context,
> + prepend_it,
> + PRIORITY_ACTION_CHANGE
> );
> }
>
> -static bool _Thread_Restore_priority_filter(
> - Thread_Control *the_thread,
> - Priority_Control *new_priority,
> - void *arg
> +void _Thread_Priority_replace(
> + Thread_Control *the_thread,
> + Priority_Node *victim_node,
> + Priority_Node *replacement_node
> )
> {
> - *new_priority = the_thread->real_priority;
> -
> - the_thread->priority_restore_hint = false;
> + Scheduler_Node *own_node;
>
> - return *new_priority != _Thread_Get_priority( the_thread );
> + own_node = _Thread_Scheduler_get_own_node( the_thread );
> + _Priority_Replace( &own_node->Wait.Priority, victim_node, replacement_node
> );
> }
>
> -void _Thread_Restore_priority( Thread_Control *the_thread )
> +void _Thread_Priority_update( Thread_queue_Context *queue_context )
> {
> - _Thread_Change_priority(
> - the_thread,
> - 0,
> - NULL,
> - _Thread_Restore_priority_filter,
> - true
> - );
> + size_t i;
> + size_t n;
> +
> + n = queue_context->Priority.update_count;
> +
> + for ( i = 0; i < n ; ++i ) {
> + Thread_Control *the_thread;
> + ISR_lock_Context lock_context;
> +
> + the_thread = queue_context->Priority.update[ i ];
> + _Thread_State_acquire( the_thread, &lock_context );
> + _Scheduler_Update_priority( the_thread );
> + _Thread_State_release( the_thread, &lock_context );
> + }
Why doesn't this decrement the update_count after completing the
update? I'm missing some important detail about the "lifetime" of
these update counters.
[...]
The rest looked relatively straightforward on a quick skim.
-Gedare
_______________________________________________
devel mailing list
devel@rtems.org
http://lists.rtems.org/mailman/listinfo/devel
