Public bug reported:
Summary:
--------
When a stack is created with a certain user,
"reauthentication_auth_method=trusts" and an update is triggered, the
update will fail with the below error:
$ openstack stack resource list -n 5 teststack
+---------------+--------------------------------------+--------------------------------------+-----------------+----------------------+------------------------------------------------+
| resource_name | physical_resource_id | resource_type
| resource_status | updated_time | stack_name
|
+---------------+--------------------------------------+--------------------------------------+-----------------+----------------------+------------------------------------------------+
| test_res | 969e8d08-ad3d-4fae-923b-1badecdebdf3 |
OS::Heat::ResourceGroup | UPDATE_FAILED | 2025-02-06T12:11:11Z |
teststack |
| 1 | 1713eb91-93dd-47a9-b0ac-cf3247ee5ebd |
file:///home/ubuntu/nestedstack.yaml | UPDATE_COMPLETE | 2025-02-06T12:08:19Z |
teststack-test_res-tany4qj3iums |
| 0 | 4d99bfcb-9fe8-4c35-83f6-d3e09de11e39 |
file:///home/ubuntu/nestedstack.yaml | UPDATE_COMPLETE | 2025-02-06T12:08:18Z |
teststack-test_res-tany4qj3iums |
| test-user | c1d165409ebe4f2193682252c8f11b2e | OS::Keystone::User
| CREATE_COMPLETE | 2025-02-06T11:40:06Z |
teststack-test_res-tany4qj3iums-1-lceepegkpdnz |
| test-user | 5695cd71bbc545a586e50dd21ff1a7e5 | OS::Keystone::User
| CREATE_COMPLETE | 2025-02-06T11:40:05Z |
teststack-test_res-tany4qj3iums-0-y4vpheh75oi4 |
+---------------+--------------------------------------+--------------------------------------+-----------------+----------------------+------------------------------------------------+
2025-02-06 12:11:11.043 279650 INFO heat.engine.service
[req-4aa4d063-e789-4fa5-ae88-02b910b1d8ec - - - - -] Updating stack
teststack-test_res-tany4qj3iums
2025-02-06 12:11:11.045 279650 WARNING oslo_config.cfg
[req-4aa4d063-e789-4fa5-ae88-02b910b1d8ec - - - - -] Deprecated: Option
"deferred_auth_method" from group "DEFAULT" is deprecated for removal (Stored
password based deferred auth is b
roken when used with keystone v3 and is not supported.). Its value may be
silently ignored in the future.
2025-02-06 12:11:11.049 279651 WARNING oslo_messaging._drivers.common
[req-4aa4d063-e789-4fa5-ae88-02b910b1d8ec - - - - -] Failed to rebuild remote
exception due to error: Error.__init__() got an unexpected keyword argument
'message'
2025-02-06 12:11:11.050 279651 INFO heat.engine.resource
[req-4aa4d063-e789-4fa5-ae88-02b910b1d8ec - - - - -] UPDATE: ResourceGroup
"test_res" [969e8d08-ad3d-4fae-923b-1badecdebdf3] Stack "teststack"
[3ea4ee9d-1686-4a97-ae92-c1f6232cd16c]
2025-02-06 12:11:11.050 279651 ERROR heat.engine.resource Traceback (most
recent call last):
2025-02-06 12:11:11.050 279651 ERROR heat.engine.resource File
"/usr/lib/python3/dist-packages/heat/engine/resource.py", line 916, in
_action_recorder
2025-02-06 12:11:11.050 279651 ERROR heat.engine.resource yield
2025-02-06 12:11:11.050 279651 ERROR heat.engine.resource File
"/usr/lib/python3/dist-packages/heat/engine/resource.py", line 1693, in update
2025-02-06 12:11:11.050 279651 ERROR heat.engine.resource yield from
self.action_handler_task(action,
2025-02-06 12:11:11.050 279651 ERROR heat.engine.resource File
"/usr/lib/python3/dist-packages/heat/engine/resource.py", line 970, in
action_handler_task
2025-02-06 12:11:11.050 279651 ERROR heat.engine.resource handler_data =
handler(*args)
2025-02-06 12:11:11.050 279651 ERROR heat.engine.resource File
"/usr/lib/python3/dist-packages/heat/engine/resources/openstack/heat/resource_group.py",
line 485, in handle_update
2025-02-06 12:11:11.050 279651 ERROR heat.engine.resource
checkers[0].start()
2025-02-06 12:11:11.050 279651 ERROR heat.engine.resource File
"/usr/lib/python3/dist-packages/heat/engine/scheduler.py", line 183, in start
2025-02-06 12:11:11.050 279651 ERROR heat.engine.resource self.step()
2025-02-06 12:11:11.050 279651 ERROR heat.engine.resource File
"/usr/lib/python3/dist-packages/heat/engine/scheduler.py", line 210, in step
2025-02-06 12:11:11.050 279651 ERROR heat.engine.resource poll_period =
next(self._runner)
2025-02-06 12:11:11.050 279651 ERROR heat.engine.resource File
"/usr/lib/python3/dist-packages/heat/engine/resources/openstack/heat/resource_group.py",
line 438, in _run_to_completion
2025-02-06 12:11:11.050 279651 ERROR heat.engine.resource updater =
self.update_with_template(template, {},
2025-02-06 12:11:11.050 279651 ERROR heat.engine.resource File
"/usr/lib/python3/dist-packages/heat/engine/resources/stack_resource.py", line
533, in update_with_template
2025-02-06 12:11:11.050 279651 ERROR heat.engine.resource
self.rpc_client()._update_stack(self.context, **kwargs)
2025-02-06 12:11:11.050 279651 ERROR heat.engine.resource File
"/usr/lib/python3/dist-packages/heat/rpc/client.py", line 343, in _update_stack
2025-02-06 12:11:11.050 279651 ERROR heat.engine.resource return
self.call(ctxt,
2025-02-06 12:11:11.050 279651 ERROR heat.engine.resource File
"/usr/lib/python3/dist-packages/heat/rpc/client.py", line 89, in call
2025-02-06 12:11:11.050 279651 ERROR heat.engine.resource return
client.call(ctxt, method, **kwargs)
2025-02-06 12:11:11.050 279651 ERROR heat.engine.resource File
"/usr/lib/python3/dist-packages/oslo_messaging/rpc/client.py", line 189, in call
2025-02-06 12:11:11.050 279651 ERROR heat.engine.resource result =
self.transport._send(
2025-02-06 12:11:11.050 279651 ERROR heat.engine.resource File
"/usr/lib/python3/dist-packages/oslo_messaging/transport.py", line 123, in _send
2025-02-06 12:11:11.050 279651 ERROR heat.engine.resource return
self._driver.send(target, ctxt, message,
2025-02-06 12:11:11.050 279651 ERROR heat.engine.resource File
"/usr/lib/python3/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line
689, in send
2025-02-06 12:11:11.050 279651 ERROR heat.engine.resource return
self._send(target, ctxt, message, wait_for_reply, timeout,
2025-02-06 12:11:11.050 279651 ERROR heat.engine.resource File
"/usr/lib/python3/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line
681, in _send
2025-02-06 12:11:11.050 279651 ERROR heat.engine.resource raise result
2025-02-06 12:11:11.050 279651 ERROR heat.engine.resource
oslo_messaging.rpc.client.RemoteError: Remote error: Error Attempt to use
stored_context with no user_creds
The above happens because the user_creds_id column in the stack table of the
Heat DB is set to NULL for all nested resources (butnot the main stack):
mysql> select id,name,action,status,user_creds_id from heat.stack where name
like "%teststack%" and deleted_at is NULL;
+--------------------------------------+------------------------------------------------+--------+----------+---------------+
| id | name
| action | status | user_creds_id |
+--------------------------------------+------------------------------------------------+--------+----------+---------------+
| 1713eb91-93dd-47a9-b0ac-cf3247ee5ebd |
teststack-test_res-tany4qj3iums-1-lceepegkpdnz | UPDATE | COMPLETE |
NULL |
| 3ea4ee9d-1686-4a97-ae92-c1f6232cd16c | teststack
| UPDATE | FAILED | 49 |
| 4d99bfcb-9fe8-4c35-83f6-d3e09de11e39 |
teststack-test_res-tany4qj3iums-0-y4vpheh75oi4 | UPDATE | COMPLETE |
NULL |
| 969e8d08-ad3d-4fae-923b-1badecdebdf3 | teststack-test_res-tany4qj3iums
| UPDATE | COMPLETE | NULL |
+--------------------------------------+------------------------------------------------+--------+----------+---------------+
I would argue that:
- if different users are not to be used then the operation should be denied or
error out in a more graceful and controlled way without "corrupting" the DB
- the user_creds_id column should not be set to NULL preventing further updates
(regardless of the user) to succeed
Versions affected:
------------------
Openstack Yoga (no other versions were tested
Reproducer:
-----------
1) Set up an Openstack cloud with Heat and set up a test user
2) Log in as admin or any other user able to do stack operations:
$ env | grep OS_
OS_REGION_NAME=RegionOne
OS_AUTH_URL=https://<ip>:5000/v3
OS_PROJECT_DOMAIN_NAME=admin_domain
OS_AUTH_PROTOCOL=https
OS_USERNAME=admin
OS_AUTH_TYPE=password
OS_USER_DOMAIN_NAME=admin_domain
OS_PROJECT_NAME=admin
OS_PASSWORD=<some password>
OS_IDENTITY_API_VERSION=3
$ openstack user list
+----------------------------------+------------+
| ID | Name |
+----------------------------------+------------+
| 95108ce766424c7aa92b20c92dd61e8f | admin |
| 415a1d3f6d324187a15391c8205ce131 | demo |
| 190c8cbd62734b9eb1f82ab482b9441e | alt_demo |
| c7ac272709164a79b2b1833db40696da | testadmin |
+----------------------------------+------------+
3) Create a stack with userfrom step 2
$ openstack stack create -t ./stack.yaml teststack --wait
2025-02-06 11:40:02Z [teststack]: CREATE_IN_PROGRESS Stack CREATE started
2025-02-06 11:40:02Z [teststack.test_res]: CREATE_IN_PROGRESS state changed
2025-02-06 11:40:08Z [teststack.test_res]: CREATE_COMPLETE state changed
2025-02-06 11:40:09Z [teststack]: CREATE_COMPLETE Stack CREATE completed
successfully
+---------------------+--------------------------------------+
| Field | Value |
+---------------------+--------------------------------------+
| id | 3ea4ee9d-1686-4a97-ae92-c1f6232cd16c |
| stack_name | teststack |
| description | No description |
| creation_time | 2025-02-06T11:40:01Z |
| updated_time | None |
| stack_status | CREATE_COMPLETE |
| stack_status_reason | Stack CREATE completed successfully |
+---------------------+--------------------------------------+
4) Update the stack, this would work:
$ openstack stack update -t stack.yaml teststack --wait
2025-02-06 11:40:20Z [teststack]: UPDATE_IN_PROGRESS Stack UPDATE started
2025-02-06 11:40:20Z [teststack.test_res]: UPDATE_IN_PROGRESS state changed
2025-02-06 11:40:29Z [teststack.test_res]: UPDATE_COMPLETE state changed
2025-02-06 11:40:29Z [teststack]: UPDATE_COMPLETE Stack UPDATE completed
successfully
+---------------------+--------------------------------------+
| Field | Value |
+---------------------+--------------------------------------+
| id | 3ea4ee9d-1686-4a97-ae92-c1f6232cd16c |
| stack_name | teststack |
| description | No description |
| creation_time | 2025-02-06T11:40:01Z |
| updated_time | 2025-02-06T11:40:20Z |
| stack_status | UPDATE_COMPLETE |
| stack_status_reason | Stack UPDATE completed successfully |
+---------------------+--------------------------------------+
5) Change users and re-run the stack update:
$ export OS_USERNAME=testadmin; export OS_PASSWORD=<somepassword>
$ openstack stack update -t stack.yaml teststack --wait
2025-02-06 11:41:35Z [teststack]: UPDATE_IN_PROGRESS Stack UPDATE started
2025-02-06 11:41:35Z [teststack.test_res]: UPDATE_IN_PROGRESS state changed
2025-02-06 11:41:36Z [teststack.test_res]: UPDATE_FAILED RemoteError:
resources.test_res: Remote error: Error Attempt to use stored_context with no
user_creds
['Traceback (most recent call last):\n', ' File
"/usr/lib/python3/dist-packages/heat/common/context.py", line 410, in wrapped\n
return func(self, ctx
2025-02-06 11:41:36Z [teststack]: UPDATE_FAILED Resource UPDATE failed:
RemoteError: resources.test_res: Remote error: Error Attempt to use
stored_context with no user_creds
['Traceback (most recent call last):\n', ' File
"/usr/lib/python3/dist-packages/heat/common/context.py", line 410, in wrapped\n
Stack teststack UPDATE_FAILED
Notice below the simple test stacks I used:
stack.yaml:
heat_template_version: 2021-04-16
resources:
test_res:
type: OS::Heat::ResourceGroup
#update_policy:
#batch_create:
# max_batch_size: 1
#rolling_update:
# max_batch_size: 1
properties:
count: 2
resource_def:
type: nestedstack.yaml
properties:
index_name: "%index%"
nestedstack.yaml:
heat_template_version: 2015-04-30
parameters:
index_name:
type: string
description: ""
resources:
test-user:
type: OS::Keystone::User
properties:
default_project: "admin"
domain: admin_domain
enabled: True
name: { list_join : ["",["test", { get_param: index_name }]] }
password: "test"
** Affects: heat (Ubuntu)
Importance: Undecided
Status: New
--
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/2097539
Title:
Heat stack updates fail when "reauthentication_auth_method=trusts"
To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/heat/+bug/2097539/+subscriptions
--
ubuntu-bugs mailing list
ubuntu-bugs@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
