Bug 2101846 - TripleO policy override file isn't rendering the correct definitions
Summary: TripleO policy override file isn't rendering the correct definitions
Keywords:
Status: CLOSED CURRENTRELEASE
Alias: None
Product: Red Hat OpenStack
Classification: Red Hat
Component: openstack-octavia
Version: 17.0 (Wallaby)
Hardware: Unspecified
OS: Unspecified
high
high
Target Milestone: Alpha
: 17.1
Assignee: Gregory Thiemonge
QA Contact: Bruna Bonguardo
URL:
Whiteboard:
Depends On:
Blocks: 1901686 2124617
TreeView+ depends on / blocked
 
Reported: 2022-06-28 14:29 UTC by Omer Schwartz
Modified: 2023-08-08 20:33 UTC (History)
15 users (show)

Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Before this update, if secure RBAC was enabled, missing roles in the RHOSP deployment could cause Load-balancing service (octavia) API failures. In RHOSP 17.1 GA, this issue has been resolved.
Clone Of:
Environment:
Last Closed: 2023-02-02 15:33:10 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
OpenStack gerrit 776942 0 None MERGED Fix missing roles for Octavia services 2022-07-19 06:40:19 UTC
RDO 43788 0 None rpm-master: MERGED openstack/octavia-distgit: Update Octavia spec for secure-RBAC (I85e303b48bd6ebde7a70ec958b89ffe56dcd2fd8) 2022-07-13 09:20:55 UTC
RDO 43979 0 None wallaby-rdo: MERGED openstack/octavia-distgit: Update Octavia spec for secure-RBAC (I85e303b48bd6ebde7a70ec958b89ffe56dcd2fd8) 2022-07-13 09:21:00 UTC
Red Hat Issue Tracker OSP-16101 0 None None None 2022-06-28 14:33:40 UTC

Description Omer Schwartz 2022-06-28 14:29:17 UTC
Description of problem:
The TripleO policy override file https://github.com/openstack/tripleo-heat-templates/blob/master/environments/enable-secure-rbac.yaml isn't rendering the correct definitions for (at least) Octavia and probably other Openstack services as well.

For example:
In the following override section for Octavia policy
https://github.com/openstack/tripleo-heat-templates/blob/master/environments/enable-secure-rbac.yaml#L3701

we see role rules, such as
value: "role:admin or rule:project-reader"
value: "role:admin or rule:project-member"

that we do not see on our rendered policy.yaml file in our Jenkins jobs (here is a link from our latest build):
http://rhos-ci-logs.lab.eng.tlv2.redhat.com/logs/rcj/DFG-network-octavia-17.0_director-rhel-virthost-3cont_3comp-ipv4-geneve-actstby/66/controller-0/var/lib/config-data/octavia/etc/octavia/policy.yaml.gz

The policy.yaml file isn't rendering a definition for "rule:project-reader" nor "rule:project-member".

In order to verify the Secure RBAC feature, each OSP component needs to have those role rules defined, e.g. Octavia define them this way:
https://github.com/openstack/octavia/blob/master/etc/policy/keystone_default_roles-policy.yaml#L13


The fact that those role rules are not rendered causes many Secure RBAC tests to fail and blocks the Secure RBAC feature.


Version-Release number of selected component (if applicable):
core_puddle: RHOS-17.0-RHEL-9-20220623.n.1

How reproducible:
100%

Steps to Reproduce:
1. Run any Octavia OSP17 job on CI and add "enable-secure-rbac" on OVERCLOUD_TEMPLATES, and add 
--config-options load_balancer.enforce_new_defaults='True'
On TEMPEST_RUN_OVERRIDE_OPTIONS.
A link for the Active standby job (could be any):
https://rhos-ci-jenkins.lab.eng.tlv2.redhat.com/view/DFG/view/network/view/octavia/job/DFG-network-octavia-17.0_director-rhel-virthost-3cont_3comp-ipv4-geneve-actstby/


Actual results:
Important role rules are not being rendered on /controller-0/var/lib/config-data/octavia/etc/octavia/policy.yaml.gz
And many Secure RBAC tests fail.

Expected results:
All Secure RBAC should pass

Comment 2 Michael Johnson 2022-06-29 15:19:21 UTC
Root issue is the roles "project_reader" and "project_member" are not defined in the policy override rendered via tripleo and the enable-secure-rbac.yaml environment file.

Comment 3 Douglas Mendizábal 2022-07-08 19:51:23 UTC
I'm not sure I understand what the issue is here?  There seems to be a misunderstanding on how the policy.yaml override files work.

IIUC, the effective policy used by a service is a combination of both the default rules defined in code in the project as well the override rules specified in the policy.yaml file.  For Octavia, the "project-member" and "project-reader" rules have default values defined here: https://opendev.org/openstack/octavia/src/branch/stable/wallaby/octavia/policies/base.py#L52-L64

Because we are not changing the value of the "project-member" and "project-reader" rules, then we do not need to specify them in the policy.yaml override file.  In fact, the oslo.policy library provides a mechanism to help identify redundant rules that are present in both the default rules and the policy.yaml, noting that "These are rules that can be removed from the policy file with no change in effective policy": https://docs.openstack.org/oslo.policy/latest/user/usage.html#list-of-redundant-configuration

Are you seeing different behavior in Octavia?  Does the effective policy not contain the default rules?

Comment 4 Michael Johnson 2022-07-11 17:12:51 UTC
Yeah, you are right, those are globally defined in the defaults, so... hmmm, what is failing here?

Omer, can you point us to a current run that is failing? The linked job is passing and not using the policy override.

Comment 5 Michael Johnson 2022-07-12 00:08:20 UTC
I have looked into this some more. I found the job Omer was running (http://rhos-ci-logs.lab.eng.tlv2.redhat.com/logs/rcj/DFG-network-octavia-17.0_director-rhel-virthost-3cont_3comp-ipv4-geneve-actstby/66/controller-0/var/lib/config-data/octavia/etc/octavia/policy.yaml.gz)

I checked the tempest.conf and the RBAC test type is incorrect:
[load_balancer]
RBAC_test_type = owner_or_admin

http://rhos-ci-logs.lab.eng.tlv2.redhat.com/logs/rcj/DFG-network-octavia-17.0_director-rhel-virthost-3cont_3comp-ipv4-geneve-actstby/66/undercloud-0/home/stack/tempest-dir/etc/tempest.conf.gz

Which is likely causing this:
2022-06-27 13:48:59.490 19 DEBUG octavia.common.policy [req-55138779-6e68-4983-bf53-0beecfa8f691 - d2859124b48d4b68b28905c53203b92b - default default] Policy check for os_load-balancer_api:provider:get_all failed with credentials {'is_admin': False, 'user_id': None, 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd2859124b48d4b68b28905c53203b92b', 'project_domain_id': 'default', 'roles': ['swiftoperator'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} authorize /usr/lib/python3.9/site-packages/octavia/common/policy.py:126

Where the user being used doesn't have the reader or member role (It only has "swiftoperator" for whatever reason).

Please make sure you configure tempest to test the proper RBAC configuration for Octavia.

[load_balancer]
RBAC_test_type = keystone_default_roles


Note You need to log in before you can comment on or make changes to this bug.