Bug 2224333

Summary: Clarify app credentials replacement procedure
Product: Red Hat OpenStack Reporter: Jon Uriarte <juriarte>
Component: openstack-keystoneAssignee: Roger Heslop <rheslop>
Status: ASSIGNED --- QA Contact: Jeremy Agee <jagee>
Severity: low Docs Contact:
Priority: unspecified    
Version: 17.1 (Wallaby)CC: ggrasza, imatza, itbrown, oblaut
Target Milestone: ---Keywords: Documentation, Triaged
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Jon Uriarte 2023-07-20 12:38:26 UTC
Description of problem:
Following 17.1 beta documentation [1] in order to rotate the application credentials I noticed an erratic behavior (or the need of clearer docs about it).

Version-Release number of selected component (if applicable):
Red Hat OpenStack Platform release 17.1.0 Beta (Wallaby)
RHOS-17.1-RHEL-9-20230712.n.1


How reproducible: always


Steps to Reproduce:
1. On a running OSP cluster, create application credentials:

$ OS_CLOUD=shiftstack openstack application credential create --description "App Creds - All roles" "MyAppCreds" -f yaml
description: App Creds - All roles                       
expires_at: null                                                        
id: xx                                    
name: MyAppCreds                                        
project_id: tt                                      
roles: member swiftoperator reader
secret: zz
system: null
unrestricted: false
user_id: yy

2. Update the clouds.yaml with the application credentials - replaced with:
[...]
    shiftstack:
        auth:
            auth_url: https://overcloud.redhat.local:13000
            application_credential_id: "xx"
            application_credential_secret: "zz"
        cacert: /etc/ipa/ca.crt
        identity_api_version: '3'
        region_name: regionOne
        auth_type: "v3applicationcredential"
[...]

3. Check app credentials are used when querying OSP API:
$ OS_CLOUD=shiftstack openstack server list --verbose
START with options: server list --verbose
command: server list -> openstackclient.compute.v2.server.ListServer (auth=True)
Using auth plugin: v3applicationcredential
+--------------------------------------+-----------------------------+--------+--------------------------------------------------------------+--------------------------+--------+
| ID                                   | Name                        | Status | Networks                                                     | Image                    | Flavor |
+--------------------------------------+-----------------------------+--------+--------------------------------------------------------------+--------------------------+--------+
| 725d4db1-2047-459f-82c5-6743a59481e2 | ostest-bxl4x-worker-2-5fjln | ACTIVE | StorageNFS=172.17.5.209; ostest-bxl4x-openshift=10.196.0.177 | N/A (booted from volume) | worker |
| 6e5e87de-9047-4325-96ed-4b7cf8be5eae | ostest-bxl4x-worker-1-mw2n4 | ACTIVE | StorageNFS=172.17.5.201; ostest-bxl4x-openshift=10.196.3.174 | N/A (booted from volume) | worker |
| 198462c4-f842-4177-8259-a34bb1e7a508 | ostest-bxl4x-worker-0-6rt6x | ACTIVE | StorageNFS=172.17.5.219; ostest-bxl4x-openshift=10.196.3.105 | N/A (booted from volume) | worker |
| 47d5f8a6-75d4-49fd-a3bc-b7ec53fabd1f | ostest-bxl4x-master-2       | ACTIVE | StorageNFS=172.17.5.175; ostest-bxl4x-openshift=10.196.1.226 | N/A (booted from volume) | master |
| 95367100-b410-4480-a31a-b8cd70dc8fe9 | ostest-bxl4x-master-1       | ACTIVE | StorageNFS=172.17.5.185; ostest-bxl4x-openshift=10.196.2.6   | N/A (booted from volume) | master |
| 40f7962e-2732-4018-987c-0f6868968600 | ostest-bxl4x-master-0       | ACTIVE | StorageNFS=172.17.5.192; ostest-bxl4x-openshift=10.196.0.165 | N/A (booted from volume) | master |
+--------------------------------------+-----------------------------+--------+--------------------------------------------------------------+--------------------------+--------+

4. Create a second application credential (following [1]):
$ OS_CLOUD=shiftstack openstack application credential create --description "App Creds - All roles" "MySecondAppCreds" -f yaml
You are not authorized to perform the requested action: Using method 'application_credential' is not allowed for managing additional application credentials.. (HTTP 403) (Request-ID: req-4535f422-e132-4dd9-839f-02a0a06d26ba)


Actual results: App credential creation fails if authenticating via previous application credential. Note that it works when authenticating via regular user/pass, but the clouds.yaml needs to be modified:
[...]
    shiftstack:
        auth:
            auth_url: https://overcloud.redhat.local:13000
            password: blabla
            project_domain_name: Default
            project_id: xx
            project_name: shiftstack
            user_domain_name: Default
            username: shiftstack_user
        cacert: /etc/ipa/ca.crt
        identity_api_version: '3'
        region_name: regionOne

[...]


Expected results: Successful app credential creation


So even
a) the behavior should be fixed if it's not correct
b) docs should state that in order to create a second app credential the regular user must be used (thus the clouds.yaml changed back to it's original content).

"
Replacing existing Application Credentials for clouds.yaml files

Replace an existing Application Credential used by clouds.yaml:

For example, if your clouds.yaml contains an Application Credential called AppCred1 that is due to expire:

   1. Create an Application Credential called 'AppCred2'. <- this fails if not authenticated via regular user/pass
"

[1] https://access.redhat.com/documentation/en-us/red_hat_openstack_platform/17.1-beta/html/managing_openstack_identity_resources/assembly_application-credentials#proc_replacing-application-credentials_application-credentials

Comment 1 Roger Heslop 2023-07-27 20:13:35 UTC
This behavior is documented in 9.3:

! Warning: Using the --unrestricted parameter enables the application credential to create and delete other application credentials and trusts. This is potentially dangerous behavior and is disabled by default. You cannot use the --unrestricted parameter in combination with other access rules.

Comment 2 Jon Uriarte 2023-08-07 07:44:42 UTC
Hello Roger, thanks for the info. Shouldn't the same note be added in 9.4 then, in the "Replacing Application Credentials" section?
On the other hand if that's the flag to use for replacing the App credentials we are suggesting to use a potentially dangerous behavior
as stated in the note.

Comment 3 Roger Heslop 2023-08-07 13:21:30 UTC
The warning is placed below the `openstack application credential create` because that's the command that would be used to apply the `--unrestricted` flag. The verbiage is copied in large part from the `--help` menu of the same command, and is not a recommendation for a course of action, rather explains what you've observed. Instead of copying the warning to another section, I could place verbiage stating that for security purposes you cannot, _by default_, use an application credential to create another application credential. This would accomplish a few objectives:

* The behavior you are observing would not confuse another customer
* It would let the customer know that the course of action is not recommended
* It would let the customer know that there is a default behavior that can be changed. From there it would be up to them to seek it out.

What are your thoughts?

Comment 4 Roger Heslop 2023-08-07 13:30:19 UTC
Actually I see what you're saying the procedure: 

Replacing existing Application Credentials for clouds.yaml files

Replace an existing Application Credential used by clouds.yaml:

For example, if your clouds.yaml contains an Application Credential called AppCred1 that is due to expire:

    Create an Application Credential called 'AppCred2'.
    Add the new AppCred2 to the clouds.yaml file, while removing the AppCred1 configuration.
    Generate a token with clouds.yaml to confirm that the credentials are working as expected. See step 4 of Using Application Credentials to generate tokens for more information. 

Let me see how this can be best revised or restated — thanks.

Comment 5 Roger Heslop 2023-08-07 14:47:32 UTC
I've discussed this procedure with the Security group.

Because the procedure requires the use of the `--unrestricted` flag, and because Red Hat does not recommend this course of action, that procedure will be removed from documentation.
I apologize for the confusion.

Please let me know if there is anything else I can do to the meet the needs of this ticket.

Comment 6 Jon Uriarte 2023-08-09 11:49:00 UTC
So the procedure:

>>Replacing existing Application Credentials for clouds.yaml files
>>
>>Replace an existing Application Credential used by clouds.yaml:
>>
>>For example, if your clouds.yaml contains an Application Credential called AppCred1 that is due to expire:
>>
>>    Create an Application Credential called 'AppCred2'.
>>    Add the new AppCred2 to the clouds.yaml file, while removing the AppCred1 configuration.
>>    Generate a token with clouds.yaml to confirm that the credentials are working as expected. See step 4 of Using Application Credentials to generate tokens for more information. 

will be removed from the 9.4 section?
Shouldn't we keep it and adapt it to a supported way?

Comment 7 Roger Heslop 2023-08-09 15:12:10 UTC
I had read the intent of the procedure as encouraging an unsupported action — but let's try it another way.
If we're just advising the user to replace the credentials a step can be added as follows.

What are your thoughts on the following?


For example, if your clouds.yaml contains an Application Credential called AppCred1 that is due to expire:

>>> Authenticate as a user from a resource credentials file or from clouds.yaml. Your application credentials cannot create another application credential by default.<<<<<<<
    Create an Application Credential called 'AppCred2'.
    Add the new AppCred2 to the clouds.yaml file, while removing the AppCred1 configuration.
    Generate a token with clouds.yaml to confirm that the credentials are working as expected. See step 4 of Using Application Credentials to generate tokens for more information.

Comment 8 Jon Uriarte 2023-08-09 16:13:13 UTC
That looks good! but please confirm it with the security DFG, I'm not part of such group and I just "suffered" the issue as a user