Bug 2217725

Summary: [RHOSP 17.0] When specifying OS_PROJECT_NAME in the rc file variables the command fails to execute.
Product: Red Hat OpenStack Reporter: Takemi Asakura <tasakura>
Component: python-openstackclientAssignee: OSP Team <rhos-maint>
Status: CLOSED NOTABUG QA Contact: Nobody <nobody>
Severity: low Docs Contact:
Priority: unspecified    
Version: 17.0 (Wallaby)CC: apevec, jpichon, lhh, tkajinam
Target Milestone: ---   
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2023-06-28 01:55:32 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Takemi Asakura 2023-06-27 03:06:28 UTC
Description of problem:
This issue is similar to BZ#2185118, but some of the differences are new to this report.

We did not encounter this issue in RHOSP 17.0.0, but we experienced a similar problem in RHOSP 17.0.1.
In our case, we did not change the Domain from Default, and when attempting to perform certain actions 
with a newly created project and user, we received an error indicating 'The request you have made requires 
authentication. (HTTP 401).'[1]

However, when explicitly specifying `OS_PROJECT_ID`, the command executed successfully.[2]

Upon examining the debug logs during command execution, it seems that despite setting `OS_PROJECT_NAME=testproject`, 
the admin project is being used instead.[3]

[1]
~~~
(overcloud) [stack@undercloud-0 ~]$ cat /etc/rhosp-release 
Red Hat OpenStack Platform release 17.0.1 (Wallaby)
(overcloud) [stack@undercloud-0 ~]$ openstack project create testproject
(overcloud) [stack@undercloud-0 ~]$ openstack user create testuser --password **** --project testproject
(overcloud) [stack@undercloud-0 ~]$ openstack role add admin --user testuser --project testproject
(overcloud) [stack@undercloud-0 ~]$ cat testprojectrc 
# Clear any old environment that may conflict.
for key in $( set | awk '{FS="="}  /^OS_/ {print $1}' ); do unset $key ; done
export OS_USERNAME=testuser
export OS_PROJECT_NAME=testproject
#export OS_PROJECT_ID=349009ea91db4117a69c79c26c6451dd  <==========(*)
export OS_USER_DOMAIN_NAME=Default
export OS_PROJECT_DOMAIN_NAME=Default
export OS_NO_CACHE=True
export OS_CLOUD=overcloud
export no_proxy=10.0.0.131,192.168.24.12
export PYTHONWARNINGS='ignore:Certificate has no, ignore:A true SSLContext object is not available'
export OS_AUTH_TYPE=password
export OS_PASSWORD=*****
export OS_AUTH_URL=http://10.0.0.131:5000
export OS_IDENTITY_API_VERSION=3
export OS_COMPUTE_API_VERSION=2.latest
export OS_IMAGE_API_VERSION=2
export OS_VOLUME_API_VERSION=3
export OS_REGION_NAME=regionOne

# Add OS_CLOUD to PS1
if [ -z "${CLOUDPROMPT_ENABLED:-}" ]; then
    export PS1=${PS1:-""}
    export PS1=\${OS_CLOUD:+"(\$OS_CLOUD)"}\ $PS1
    export CLOUDPROMPT_ENABLED=1
fi

(overcloud) [stack@undercloud-0 ~]$ openstack endpoint list
The request you have made requires authentication. (HTTP 401) (Request-ID: req-9d67c35e-7005-4510-9293-62c8a8a42e31)
~~~

[2]
~~~
(overcloud) [stack@undercloud-0 ~]$ vi testprojectrc 
(overcloud) [stack@undercloud-0 ~]$ cat testprojectrc 
# Clear any old environment that may conflict.
for key in $( set | awk '{FS="="}  /^OS_/ {print $1}' ); do unset $key ; done
export OS_USERNAME=testuser
export OS_PROJECT_NAME=testproject
export OS_PROJECT_ID=349009ea91db4117a69c79c26c6451dd <==========(*)
export OS_USER_DOMAIN_NAME=Default
export OS_PROJECT_DOMAIN_NAME=Default
export OS_NO_CACHE=True
export OS_CLOUD=overcloud
export no_proxy=10.0.0.131,192.168.24.12
export PYTHONWARNINGS='ignore:Certificate has no, ignore:A true SSLContext object is not available'
export OS_AUTH_TYPE=password
export OS_PASSWORD=*****
export OS_AUTH_URL=http://10.0.0.131:5000
export OS_IDENTITY_API_VERSION=3
export OS_COMPUTE_API_VERSION=2.latest
export OS_IMAGE_API_VERSION=2
export OS_VOLUME_API_VERSION=3
export OS_REGION_NAME=regionOne

# Add OS_CLOUD to PS1
if [ -z "${CLOUDPROMPT_ENABLED:-}" ]; then
    export PS1=${PS1:-""}
    export PS1=\${OS_CLOUD:+"(\$OS_CLOUD)"}\ $PS1
    export CLOUDPROMPT_ENABLED=1
fi
(overcloud) [stack@undercloud-0 ~]$ source testprojectrc 
(overcloud) [stack@undercloud-0 ~]$ openstack endpoint list
+----------------------------------+-----------+--------------+----------------+---------+-----------+-----------------------------------------------+
| ID                               | Region    | Service Name | Service Type   | Enabled | Interface | URL                                           |
+----------------------------------+-----------+--------------+----------------+---------+-----------+-----------------------------------------------+
| 0209c4bd24c0440fbc17acb8986c5e6f | regionOne | heat         | orchestration  | True    | public    | http://10.0.0.131:8004/v1/%(tenant_id)s       |
| 06610226aaed4836b47fb549fd9cb436 | regionOne | glance       | image          | True    | public    | http://10.0.0.131:9292                        |
...
~~~

[3]
~~~
(overcloud) [stack@undercloud-0 ~]$ openstack endpoint list --debug
START with options: endpoint list --debug
options: Namespace(verbose_level=3, log_file=None, deferred_help=False, debug=True, cloud='overcloud', region_name='regionOne', cacert=None, cert='', key='', verify=None, insecure=None, default_domain='default', interface='public', service_provider='', remote_project_name='', remote_project_id='', remote_project_domain_name='', remote_project_domain_id='', timing=False, os_beta_command=False, profile='', os_compute_api_version='2.latest', os_identity_api_version='3', os_image_api_version='2', os_network_api_version='', os_object_api_version='', os_volume_api_version='3', os_orchestration_api_version='1', os_baremetal_api_version='1.69', os_queues_api_version='2', inspector_api_version='1', inspector_url=None, os_workflow_api_version='2', os_database_api_version='1', os_loadbalancer_api_version='2.0', os_data_processing_api_version='1.1', os_data_processing_url='', os_share_api_version='2.63', os_dns_api_version='2', os_container_infra_api_version='1', os_alarming_api_version='2', os_key_manager_api_version='1', os_tripleoclient_api_version='2', os_metrics_api_version='1', os_placement_api_version='1.0', auth_type='password', auth_url='http://10.0.0.131:5000', consumer_key='', consumer_secret='***', access_key='', access_secret='***', project_name='testproject', username='testuser', password='***', system_scope='', domain_id='', domain_name='', project_id='349009ea91db4117a69c79c26c6451dd', project_domain_id='', project_domain_name='Default', trust_id='', identity_provider='', protocol='', identity_provider_url='', service_provider_endpoint='', service_provider_entity_id='', user_id='', user_domain_id='', user_domain_name='Default', passcode='', default_domain_id='', default_domain_name='', token='***', user='', endpoint='', roles='', aodh_endpoint='', client_id='', client_secret='***', openid_scope='', access_token_endpoint='', discovery_endpoint='', access_token_type='', redirect_uri='', code='', application_credential_secret='***', application_credential_id='', application_credential_name='', access_token='***', auth_methods='', os_project_name=None, os_project_id=None)
Auth plugin password selected
...
auth_config_hook(): {'api_timeout': None, 'verify': True, 'cacert': '', 'cert': None, 'key': None, 'baremetal_status_code_retries': '5', 'baremetal_introspection_status_code_retries': '5', 'image_status_code_retries': '5', 'disable_vendor_agent': {}, 'interface': 'public', 'floating_ip_source': 'neutron', 'image_api_use_tasks': False, 'image_format': 'qcow2', 'message': '', 'network_api_version': '2', 'object_store_api_version': '1', 'secgroup_source': 'neutron', 'status': 'active', 'auth': {'auth_url': 'http://10.0.0.131:5000', 'password': '***', 'username': 'admin', 'user_domain_name': 'Default', 'project_domain_name': 'Default', 'project_id': '349009ea91db4117a69c79c26c6451dd', 'project_name': 'admin'}, 'identity_api_version': '3', 'region_name': 'regionOne', 'volume_api_version': '3', 'verbose_level': 3, 'deferred_help': False, 'debug': True, 'cloud': 'overcloud', 'default_domain': 'default', 'timing': False, 'inspector_api_version': '1', 'auth_url': 'http://10.0.0.131:5000', 'username': 'testuser', 'password': '***', 'beta_command': False, 'compute_api_version': '2.latest', 'image_api_version': '2', 'orchestration_api_version': '1', 'baremetal_api_version': '1.69', 'queues_api_version': '2', 'workflow_api_version': '2', 'database_api_version': '1', 'loadbalancer_api_version': '2.0', 'data_processing_api_version': '1.1', 'share_api_version': '2.63', 'dns_api_version': '2', 'container_infra_api_version': '1', 'alarming_api_version': '2', 'key_manager_api_version': '1', 'tripleoclient_api_version': '2', 'metrics_api_version': '1', 'placement_api_version': '1.0', 'auth_type': 'password', ': []}
defaults: {'api_timeout': None, 'verify': True, 'cacert': None, 'cert': None, 'key': None, 'auth_type': 'password', 'baremetal_status_code_retries': 5, 'baremetal_introspection_status_code_retries': 5, 'image_status_code_retries': 5, 'disable_vendor_agent': {}, 'interface': None, 'floating_ip_source': 'neutron', 'image_api_use_tasks': False, 'image_format': 'qcow2', 'message': '', 'network_api_version': '2', 'object_store_api_version': '1', 'secgroup_source': 'neutron', 'status': 'active'}
cloud cfg: {'api_timeout': None, 'verify': True, 'cacert': '', 'cert': None, 'key': None, 'baremetal_status_code_retries': '5', 'baremetal_introspection_status_code_retries': '5', 'image_status_code_retries': '5', 'disable_vendor_agent': {}, 'interface': 'public', 'floating_ip_source': 'neutron', 'image_api_use_tasks': False, 'image_format': 'qcow2', 'message': '', 'network_api_version': '2', 'object_store_api_version': '1', 'secgroup_source': 'neutron', 'status': 'active', 'auth': {'auth_url': 'http://10.0.0.131:5000', 'password': '***', 'username': 'admin', 'user_domain_name': 'Default', 'project_domain_name': 'Default', 'project_id': '349009ea91db4117a69c79c26c6451dd', 'project_name': 'admin'}, 'identity_api_version': '3', 'region_name': 'regionOne', 'volume_api_version': '3', 'verbose_level': 3, 'deferred_help': False, 'debug': True, 'cloud': 'overcloud', 'default_domain': 'default', 'timing': False, 'inspector_api_version': '1', 'auth_url': 'http://10.0.0.131:5000', 'username': 'testuser', 'password': '***', 'beta_command': False, 'compute_api_version': '2.latest', 'image_api_version': '2', 'orchestration_api_version': '1', 'baremetal_api_version': '1.69', 'queues_api_version': '2', 'workflow_api_version': '2', 'database_api_version': '1', 'loadbalancer_api_version': '2.0', 'data_processing_api_version': '1.1', 'share_api_version': '2.63', 'dns_api_version': '2', 'container_infra_api_version': '1', 'alarming_api_version': '2', 'key_manager_api_version': '1', 'tripleoclient_api_version': '2', 'metrics_api_version': '1', 'placement_api_version': '1.0', 'auth_type': 'password', ': []}
...
Using parameters {'auth_url': 'http://10.0.0.131:5000', 'project_id': '349009ea91db4117a69c79c26c6451dd', 'project_name': 'admin', 'project_domain_name': 'Default', 'username': 'testuser', 'user_domain_name': 'Default', 'password': '***'} <======(*)
Get auth_ref
REQ: curl -g -i -X GET http://10.0.0.131:5000 -H "Accept: application/json" -H "User-Agent: openstacksdk/0.55.1 keystoneauth1/4.3.1 python-requests/2.25.1 CPython/3.9.10"
...
~~~

Version-Release number of selected component (if applicable):
(overcloud) [stack@undercloud-0 ~]$ cat /etc/rhosp-release 
Red Hat OpenStack Platform release 17.0.1 (Wallaby)
(overcloud) [stack@undercloud-0 ~]$ openstack --version
openstack 5.5.1


How reproducible:
Always


Steps to Reproduce:
Please see [1] and [2].

Actual results:
The request you have made requires authentication. (HTTP 401)

Expected results:
The valid output of the command.

Additional info:

Comment 1 Takashi Kajinami 2023-06-27 05:12:54 UTC
Did you try removing OS_CLOUD fro your customized rc file?
The described example contains OS_CLOUD=overcloud, and that means you are loading the information in cloud config.

We intentionally kept both OS_CLOUD and individual environment parameters to allow legacy clients to load these parameters
but we don't support mixing up different credential by OS_CLOUD and OS_<foo>.

Comment 2 Takemi Asakura 2023-06-27 07:27:43 UTC
Hello Takashi,

Thank you for your prompt response! 
I can confirm that the command executed successfully when I commented `export OS_CLOUD=overcloud`.

Now I understand setting `export OS_CLOUD=overcloud` loads the clouds.yaml file[1]. 
Upon examining the contents of clouds.yaml, I discovered that the overcloud configuration 
is loading the admin project[2].

I created the rc file from overcloudrc, but until RHOSP 17.0.0, we used `OS_CLOUDNAME` 
instead of `OS_CLOUD`. Was `OS_CLOUDNAME` merely specifying the name of the cloud without 
loading clouds.yaml?

Best regards,
--
Takemi Asakura
Technical Support Engineer, OpenStack
Tokyo, Japan

[1] https://docs.openstack.org/python-openstackclient/wallaby/cli/man/openstack.html#environment-variables
~~~
OS_CLOUD
The name of a cloud configuration in clouds.yaml.
~~~

[2] 
~~~
[stack@undercloud-0 ~]$ cat ~/.config/openstack/clouds.yaml 
clouds:
  heat:
    auth_type: none
    endpoint: http://127.0.0.1:8006/v1/admin
  overcloud:
    auth:
      auth_url: http://10.0.0.131:5000
      password: ****
      project_domain_name: Default
      project_name: admin
      user_domain_name: Default
      username: admin
    cacert: ''
    identity_api_version: '3'
    region_name: regionOne
    volume_api_version: '3'
  undercloud:
    auth:
      auth_url: https://192.168.24.2:13000
      password: ****
      project_domain_name: Default
      project_name: admin
      user_domain_name: Default
      username: admin
    cacert: /etc/pki/ca-trust/source/anchors/cm-local-ca.pem
    identity_api_version: '3'
    region_name: regionOne
    volume_api_version: '3'
~~~

Comment 3 Takashi Kajinami 2023-06-27 08:12:18 UTC
(In reply to Takemi Asakura from comment #2)
...
> I created the rc file from overcloudrc, but until RHOSP 17.0.0, we used
> `OS_CLOUDNAME` 
> instead of `OS_CLOUD`. Was `OS_CLOUDNAME` merely specifying the name of the
> cloud without 
> loading clouds.yaml?

You are correct. OS_CLOUDNAME was used to determine the PS1 and is not used by openstackclient.

If OS_CLOUD was causing the problem then we can probably just close this as NOTABUG because
it's "reasonable" that mixing up two credentials in two formats breaks the discovery.

Comment 4 Takemi Asakura 2023-06-27 23:44:14 UTC
Hello Takashi,

Thank you for your response. 
I have understood the functionality of OS_CLOUD and OS_CLOUDNAME with openstackclient. 
The issue was caused by loading clouds.yaml with OS_CLOUD, so please close it as NOTABUG.

Thank you,
Takemi