Bug 1891411

Summary: In LDAP integration, local_id parameter may have bytes data type and it caused an exception because the data type doesn't have encode type.
Product: Red Hat OpenStack Reporter: Keigo Noha <knoha>
Component: openstack-keystoneAssignee: Grzegorz Grasza <ggrasza>
Status: CLOSED ERRATA QA Contact: Jeremy Agee <jagee>
Severity: medium Docs Contact:
Priority: medium    
Version: 16.1 (Train)CC: alee, dwilde, ggrasza, hrybacki, lbragsta, oblaut
Target Milestone: zstreamKeywords: Triaged, ZStream
Target Release: 16.1 (Train on RHEL 8.2)   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: openstack-keystone-16.0.2-1.20210512163310.fb7d545 Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2021-12-09 20:17:24 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Keigo Noha 2020-10-26 07:56:35 UTC
Description of problem:
On LDAP integration in keystone, local_id parameter may have bytes data type and it caused an exception because the data type doesn't have encode type.
In the customer's site, the login through horizon always failed with following error in var/log/containers/httpd/keystone/keystone_wsgi_error.log.

~~~
  File "/usr/lib/python3.6/site-packages/keystone/assignment/core.py", line 851, in _list_effective_role_assignments
    group_ids = self._get_group_ids_for_user_id(user_id)
  File "/usr/lib/python3.6/site-packages/keystone/assignment/core.py", line 88, in _get_group_ids_for_user_id
    x in PROVIDERS.identity_api.list_groups_for_user(user_id)]
  File "/usr/lib/python3.6/site-packages/keystone/common/manager.py", line 116, in wrapped
    __ret_val = __f(*args, **kwargs)
  File "/usr/lib/python3.6/site-packages/keystone/identity/core.py", line 414, in wrapper
    return f(self, *args, **kwargs)
  File "/usr/lib/python3.6/site-packages/keystone/identity/core.py", line 424, in wrapper
    return f(self, *args, **kwargs)
  File "/usr/lib/python3.6/site-packages/keystone/identity/core.py", line 1312, in list_groups_for_user
    ref_list, domain_id, driver, mapping.EntityType.GROUP)
  File "/usr/lib/python3.6/site-packages/keystone/identity/core.py", line 603, in _set_domain_id_and_mapping
    ref, domain_id, driver, entity_type, conf)
  File "/usr/lib/python3.6/site-packages/keystone/identity/core.py", line 693, in _set_domain_id_and_mapping_for_list
    self._insert_new_public_id(local_entity, ref, driver)
  File "/usr/lib/python3.6/site-packages/keystone/identity/core.py", line 619, in _insert_new_public_id
    local_entity, public_id)
  File "/usr/lib/python3.6/site-packages/keystone/common/manager.py", line 116, in wrapped
    __ret_val = __f(*args, **kwargs)
  File "/usr/lib/python3.6/site-packages/keystone/identity/core.py", line 1465, in create_id_mapping
    public_id = self.driver.create_id_mapping(local_entity, public_id)
  File "/usr/lib/python3.6/site-packages/keystone/identity/mapping_backends/sql.py", line 77, in create_id_mapping
    entity)
  File "/usr/lib/python3.6/site-packages/keystone/identity/id_generators/sha256.py", line 25, in generate_public_ID
    m.update(mapping[key].encode('utf-8'))
AttributeError: 'bytes' object has no attribute 'encode'
~~~

Version-Release number of selected component (if applicable):
RHOSP16.1 keystone

How reproducible:
Always

Steps to Reproduce:
1. Setup RHOSP16.1 with LDAP integration in keystone
2.
3.

Actual results:
The login through horizon always failed.

Expected results:
The login succeeds.

Additional info:

In RHOSP16, python-ldap is updated to python3-ldap-3.1.0-5.el8.x86_64.
From Python3, python distinguish str(sequence of characters) and bytes(sequence of bytes).
For the attribute values, python-ldap uses bytes data type to store the data because attributes may contain any type of data.

c.f. https://www.python-ldap.org/en/python-ldap-3.3.0/bytes_mode.html

That means keystone/identity/id_generators/sha256.py should be
~~~
import hashlib
import six

from keystone.identity import generator


class Generator(generator.IDGenerator):

    def generate_public_ID(self, mapping):
        m = hashlib.sha256()
        for key in sorted(mapping.keys()):
            if six.PY3 and isinstance(mapping[key], six.binary_type):
                m.update(mapping[key])
            else:
                m.update(mapping[key].encode('utf-8'))
        return m.hexdigest()
~~~

This modification works fine on the customer's site.

Comment 4 Keigo Noha 2020-11-16 08:51:46 UTC
Hello Raildo,

Thank you for your work in upstream.
Unfortunately, we haven't had a response on upstream yet.
Would you please some reviewers to take a look at the patch?

Thank you,
Keigo

Comment 11 Keigo Noha 2021-04-22 06:28:16 UTC
Hi Lance,

Thank you for your assistance in upstream!

In upstream, the fix is now merged into stable/train branch. 
https://review.opendev.org/c/openstack/keystone/+/781490 

Would you proceed the backport into downstream?

Kind Regards,
Keigo Noha

Comment 32 errata-xmlrpc 2021-12-09 20:17:24 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (Red Hat OpenStack Platform 16.1.7 (Train) bug fix and enhancement advisory), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2021:3762