Note: This bug is displayed in read-only format because the product is no longer active in Red Hat Bugzilla.

Bug 2055635

Summary: Errors deleting users: sequence item 0: expected str instance, bytes found
Product: Red Hat OpenStack Reporter: Eduardo Olivares <eolivare>
Component: python-memcachedAssignee: OSP Team <rhos-maint>
Status: CLOSED ERRATA QA Contact: nlevinki <nlevinki>
Severity: high Docs Contact:
Priority: high    
Version: 16.1 (Train)CC: dwilde, ekuris, ggrasza, oblaut, rhos-maint
Target Milestone: z9Keywords: AutomationBlocker, Rebase, Triaged
Target Release: 16.1 (Train on RHEL 8.2)Flags: lhh: needinfo-
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: python-memcached-1.59-2.el8ost Doc Type: No Doc Update
Doc Text:
Story Points: ---
Clone Of:
: 2066417 (view as bug list) Environment:
Last Closed: 2022-12-07 20:25:58 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 2066417    

Description Eduardo Olivares 2022-02-17 12:32:21 UTC
Description of problem:
Some tempest tests failed executing their teardown because of the following error:
tempest.lib.exceptions.BadRequest: Bad request
Details: {'code': 400, 'message': 'sequence item 0: expected str instance, bytes found', 'title': 'Bad Request'}

Link to the tempest logs:
https://rhos-ci-jenkins.lab.eng.tlv2.redhat.com/job/DFG-network-networking-ovn-16.1_director-rhel-virthost-3cont_2comp-ipv4-geneve-gate-ovn/90//artifact/tempest-results/tempest-results-neutron_single_threaded.1.html


keystone logs:
http://rhos-ci-logs.lab.eng.tlv2.redhat.com/logs/rcj/DFG-network-networking-ovn-16.1_director-rhel-virthost-3cont_2comp-ipv4-geneve-gate-ovn/90/controller-0/var/log/containers/keystone/keystone.log.gz

Occurrence 1:
2022-02-11 19:46:30.358 189 DEBUG keystone.server.flask.request_processing.middleware.auth_context [req-bf4a399e-d3ad-471c-bdee-3bac0a34ce95 7c0ae35fa7fb4cd88a34c7b6f51c1f43 cfe8694106044c3dab72acd44d4df48e - default default] RBAC: auth_context: {'token': <TokenModel (audit_id=2GrPevCfSbuaByNCjruekQ, audit_chain_id=['2GrPevCfSbuaByNCjruekQ']) at 0x7f7bf9707f98>, 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c0ae35fa7fb4cd88a34c7b6f51c1f43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfe8694106044c3dab72acd44d4df48e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} fill_context /usr/lib/python3.6/site-packages/keystone/server/flask/request_processing/middleware/auth_context.py:479
2022-02-11 19:46:30.358 189 DEBUG keystone.server.flask.request_processing.req_logging [req-bf4a399e-d3ad-471c-bdee-3bac0a34ce95 7c0ae35fa7fb4cd88a34c7b6f51c1f43 cfe8694106044c3dab72acd44d4df48e - default default] REQUEST_METHOD: `DELETE` log_request_info /usr/lib/python3.6/site-packages/keystone/server/flask/request_processing/req_logging.py:27
2022-02-11 19:46:30.359 189 DEBUG keystone.server.flask.request_processing.req_logging [req-bf4a399e-d3ad-471c-bdee-3bac0a34ce95 7c0ae35fa7fb4cd88a34c7b6f51c1f43 cfe8694106044c3dab72acd44d4df48e - default default] SCRIPT_NAME: `` log_request_info /usr/lib/python3.6/site-packages/keystone/server/flask/request_processing/req_logging.py:28
2022-02-11 19:46:30.359 189 DEBUG keystone.server.flask.request_processing.req_logging [req-bf4a399e-d3ad-471c-bdee-3bac0a34ce95 7c0ae35fa7fb4cd88a34c7b6f51c1f43 cfe8694106044c3dab72acd44d4df48e - default default] PATH_INFO: `/v3/users/3fed504c9abd4e1685e29d61435fec4b` log_request_info /usr/lib/python3.6/site-packages/keystone/server/flask/request_processing/req_logging.py:29
2022-02-11 19:46:30.360 189 DEBUG keystone.common.rbac_enforcer.enforcer [req-bf4a399e-d3ad-471c-bdee-3bac0a34ce95 7c0ae35fa7fb4cd88a34c7b6f51c1f43 cfe8694106044c3dab72acd44d4df48e - default default] RBAC: Authorizing `identity:delete_user(user_id=3fed504c9abd4e1685e29d61435fec4b)` enforce_call /usr/lib/python3.6/site-packages/keystone/common/rbac_enforcer/enforcer.py:442
2022-02-11 19:46:30.378 189 DEBUG keystone.common.rbac_enforcer.enforcer [req-bf4a399e-d3ad-471c-bdee-3bac0a34ce95 7c0ae35fa7fb4cd88a34c7b6f51c1f43 cfe8694106044c3dab72acd44d4df48e - default default] RBAC: Authorization granted enforce_call /usr/lib/python3.6/site-packages/keystone/common/rbac_enforcer/enforcer.py:449
2022-02-11 19:46:30.427 189 WARNING keystone.server.flask.application [req-bf4a399e-d3ad-471c-bdee-3bac0a34ce95 7c0ae35fa7fb4cd88a34c7b6f51c1f43 cfe8694106044c3dab72acd44d4df48e - default default] sequence item 0: expected str instance, bytes found


Ocurrence 2:
2022-02-11 19:56:00.745 185 DEBUG keystone.server.flask.request_processing.middleware.auth_context [req-e1147925-db8a-44cc-b949-e1e705ef6339 7c0ae35fa7fb4cd88a34c7b6f51c1f43 cfe8694106044c3dab72acd44d4df48e - default default] RBAC: auth_context: {'token': <TokenModel (audit_id=zv_FdcRZTFSeyEYQ6DPPiA, audit_chain_id=['zv_FdcRZTFSeyEYQ6DPPiA']) at 0x7f7bf8517240>, 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c0ae35fa7fb4cd88a34c7b6f51c1f43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfe8694106044c3dab72acd44d4df48e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} fill_context /usr/lib/python3.6/site-packages/keystone/server/flask/request_processing/middleware/auth_context.py:479
2022-02-11 19:56:00.746 185 DEBUG keystone.server.flask.request_processing.req_logging [req-e1147925-db8a-44cc-b949-e1e705ef6339 7c0ae35fa7fb4cd88a34c7b6f51c1f43 cfe8694106044c3dab72acd44d4df48e - default default] REQUEST_METHOD: `DELETE` log_request_info /usr/lib/python3.6/site-packages/keystone/server/flask/request_processing/req_logging.py:27
2022-02-11 19:56:00.746 185 DEBUG keystone.server.flask.request_processing.req_logging [req-e1147925-db8a-44cc-b949-e1e705ef6339 7c0ae35fa7fb4cd88a34c7b6f51c1f43 cfe8694106044c3dab72acd44d4df48e - default default] SCRIPT_NAME: `` log_request_info /usr/lib/python3.6/site-packages/keystone/server/flask/request_processing/req_logging.py:28
2022-02-11 19:56:00.746 185 DEBUG keystone.server.flask.request_processing.req_logging [req-e1147925-db8a-44cc-b949-e1e705ef6339 7c0ae35fa7fb4cd88a34c7b6f51c1f43 cfe8694106044c3dab72acd44d4df48e - default default] PATH_INFO: `/v3/users/8e97e83a70414a28b2c1417ebedbe726` log_request_info /usr/lib/python3.6/site-packages/keystone/server/flask/request_processing/req_logging.py:29
2022-02-11 19:56:00.747 185 DEBUG keystone.common.rbac_enforcer.enforcer [req-e1147925-db8a-44cc-b949-e1e705ef6339 7c0ae35fa7fb4cd88a34c7b6f51c1f43 cfe8694106044c3dab72acd44d4df48e - default default] RBAC: Authorizing `identity:delete_user(user_id=8e97e83a70414a28b2c1417ebedbe726)` enforce_call /usr/lib/python3.6/site-packages/keystone/common/rbac_enforcer/enforcer.py:442
2022-02-11 19:56:00.770 185 DEBUG keystone.common.rbac_enforcer.enforcer [req-e1147925-db8a-44cc-b949-e1e705ef6339 7c0ae35fa7fb4cd88a34c7b6f51c1f43 cfe8694106044c3dab72acd44d4df48e - default default] RBAC: Authorization granted enforce_call /usr/lib/python3.6/site-packages/keystone/common/rbac_enforcer/enforcer.py:449
2022-02-11 19:56:00.822 185 WARNING keystone.server.flask.application [req-e1147925-db8a-44cc-b949-e1e705ef6339 7c0ae35fa7fb4cd88a34c7b6f51c1f43 cfe8694106044c3dab72acd44d4df48e - default default] sequence item 0: expected str instance, bytes found








Version-Release number of selected component (if applicable):
RHOS-16.1-RHEL-8-20220210.n.1

How reproducible:
not always, but the following job reproduces it often: DFG-network-networking-ovn-16.1_director-rhel-virthost-3cont_2comp-ipv4-geneve-gate-ovn

Steps to Reproduce:
1. run the job mentioned above
2.
3.

Comment 2 Grzegorz Grasza 2022-03-17 15:03:17 UTC
I think we didn't get a full traceback in the server logs, because a bug in keystone. I submitted a fix upstream. If you can, please apply it in the job run, so that we can get some more output.

Comment 18 Lon Hohberger 2022-11-15 18:51:51 UTC
python-memcached 1.59-2 is in the current compose which has passed phase2

Comment 23 errata-xmlrpc 2022-12-07 20:25:58 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (Red Hat OpenStack Platform 16.1.9 bug fix and enhancement advisory), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2022:8795