Bug 1779737

Summary: Google backup driver failing cinder.backup.drivers.gcs.GCSConnectionFailure: Google Cloud Storage connection failure: 'bytes' object has no attribute 'encode'
Product: Red Hat OpenStack Reporter: Tzach Shefi <tshefi>
Component: openstack-cinderAssignee: Cinder Bugs List <cinder-bugs>
Status: CLOSED WONTFIX QA Contact: Tzach Shefi <tshefi>
Severity: medium Docs Contact: RHOS Documentation Team <rhos-docs>
Priority: low    
Version: 16.0 (Train)CC: eharney, gcharot
Target Milestone: ---Keywords: Regression, Triaged
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of:
: 1984887 (view as bug list) Environment:
Last Closed: 2021-11-18 14:55:46 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 1984887, 1984889    
Attachments:
Description Flags
Cinder backup log none

Description Tzach Shefi 2019-12-04 15:48:10 UTC 
Created attachment 1642115 [details]
Cinder backup log

Description of problem: When I try to backup a volume with Google backup driver it fails with $summary. 

Version-Release number of selected component (if applicable):
openstack-cinder-15.0.1-0.20191114132949.b91f514.el8ost.noarch

How reproducible:
First time, probably reproducible. 

Steps to Reproduce:
1. Configure Google backup
2. Create a volume in my case backed by Ceph,shouldn't matter
3. Try to backup the volume:
(overcloud) [stack@undercloud-0 ~]$ cinder backup-create 2b0782a2-88f7-42ed-ad25-51a0c8cc1463
+-----------+--------------------------------------+
| Property  | Value                                |
+-----------+--------------------------------------+
| id        | c9140798-f6a5-458b-8fa6-fdfb37c40d3b |
| name      | None                                 |
| volume_id | 2b0782a2-88f7-42ed-ad25-51a0c8cc1463 |
+-----------+--------------------------------------+


Actual results:
Backup "starts"
(overcloud) [stack@undercloud-0 ~]$ cinder list
+--------------------------------------+------------+------+------+-------------+----------+-------------+
| ID                                   | Status     | Name | Size | Volume Type | Bootable | Attached to |
+--------------------------------------+------------+------+------+-------------+----------+-------------+
| 2b0782a2-88f7-42ed-ad25-51a0c8cc1463 | backing-up | -    | 1    | tripleo     | false    |             |
+--------------------------------------+------------+------+------+-------------+----------+-------------+

but log c-vol log then spits this out: 

5a7c386672 - default default] Exception during message handling: cinder.backup.drivers.gcs.GCSConnectionFailure: Google Cloud Storage connection failure: 'bytes' object has no attribute 'encode'
2019-12-04 15:27:02.236 24 ERROR oslo_messaging.rpc.server Traceback (most recent call last):
2019-12-04 15:27:02.236 24 ERROR oslo_messaging.rpc.server   File "/usr/lib/python3.6/site-packages/cinder/backup/drivers/gcs.py", line 134, in func_wrapper
2019-12-04 15:27:02.236 24 ERROR oslo_messaging.rpc.server     return func(self, *args, **kwargs)
2019-12-04 15:27:02.236 24 ERROR oslo_messaging.rpc.server   File "/usr/lib/python3.6/site-packages/cinder/backup/drivers/gcs.py", line 325, in close
2019-12-04 15:27:02.236 24 ERROR oslo_messaging.rpc.server     md5 = md5.encode('utf-8')
2019-12-04 15:27:02.236 24 ERROR oslo_messaging.rpc.server AttributeError: 'bytes' object has no attribute 'encode'
2019-12-04 15:27:02.236 24 ERROR oslo_messaging.rpc.server 
2019-12-04 15:27:02.236 24 ERROR oslo_messaging.rpc.server During handling of the above exception, another exception occurred:
2019-12-04 15:27:02.236 24 ERROR oslo_messaging.rpc.server 
2019-12-04 15:27:02.236 24 ERROR oslo_messaging.rpc.server Traceback (most recent call last):
2019-12-04 15:27:02.236 24 ERROR oslo_messaging.rpc.server   File "/usr/lib/python3.6/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
2019-12-04 15:27:02.236 24 ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
2019-12-04 15:27:02.236 24 ERROR oslo_messaging.rpc.server   File "/usr/lib/python3.6/site-packages/oslo_messaging/rpc/dispatcher.py", line 274, in dispatch
2019-12-04 15:27:02.236 24 ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
2019-12-04 15:27:02.236 24 ERROR oslo_messaging.rpc.server   File "/usr/lib/python3.6/site-packages/oslo_messaging/rpc/dispatcher.py", line 194, in _do_dispatch
2019-12-04 15:27:02.236 24 ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
2019-12-04 15:27:02.236 24 ERROR oslo_messaging.rpc.server   File "/usr/lib/python3.6/site-packages/cinder/backup/manager.py", line 417, in create_backup
2019-12-04 15:27:02.236 24 ERROR oslo_messaging.rpc.server     self._update_backup_error(backup, six.text_type(err))
2019-12-04 15:27:02.236 24 ERROR oslo_messaging.rpc.server   File "/usr/lib/python3.6/site-packages/oslo_utils/excutils.py", line 220, in __exit__
2019-12-04 15:27:02.236 24 ERROR oslo_messaging.rpc.server     self.force_reraise()
2019-12-04 15:27:02.236 24 ERROR oslo_messaging.rpc.server   File "/usr/lib/python3.6/site-packages/oslo_utils/excutils.py", line 196, in force_reraise
2019-12-04 15:27:02.236 24 ERROR oslo_messaging.rpc.server     six.reraise(self.type_, self.value, self.tb)
2019-12-04 15:27:02.236 24 ERROR oslo_messaging.rpc.server   File "/usr/lib/python3.6/site-packages/six.py", line 693, in reraise
2019-12-04 15:27:02.236 24 ERROR oslo_messaging.rpc.server     raise value
2019-12-04 15:27:02.236 24 ERROR oslo_messaging.rpc.server   File "/usr/lib/python3.6/site-packages/cinder/backup/manager.py", line 406, in create_backup
2019-12-04 15:27:02.236 24 ERROR oslo_messaging.rpc.server     updates = self._run_backup(context, backup, volume)
2019-12-04 15:27:02.236 24 ERROR oslo_messaging.rpc.server   File "/usr/lib/python3.6/site-packages/cinder/backup/manager.py", line 494, in _run_backup
2019-12-04 15:27:02.236 24 ERROR oslo_messaging.rpc.server     tpool.Proxy(device_path))
2019-12-04 15:27:02.236 24 ERROR oslo_messaging.rpc.server   File "/usr/lib/python3.6/site-packages/cinder/backup/chunkeddriver.py", line 626, in backup
2019-12-04 15:27:02.236 24 ERROR oslo_messaging.rpc.server     object_meta, extra_metadata)
2019-12-04 15:27:02.236 24 ERROR oslo_messaging.rpc.server   File "/usr/lib/python3.6/site-packages/cinder/backup/chunkeddriver.py", line 375, in _backup_chunk
2019-12-04 15:27:02.236 24 ERROR oslo_messaging.rpc.server     writer.write(output_data)
2019-12-04 15:27:02.236 24 ERROR oslo_messaging.rpc.server   File "/usr/lib/python3.6/site-packages/eventlet/tpool.py", line 221, in __exit__
2019-12-04 15:27:02.236 24 ERROR oslo_messaging.rpc.server     return proxy_call(self._autowrap, self._obj.__exit__, *exc)
2019-12-04 15:27:02.236 24 ERROR oslo_messaging.rpc.server   File "/usr/lib/python3.6/site-packages/eventlet/tpool.py", line 148, in proxy_call
2019-12-04 15:27:02.236 24 ERROR oslo_messaging.rpc.server     rv = execute(f, *args, **kwargs)
2019-12-04 15:27:02.236 24 ERROR oslo_messaging.rpc.server   File "/usr/lib/python3.6/site-packages/eventlet/tpool.py", line 129, in execute
2019-12-04 15:27:02.236 24 ERROR oslo_messaging.rpc.server     six.reraise(c, e, tb)
2019-12-04 15:27:02.236 24 ERROR oslo_messaging.rpc.server   File "/usr/lib/python3.6/site-packages/six.py", line 693, in reraise
2019-12-04 15:27:02.236 24 ERROR oslo_messaging.rpc.server     raise value
2019-12-04 15:27:02.236 24 ERROR oslo_messaging.rpc.server   File "/usr/lib/python3.6/site-packages/eventlet/tpool.py", line 83, in tworker
2019-12-04 15:27:02.236 24 ERROR oslo_messaging.rpc.server     rv = meth(*args, **kwargs)
2019-12-04 15:27:02.236 24 ERROR oslo_messaging.rpc.server   File "/usr/lib/python3.6/site-packages/cinder/backup/drivers/gcs.py", line 306, in __exit__
2019-12-04 15:27:02.236 24 ERROR oslo_messaging.rpc.server     self.close()
2019-12-04 15:27:02.236 24 ERROR oslo_messaging.rpc.server   File "/usr/lib/python3.6/site-packages/cinder/backup/drivers/gcs.py", line 140, in func_wrapper
2019-12-04 15:27:02.236 24 ERROR oslo_messaging.rpc.server     raise GCSConnectionFailure(reason=err)
2019-12-04 15:27:02.236 24 ERROR oslo_messaging.rpc.server cinder.backup.drivers.gcs.GCSConnectionFailure: Google Cloud Storage connection failure: 'bytes' object has no attribute 'encode'
2019-12-04 15:27:02.236 24 ERROR oslo_messaging.rpc.server 



Expected results:
Backup should complete without errors. 

Additional info: 
We also hit a similar (maybe same) issue when we moved to OSP15/Python3 https://bugzilla.redhat.com/show_bug.cgi?id=1717375
Comment 3 Gregory Charot 2021-11-18 14:55:46 UTC 
Closing won't fix, google backup driver is deprecated and we're planning support removal. no customer case attached