Bug 1760882

Summary: Gnocchi database error during the overcloud deployment on osp13.
Product: Red Hat OpenStack Reporter: mbollo
Component: openstack-tripleo-heat-templatesAssignee: RHOS Maint <rhos-maint>
Status: CLOSED DUPLICATE QA Contact: Sasha Smolyak <ssmolyak>
Severity: high Docs Contact:
Priority: unspecified    
Version: 13.0 (Queens)CC: mburns, michele
Target Milestone: ---   
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2019-10-12 08:14:34 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description mbollo 2019-10-11 15:19:10 UTC
Description of problem:


During the deployment of osp13 we got an redis connection error, because it tried to use redis.

The error is:

2019-10-10 19:04:03,942 [22] CRITICAL root: Traceback (most recent call last):
  File "/usr/bin/gnocchi-upgrade", line 10, in <module>
    sys.exit(upgrade())
  File "/usr/lib/python2.7/site-packages/gnocchi/cli/manage.py", line 73, in upgrade
    i.upgrade(conf.sacks_number)
  File "/usr/lib/python2.7/site-packages/gnocchi/incoming/__init__.py", line 71, in upgrade
    self.set_storage_settings(num_sacks)
  File "/usr/lib/python2.7/site-packages/gnocchi/incoming/redis.py", line 38, in set_storage_settings
    self._client.hset(self.CFG_PREFIX, self.CFG_SACKS, num_sacks)
  File "/usr/lib/python2.7/site-packages/redis/client.py", line 1853, in hset
    return self.execute_command('HSET', name, key, value)
  File "/usr/lib/python2.7/site-packages/redis/client.py", line 570, in execute_command
    connection.send_command(*args)
  File "/usr/lib/python2.7/site-packages/redis/connection.py", line 556, in send_command
    self.send_packed_command(self.pack_command(*args))
  File "/usr/lib/python2.7/site-packages/redis/connection.py", line 532, in send_packed_command
    self.connect()
  File "/usr/lib/python2.7/site-packages/redis/connection.py", line 440, in connect
    self.on_connect()
  File "/usr/lib/python2.7/site-packages/redis/connection.py", line 508, in on_connect
    if nativestr(self.read_response()) != 'OK':
  File "/usr/lib/python2.7/site-packages/redis/connection.py", line 569, in read_response
    response = self._parser.read_response()
  File "/usr/lib/python2.7/site-packages/redis/connection.py", line 224, in read_response
    response = self._buffer.readline()
  File "/usr/lib/python2.7/site-packages/redis/connection.py", line 162, in readline
    self._read_from_socket()
  File "/usr/lib/python2.7/site-packages/redis/connection.py", line 137, in _read_from_socket
    (e.args,))
ConnectionError: Error while reading from socket: ('Connection closed by server.',)

You can see the log here:

http://cougar11.scl.lab.tlv.redhat.com/DFG-upgrades-updates-13-from-7.5-latest-HA-ipv4/9/controller-0.tar.gz?controller-0/var/log/extra/containers/containers/gnocchi_db_sync/stdout.log


How reproducible:

To reproduce the error, you simply rebuild the job:

http://staging-jenkins2-qe-playground.usersys.redhat.com/view/DFG/view/upgrades/view/update/job/DFG-upgrades-updates-13-from-7.5-latest-HA-ipv4/9/rebuild/parameterized

Be careful to have the gerrit change 470868 from infrared, otherwise you will get another error.


Actual results:

Deployment failed on the undercloud.

Expected results:

Deployment succeed.

Comment 1 Michele Baldessari 2019-10-12 08:14:34 UTC

*** This bug has been marked as a duplicate of bug 1745857 ***