Bug 1956930

Summary: OSP 16.1 ceph minor update failed
Product: Red Hat OpenStack Reporter: Jason Paroly <jparoly>
Component: cephAssignee: Francesco Pantano <fpantano>
Status: CLOSED DUPLICATE QA Contact: Yogev Rabl <yrabl>
Severity: unspecified Docs Contact:
Priority: unspecified    
Version: 16.1 (Train)CC: fpantano, gfidente, jdurgin, lhh, mhicks, pbabbar
Target Milestone: ---Keywords: Regression, Tracking
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2021-06-09 12:55:45 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On: 1955393    
Bug Blocks:    

Description Jason Paroly 2021-05-04 17:04:13 UTC
Description of problem:

The hardprov regression job failed at the minor update of ceph stage

log excerpt:
STDOUT:

2021-05-03 23:12:40 | 2021-05-03 23:12:40.607 289176 ERROR tripleoclient.v1.overcloud_external_update.ExternalUpdateRun   File "/usr/lib/python3.6/site-packages/tripleoclient/v1/overcloud_external_update.py", line 149, in take_action
2021-05-03 23:12:40 | 2021-05-03 23:12:40.607 289176 ERROR tripleoclient.v1.overcloud_external_update.ExternalUpdateRun     priv_key=key)
2021-05-03 23:12:40 | 2021-05-03 23:12:40.607 289176 ERROR tripleoclient.v1.overcloud_external_update.ExternalUpdateRun   File "/usr/lib/python3.6/site-packages/tripleoclient/utils.py", line 1409, in run_update_ansible_action
2021-05-03 23:12:40 | 2021-05-03 23:12:40.607 289176 ERROR tripleoclient.v1.overcloud_external_update.ExternalUpdateRun     verbosity=verbosity, extra_vars=extra_vars)
2021-05-03 23:12:40 | 2021-05-03 23:12:40.607 289176 ERROR tripleoclient.v1.overcloud_external_update.ExternalUpdateRun   File "/usr/lib/python3.6/site-packages/tripleoclient/workflows/package_update.py", line 127, in update_ansible
2021-05-03 23:12:40 | 2021-05-03 23:12:40.607 289176 ERROR tripleoclient.v1.overcloud_external_update.ExternalUpdateRun     raise RuntimeError('Update failed with: {}'.format(payload['message']))
2021-05-03 23:12:40 | 2021-05-03 23:12:40.607 289176 ERROR tripleoclient.v1.overcloud_external_update.ExternalUpdateRun RuntimeError: Update failed with: Ansible failed, check log at /var/log/containers/mistral/package_update.log.
2021-05-03 23:12:40 | 2021-05-03 23:12:40.607 289176 ERROR tripleoclient.v1.overcloud_external_update.ExternalUpdateRun 
2021-05-03 23:12:40 | 2021-05-03 23:12:40.613 289176 ERROR openstack [-] Update failed with: Ansible failed, check log at /var/log/containers/mistral/package_update.log.: RuntimeError: Update failed with: Ansible failed, check log at /var/log/containers/mistral/package_update.log.
2021-05-03 23:12:40 | 2021-05-03 23:12:40.614 289176 INFO osc_lib.shell [-] END return value: 1

TASK [tripleo-upgrade : fail Ceph update] **************************************
task path: /home/rhos-ci/jenkins/workspace/DFG-hardware_provisioning-rqci-16.1_director-virthost-3cont_2comp_3ceph_2ironrole_2ironic-ipv4-geneve-composable-minor_update-RHELOSP-60960/infrared/plugins/tripleo-upgrade/infrared_plugin/roles/tripleo-upgrade/tasks/update/ceph_update_run.yml:17
Monday 03 May 2021  23:12:42 +0000 (0:00:01.354)       2:52:39.261 ************ 
fatal: [undercloud-0]: FAILED! => {
    "changed": false
}

MSG:

Ceph minor update failed


Version-Release number of selected component (if applicable):
16.1


How reproducible:
every time

Steps to Reproduce:
1. deploy OSP 16.1
2. do a minor update with or without composable roles
3.

Actual results:
update failed

Expected results:
update passed

Additional info: