Bug 1856711
| Summary: | [updates] overcloud update failed on ceph update - inspect ceph mon container | ||||||
|---|---|---|---|---|---|---|---|
| Product: | [Red Hat Storage] Red Hat Ceph Storage | Reporter: | Ronnie Rasouli <rrasouli> | ||||
| Component: | Ceph-Ansible | Assignee: | Guillaume Abrioux <gabrioux> | ||||
| Status: | CLOSED DUPLICATE | QA Contact: | Vasishta <vashastr> | ||||
| Severity: | high | Docs Contact: | |||||
| Priority: | high | ||||||
| Version: | 3.2 | CC: | aschoen, ceph-eng-bugs, fpantano, gabrioux, gmeno, jdurgin, johfulto, lhh, nthomas, ykaul | ||||
| Target Milestone: | z7 | ||||||
| Target Release: | 3.3 | ||||||
| Hardware: | Unspecified | ||||||
| OS: | Unspecified | ||||||
| Whiteboard: | |||||||
| Fixed In Version: | Doc Type: | If docs needed, set a value | |||||
| Doc Text: | Story Points: | --- | |||||
| Clone Of: | Environment: | ||||||
| Last Closed: | 2020-09-08 22:59:01 UTC | Type: | Bug | ||||
| Regression: | --- | Mount Type: | --- | ||||
| Documentation: | --- | CRM: | |||||
| Verified Versions: | Category: | --- | |||||
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |||||
| Cloudforms Team: | --- | Target Upstream Version: | |||||
| Embargoed: | |||||||
| Attachments: |
|
||||||
|
Description
Ronnie Rasouli
2020-07-14 09:26:37 UTC
Please attach /var/log/mistral/ceph-install-workflow.log from your undercloud to this BZ. Created attachment 1701758 [details]
/var/log/mistral/ceph-install-workflow.log
2020-07-12 13:18:18,166 p=24136 u=mistral | TASK [ceph-mon : wait for monitor socket to exist] *****************************
2020-07-12 13:18:18,166 p=24136 u=mistral | task path: /usr/share/ceph-ansible/roles/ceph-mon/tasks/docker/main.yml:12
2020-07-12 13:18:18,166 p=24136 u=mistral | Sunday 12 July 2020 13:18:18 -0400 (0:00:00.988) 0:01:49.068 ***********
2020-07-12 13:18:19,236 p=24136 u=mistral | FAILED - RETRYING: wait for monitor socket to exist (5 retries left).
2020-07-12 13:18:34,748 p=24136 u=mistral | FAILED - RETRYING: wait for monitor socket to exist (4 retries left).
2020-07-12 13:18:50,213 p=24136 u=mistral | FAILED - RETRYING: wait for monitor socket to exist (3 retries left).
2020-07-12 13:19:05,669 p=24136 u=mistral | FAILED - RETRYING: wait for monitor socket to exist (2 retries left).
2020-07-12 13:19:21,123 p=24136 u=mistral | FAILED - RETRYING: wait for monitor socket to exist (1 retries left).
2020-07-12 13:19:36,635 p=24136 u=mistral | fatal: [192.168.24.20]: FAILED! => {"attempts": 5, "changed": true, "cmd": ["docker", "exec", "ceph-mon-controller-1", "sh", "-c", "stat /var/run/ceph/ceph-mon.controller-1.asok || stat /var/run/ceph/ceph-mon.controller-1.redhat.local.asok"], "delta": "0:00:00.162916", "end": "2020-07-12 17:19:36.584472", "msg": "non-zero return code", "rc": 1, "start": "2020-07-12 17:19:36.421556", "stderr": "stat: cannot stat '/var/run/ceph/ceph-mon.controller-1.asok': No such file or directory\nstat: cannot stat '/var/run/ceph/ceph-mon.controller-1.redhat.local.asok': No such file or directory", "stderr_lines": ["stat: cannot stat '/var/run/ceph/ceph-mon.controller-1.asok': No such file or directory", "stat: cannot stat '/var/run/ceph/ceph-mon.controller-1.redhat.local.asok': No such file or directory"], "stdout": "", "stdout_lines": []}
At first I thought bz 1846830 was a duplicate of this (same symptoms) but the fixed in of bz 1846830 is v3.2.44 and we're hitting having the same symptoms in this bug with v3.2.45 The needinfo request[s] on this closed bug have been removed as they have been unresolved for 1000 days |