Note: This bug is displayed in read-only format because the product is no longer active in Red Hat Bugzilla.
This project is now read‑only. Starting Monday, February 2, please use https://ibm-ceph.atlassian.net/ for all bug tracking management.

Bug 1903504

Summary: [GSS][ceph-ansible] rolling-update.yml fails with: TASK [ceph-container-common : container registry authentication]
Product: [Red Hat Storage] Red Hat Ceph Storage Reporter: Geo Jose <gjose>
Component: Ceph-AnsibleAssignee: Dimitri Savineau <dsavinea>
Status: CLOSED ERRATA QA Contact: Ameena Suhani S H <amsyedha>
Severity: medium Docs Contact:
Priority: medium    
Version: 4.1CC: aschoen, ceph-eng-bugs, gabrioux, gmeno, gsitlani, kdreyer, lithomas, mmuench, nthomas, tserlin, vashastr, ykaul
Target Milestone: ---   
Target Release: 4.2z2   
Hardware: x86_64   
OS: Linux   
Whiteboard:
Fixed In Version: ceph-ansible-4.0.54-1.el8cp, ceph-ansible-4.0.54-1.el7cp Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2021-06-15 17:13:09 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Geo Jose 2020-12-02 08:16:39 UTC 
Description of problem:
 * While upgrading to RHCS 4, the rolling-update.yml playbook fails at task 'ceph-container-common : container registry authentication'.

Version-Release number of selected component (if applicable):
 * RHCS 3 to RHCS 4 upgrade.

How reproducible:
 * Upgrade from RHCS 3 to RHCS 4 with newly added grafana-server.

Steps to Reproduce:
1. Install RHCS 3.3 cluster.
  - In ansible inventory file, mention [mons], [mgrs] and [osds] [a].

2. Upgrade to 3.x latest (if needed) [b].

3. Upgrade from 3.x(latest) to 4.x(latest)
  - For RHCS 4, add [grafana-server] section and mention the grafana server details.
  - Update all.yml and osds.yml accordingly and run rolling-update.yml [c].


Reference: 
[a]. https://access.redhat.com/documentation/en-us/red_hat_ceph_storage/3/html/container_guide/deploying-red-hat-ceph-storage-in-containers#installing-a-red-hat-ceph-storage-cluster-in-containers

[b]. https://access.redhat.com/documentation/en-us/red_hat_ceph_storage/3/html/container_guide/upgrading-red-hat-ceph-storage-within-containers

[c]. https://access.redhat.com/documentation/en-us/red_hat_ceph_storage/4/html-single/installation_guide/index#upgrading-a-red-hat-ceph-storage-cluster


Actual results:

 - The play is failing on newly added grafana-server node with the below error:
---
TASK [ceph-container-common : container registry authentication] *************************************************************************************************************
Tuesday 24 November 2020  17:35:19 +0530 (0:00:01.431)   0:01:59.875 ******
fatal: [10.10.10.1]: FAILED! => changed=false
  censored: 'the output has been hidden due to the fact that ''no_log: true'' was specified for this result'
---

Expected results:
 - Upgrade should complete without any error.

Additional info:
 - This issue happens while mentioning grafana-server on a new node where there is no docker package pre-installed.
Comment 1 Geo Jose 2020-12-02 08:18:36 UTC 
- The play is failing at:
~~~
 - name: container registry authentication
  command: '{{ container_binary }} login -u {{ ceph_docker_registry_username }} -p {{ ceph_docker_registry_password }} {{ ceph_docker_registry }}'
  changed_when: false
  no_log: true
~~~
- Since the parameter "no_log: True" is set to the task, there is no verbose error.
 
- While removing the parameter 'no_log: true' from play, we will get the error 'docker service/socket not found'

**Workaround**
 
- Install `docker` manually on the `grafana-server` node and start/enable the service.
~~~
$ sudo yum install docker -y
$ sudo systemctl restart docker.service
$ sudo systemctl enable docker.service
~~~
 
- After starting the docker service, run the playbook again.
Comment 5 Ken Dreyer (Red Hat) 2021-02-24 19:08:23 UTC 
I updated this bug to MODIFIED for RHCS 5.0, but this bug is actually targeted to 4.2 z2, so I will reset this back in order to track fixing it in RHCS 4.
Comment 10 Ameena Suhani S H 2021-06-01 13:18:45 UTC 
Verified using 
ansible-2.9.22-1.el7ae.noarch
ceph-ansible-4.0.56-1.el7cp.noarch
Comment 12 errata-xmlrpc 2021-06-15 17:13:09 UTC 
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (Important: Red Hat Ceph Storage 4.2 Security and Bug Fix Update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2021:2445