Bug 1564232 - ceph-ansible unable to upgrade ceph-base when ceph-test is installed [NEEDINFO]
Summary: ceph-ansible unable to upgrade ceph-base when ceph-test is installed
Status: CLOSED ERRATA
Alias: None
Product: Red Hat Ceph Storage
Classification: Red Hat
Component: Ceph-Ansible
Version: 3.0
Hardware: Unspecified
OS: Unspecified
medium
medium
Target Milestone: z2
: 3.0
Assignee: Ken Dreyer (Red Hat)
QA Contact: Parikshith
URL:
Whiteboard:
Keywords: Regression
Depends On:
Blocks: 1557269 1600697
TreeView+ depends on / blocked
 
Reported: 2018-04-05 18:42 UTC by Coady LaCroix
Modified: 2018-07-12 19:43 UTC (History)
12 users (show)

(edit)
.Upgrading a node in a Ceph cluster installed with `ceph-test` packages must have `ceph_test = true` in `/etc/ansible/hosts` file

When using the `ceph-ansible` `rolling_update.yml` playbook to upgrade a Ceph node in a RHEL cluster that was installed with `ceph-test` packages, set `ceph_test = true` in the `/etc/ansible/hosts` file for each node that has `ceph-test` package installed:

----
[mons]
mon_node1 ceph_test=true

[osds]
osd_node1 ceph_test=true
----

Not applicable for clients and MDS nodes.
Clone Of:
(edit)
Last Closed: 2018-04-26 17:38:40 UTC
pbyregow: needinfo? (kdreyer)


Attachments (Terms of Use)
error during install_redhat_packages (5.54 KB, text/plain)
2018-04-05 18:42 UTC, Coady LaCroix
no flags Details


External Trackers
Tracker ID Priority Status Summary Last Updated
Red Hat Product Errata RHBA-2018:1259 None None None 2018-04-26 17:39 UTC
Github ceph ceph-ansible pull 2500 None None None 2018-04-06 21:02 UTC

Description Coady LaCroix 2018-04-05 18:42:10 UTC
Created attachment 1417865 [details]
error during install_redhat_packages

Description of problem:
During an upgrade from 3.0 to 3.x using ceph-ansible the install_redhat_packages task fails with the attached error. Essentially, ceph-base is failing to install due to conflicts with the ceph-test package.

When manually attempting to install ceph-base on the node, I see the following:

Transaction check error:
  file /usr/bin/ceph-kvstore-tool from install of ceph-base-2:12.2.4-4.el7cp.x86_64 conflicts with file from package ceph-test-2:12.2.1-46.el7cp.x86_64
  file /usr/bin/ceph-monstore-tool from install of ceph-mon-2:12.2.4-4.el7cp.x86_64 conflicts with file from package ceph-test-2:12.2.1-46.el7cp.x86_64

Error Summary


Version-Release number of selected component (if applicable):


How reproducible:
Every attempt at upgrading to latest 3.x

Steps to Reproduce:
1. Install ceph 3.0 using ceph-ansible
2. Upgrade to latest using ceph-ansible

Actual results:
Failure during install_redhat_packages

Expected results:
Successful upgrade to latest ceph version.

Additional info:

Comment 3 Ken Dreyer (Red Hat) 2018-04-05 19:33:13 UTC
Discussed with Coady & Vasu in #rh-ceph-qe. The problem is that when ceph_test is true, we try to upgrade ceph-base before upgrading ceph-test.

Comment 7 Ken Dreyer (Red Hat) 2018-04-06 21:02:43 UTC
I can't reproduce on Ubuntu, so https://github.com/ceph/ceph-ansible/pull/2500 is just for RHEL for now.

Comment 9 Coady LaCroix 2018-04-09 21:48:34 UTC
I have reproduced this issue on ubuntu as well. Logs are located here: http://magna002.ceph.redhat.com/cephci-jenkins/cephci-run-1523306398/upgrade_to_3.x_0.log. Initial install logs can also be found in that same directory.

Comment 10 Ken Dreyer (Red Hat) 2018-04-09 22:47:06 UTC
10.2.10-1xenial is in that log, and that is an upstream (download.ceph.com) version number scheme. Would you please let us know the entire contents of /etc/apt.sources.d/* ?

Comment 11 Ramakrishnan Periyasamy 2018-04-10 06:42:58 UTC
Observing conflict failure while upgrading cluster to 3.0z2 package.
Moving this bug to assigned state.

Steps followed:
1. Configured cluster with 3.0 live package.
2. Installed ceph-test pacakge manually in cluster
3. Tried to upgrade the cluster to latest 3.0z2 package.

Note: ceph-test pacakges are available in live, also in latest 3.0z2 repos, package versions also correct.

ceph-ansible version: ceph-ansible-3.0.30-1.el7cp.noarch

ceph-ansible failure message:

TASK [ceph-common : install redhat ceph-common] *************************************
fatal: [magna034]: FAILED! => {"changed": true, "msg": "\n\nTransaction check error:\n  file /usr/bin/ceph-kvstore-tool from install of ceph-base-2:12.2.4-6.el7cp.x86_64 conflicts with file from package ceph-test-2:12.2.1-46.el7cp.x86_64\n  file /usr/bin/ceph-monstore-tool from install of ceph-mon-2:12.2.4-6.el7cp.x86_64 conflicts with file from package ceph-test-2:12.2.1-46.el7cp.x86_64\n\nError Summary\n-------------\n\n", "rc": 1, "results": ["Loaded plugins: fastestmirror, langpacks, priorities, product-id, search-\n              : disabled-repos, subscription-manager\nLoading mirror speeds from cached hostfile\nResolving Dependencies\n--> Running transaction check\n---> Package ceph-common.x86_64 2:12.2.1-46.el7cp will be updated\n--> Processing Dependency: ceph-common = 2:12.2.1-46.el7cp for package: 2:ceph-base-12.2.1-46.el7cp.x86_64\n---> Package ceph-common.x86_64 2:12.2.4-6.el7cp will be an update\n--> Processing Dependency: libcephfs2 = 2:12.2.4-6.el7cp for package: 2:ceph-common-12.2.4-6.el7cp.x86_64\n--> Processing Dependency: librados2 = 2:12.2.4-6.el7cp for package: 2:ceph-common-12.2.4-6.el7cp.x86_64\n--> Processing Dependency: librbd1 = 2:12.2.4-6.el7cp for package: 2:ceph-common-12.2.4-6.el7cp.x86_64\n--> Processing Dependency: python-cephfs = 2:12.2.4-6.el7cp for package: 2:ceph-common-12.2.4-6.el7cp.x86_64\n--> Processing Dependency: python-rados = 2:12.2.4-6.el7cp for package: 2:ceph-common-12.2.4-6.el7cp.x86_64\n--> Processing Dependency: python-rbd = 2:12.2.4-6.el7cp for package: 2:ceph-common-12.2.4-6.el7cp.x86_64\n--> Processing Dependency: python-rgw = 2:12.2.4-6.el7cp for package: 2:ceph-common-12.2.4-6.el7cp.x86_64\n--> Running transaction check\n---> Package ceph-base.x86_64 2:12.2.1-46.el7cp will be updated\n--> Processing Dependency: ceph-base = 2:12.2.1-46.el7cp for package: 2:ceph-mgr-12.2.1-46.el7cp.x86_64\n--> Processing Dependency: ceph-base = 2:12.2.1-46.el7cp for package: 2:ceph-mon-12.2.1-46.el7cp.x86_64\n--> Processing Dependency: ceph-base = 2:12.2.1-46.el7cp for package: 2:ceph-selinux-12.2.1-46.el7cp.x86_64\n--> Processing Dependency: ceph-base = 2:12.2.1-46.el7cp for package: 2:ceph-selinux-12.2.1-46.el7cp.x86_64\n---> Package ceph-base.x86_64 2:12.2.4-6.el7cp will be an update\n--> Processing Dependency: librgw2 = 2:12.2.4-6.el7cp for package: 2:ceph-base-12.2.4-6.el7cp.x86_64\n---> Package libcephfs2.x86_64 2:12.2.1-46.el7cp will be updated\n---> Package libcephfs2.x86_64 2:12.2.4-6.el7cp will be an update\n---> Package librados2.x86_64 2:12.2.1-46.el7cp will be updated\n--> Processing Dependency: librados2 = 2:12.2.1-46.el7cp for package: 2:libradosstriper1-12.2.1-46.el7cp.x86_64\n---> Package librados2.x86_64 2:12.2.4-6.el7cp will be an update\n---> Package librbd1.x86_64 2:12.2.1-46.el7cp will be updated\n---> Package librbd1.x86_64 2:12.2.4-6.el7cp will be an update\n---> Package python-cephfs.x86_64 2:12.2.1-46.el7cp will be updated\n---> Package python-cephfs.x86_64 2:12.2.4-6.el7cp will be an update\n---> Package python-rados.x86_64 2:12.2.1-46.el7cp will be updated\n---> Package python-rados.x86_64 2:12.2.4-6.el7cp will be an update\n---> Package python-rbd.x86_64 2:12.2.1-46.el7cp will be updated\n---> Package python-rbd.x86_64 2:12.2.4-6.el7cp will be an update\n---> Package python-rgw.x86_64 2:12.2.1-46.el7cp will be updated\n---> Package python-rgw.x86_64 2:12.2.4-6.el7cp will be an update\n--> Running transaction check\n---> Package ceph-mgr.x86_64 2:12.2.1-46.el7cp will be updated\n---> Package ceph-mgr.x86_64 2:12.2.4-6.el7cp will be an update\n---> Package ceph-mon.x86_64 2:12.2.1-46.el7cp will be updated\n---> Package ceph-mon.x86_64 2:12.2.4-6.el7cp will be an update\n---> Package ceph-selinux.x86_64 2:12.2.1-46.el7cp will be updated\n---> Package ceph-selinux.x86_64 2:12.2.4-6.el7cp will be an update\n---> Package libradosstriper1.x86_64 2:12.2.1-46.el7cp will be updated\n---> Package libradosstriper1.x86_64 2:12.2.4-6.el7cp will be an update\n---> Package librgw2.x86_64 2:12.2.1-46.el7cp will be updated\n---> Package librgw2.x86_64 2:12.2.4-6.el7cp will be an update\n--> Finished Dependency Resolution\n\nDependencies Resolved\n\n================================================================================\n Package          Arch   Version          Repository                       Size\n================================================================================\nUpdating:\n ceph-common      x86_64 2:12.2.4-6.el7cp download-node-02.eng.bos.redhat.com_composes_auto_ceph-3.0-rhel-7_latest-RHCEPH-3-RHEL-7_compose_MON_x86_64_os_\n                                                                           15 M\nUpdating for dependencies:\n ceph-base        x86_64 2:12.2.4-6.el7cp download-node-02.eng.bos.redhat.com_composes_auto_ceph-3.0-rhel-7_latest-RHCEPH-3-RHEL-7_compose_MON_x86_64_os_\n                                                                          3.9 M\n ceph-mgr         x86_64 2:12.2.4-6.el7cp download-node-02.eng.bos.redhat.com_composes_auto_ceph-3.0-rhel-7_latest-RHCEPH-3-RHEL-7_compose_MON_x86_64_os_\n                                                                          3.6 M\n ceph-mon         x86_64 2:12.2.4-6.el7cp download-node-02.eng.bos.redhat.com_composes_auto_ceph-3.0-rhel-7_latest-RHCEPH-3-RHEL-7_compose_MON_x86_64_os_\n                                                                          5.0 M\n ceph-selinux     x86_64 2:12.2.4-6.el7cp download-node-02.eng.bos.redhat.com_composes_auto_ceph-3.0-rhel-7_latest-RHCEPH-3-RHEL-7_compose_MON_x86_64_os_\n                                                                           37 k\n libcephfs2       x86_64 2:12.2.4-6.el7cp download-node-02.eng.bos.redhat.com_composes_auto_ceph-3.0-rhel-7_latest-RHCEPH-3-RHEL-7_compose_MON_x86_64_os_\n                                                                          446 k\n librados2        x86_64 2:12.2.4-6.el7cp download-node-02.eng.bos.redhat.com_composes_auto_ceph-3.0-rhel-7_latest-RHCEPH-3-RHEL-7_compose_MON_x86_64_os_\n                                                                          2.9 M\n libradosstriper1 x86_64 2:12.2.4-6.el7cp download-node-02.eng.bos.redhat.com_composes_auto_ceph-3.0-rhel-7_latest-RHCEPH-3-RHEL-7_compose_MON_x86_64_os_\n                                                                          346 k\n librbd1          x86_64 2:12.2.4-6.el7cp download-node-02.eng.bos.redhat.com_composes_auto_ceph-3.0-rhel-7_latest-RHCEPH-3-RHEL-7_compose_MON_x86_64_os_\n                                                                          1.1 M\n librgw2          x86_64 2:12.2.4-6.el7cp download-node-02.eng.bos.redhat.com_composes_auto_ceph-3.0-rhel-7_latest-RHCEPH-3-RHEL-7_compose_MON_x86_64_os_\n                                                                          1.7 M\n python-cephfs    x86_64 2:12.2.4-6.el7cp download-node-02.eng.bos.redhat.com_composes_auto_ceph-3.0-rhel-7_latest-RHCEPH-3-RHEL-7_compose_MON_x86_64_os_\n                                                                           98 k\n python-rados     x86_64 2:12.2.4-6.el7cp download-node-02.eng.bos.redhat.com_composes_auto_ceph-3.0-rhel-7_latest-RHCEPH-3-RHEL-7_compose_MON_x86_64_os_\n                                                                          189 k\n python-rbd       x86_64 2:12.2.4-6.el7cp download-node-02.eng.bos.redhat.com_composes_auto_ceph-3.0-rhel-7_latest-RHCEPH-3-RHEL-7_compose_MON_x86_64_os_\n                                                                          122 k\n python-rgw       x86_64 2:12.2.4-6.el7cp download-node-02.eng.bos.redhat.com_composes_auto_ceph-3.0-rhel-7_latest-RHCEPH-3-RHEL-7_compose_MON_x86_64_os_\n                                                                           89 k\n\nTransaction Summary\n================================================================================\nUpgrade  1 Package (+13 Dependent packages)\n\nTotal download size: 34 M\nDownloading packages:\nDelta RPMs disabled because /usr/bin/applydeltarpm not installed.\n--------------------------------------------------------------------------------\nTotal                                              9.2 MB/s |  34 MB  00:03     \nRunning transaction check\nRunning transaction test\n"]}

PLAY RECAP **************************************************************************
localhost                  : ok=1    changed=0    unreachable=0    failed=0   
magna034                   : ok=31   changed=1    unreachable=0    failed=1

Comment 18 Parikshith 2018-04-12 17:01:09 UTC
Updated the doc text. Moving this bz to verified.

Comment 22 errata-xmlrpc 2018-04-26 17:38:40 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2018:1259


Note You need to log in before you can comment on or make changes to this bug.