Bug 2026582 - [cee/sd][ceph-iscsi] Upgrading ceph-iscsi from RHCS 3 to RHCS 4 fails if ceph-iscsi-tools is installed in the cluster.
Summary: [cee/sd][ceph-iscsi] Upgrading ceph-iscsi from RHCS 3 to RHCS 4 fails if ceph...
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat Ceph Storage
Classification: Red Hat Storage
Component: iSCSI
Version: 4.3
Hardware: x86_64
OS: Linux
unspecified
medium
Target Milestone: ---
: 5.0z4
Assignee: Xiubo Li
QA Contact: Preethi
URL:
Whiteboard:
Depends On:
Blocks: 1959686
TreeView+ depends on / blocked
 
Reported: 2021-11-25 08:25 UTC by Lijo Stephen Thomas
Modified: 2022-04-05 10:32 UTC (History)
9 users (show)

Fixed In Version: ceph-iscsi-3.5-2.el8cp
Doc Type: Bug Fix
Doc Text:
.The upgrade of the storage cluster with iSCSI configured now works as expected Previously, the upgrade of the storage cluster with iSCSI configured would fail as the latest ceph-iscsi packages would not have the ceph-iscsi-tools packages that were deprecated. With this release, the `ceph-iscsi-tools` package is marked as obsolete in the RPM specification file and the upgrade succeeds as expected.
Clone Of:
: 2047694 (view as bug list)
Environment:
Last Closed: 2022-02-08 13:01:20 UTC
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Github ceph ceph-iscsi pull 250 0 None Merged spec: mark the ceph-iscsi-tools package deprecated 2022-01-07 10:03:16 UTC
Red Hat Issue Tracker RHCEPH-2414 0 None None None 2021-11-25 08:26:16 UTC
Red Hat Knowledge Base (Solution) 6827391 0 None None None 2022-04-05 10:32:54 UTC
Red Hat Product Errata RHBA-2022:0466 0 None None None 2022-02-08 13:01:44 UTC

Description Lijo Stephen Thomas 2021-11-25 08:25:29 UTC
Description of problem:
-----------------------
While upgrading the RHCS 3.x cluster to RHCS 4.x, if we have iscsi configured, then the upgarde fails if 'ceph-iscsi-tools' package exists on the iscsi nodes.



Version-Release number of selected component (if applicable):
RHCS 4.x


How reproducible:
-----------------
Every time


Steps to Reproduce:
-------------------
1. Install RHCS 3.x with iscsi configured via local repository.
2. Install ceph-iscsi-tools on the iscsi nodes.
2. Upgrade the cluster to RHCS 4.x. It fails to upgrade ceph-iscsi package if we have already installed ceph-iscsi-tools in RHCS 3.x.



Actual results:
---------------
Upgrading ceph-iscsi package fails


Expected results:
-----------------
Upgrading ceph-iscsi package should be successful.

Comment 1 Lijo Stephen Thomas 2021-11-25 08:27:36 UTC
Additional Info:
----------------

From Lab environment:
---------------------

[root@ceph72 ~]# yum repolist
Loaded plugins: product-id, search-disabled-repos, subscription-manager
repo id                                                           repo name                                                       status
lo-rhel-7-server-extras-rpms                                      rhel-7-server-extras-rpms                                         160
lo-rhel-7-server-rhceph-4-mon-rpms                                rhel-7-server-rhceph-4-mon-rpms                                    54
lo-rhel-7-server-rhceph-4-osd-rpms                                rhel-7-server-rhceph-4-osd-rpms                                    29
lo-rhel-7-server-rhceph-4-tools-rpms                              rhel-7-server-rhceph-4-tools-rpms                                  55
lo-rhel-7-server-rpms                                             rhel-7-server-rpms                                              5,493
repolist: 5,791


- ceph-iscsi packages on the iscsi nodes:
~~~
[root@ceph72 ~]# rpm -qa | grep iscsi
ceph-iscsi-config-2.6-23.el7cp.noarch
ceph-iscsi-tools-2.1-3.el7cp.noarch
ceph-iscsi-cli-2.7-17.el7cp.noarch
~~~


- Try upgrading the ceph-iscsi package
~~~
[root@ceph72 ~]# yum update ceph-iscsi
Loaded plugins: product-id, search-disabled-repos, subscription-manager
Resolving Dependencies
--> Running transaction check
---> Package ceph-iscsi.noarch 0:3.4-4.el7cp will be obsoleting
--> Processing Dependency: python-cryptography for package: ceph-iscsi-3.4-4.el7cp.noarch
---> Package ceph-iscsi-cli.noarch 0:2.7-17.el7cp will be obsoleted
---> Package ceph-iscsi-config.noarch 0:2.6-23.el7cp will be obsoleted
--> Processing Dependency: ceph-iscsi-config >= 2.3-1 for package: ceph-iscsi-tools-2.1-3.el7cp.noarch
--> Running transaction check
---> Package ceph-iscsi-config.noarch 0:2.6-23.el7cp will be obsoleted
--> Processing Dependency: ceph-iscsi-config >= 2.3-1 for package: ceph-iscsi-tools-2.1-3.el7cp.noarch
---> Package python2-cryptography.x86_64 0:1.7.2-2.el7 will be installed
--> Processing Dependency: python-pyasn1 >= 0.1.8 for package: python2-cryptography-1.7.2-2.el7.x86_64
--> Processing Dependency: python-idna >= 2.0 for package: python2-cryptography-1.7.2-2.el7.x86_64
--> Processing Dependency: python-cffi >= 1.4.1 for package: python2-cryptography-1.7.2-2.el7.x86_64
--> Processing Dependency: python-enum34 for package: python2-cryptography-1.7.2-2.el7.x86_64
--> Running transaction check
---> Package ceph-iscsi-config.noarch 0:2.6-23.el7cp will be obsoleted
--> Processing Dependency: ceph-iscsi-config >= 2.3-1 for package: ceph-iscsi-tools-2.1-3.el7cp.noarch
---> Package python-cffi.x86_64 0:1.6.0-5.el7 will be installed
--> Processing Dependency: python-pycparser for package: python-cffi-1.6.0-5.el7.x86_64
---> Package python-enum34.noarch 0:1.0.4-1.el7 will be installed
---> Package python-idna.noarch 0:2.4-1.el7 will be installed
---> Package python2-pyasn1.noarch 0:0.1.9-7.el7 will be installed
--> Running transaction check
---> Package ceph-iscsi-config.noarch 0:2.6-23.el7cp will be obsoleted
--> Processing Dependency: ceph-iscsi-config >= 2.3-1 for package: ceph-iscsi-tools-2.1-3.el7cp.noarch
---> Package python-pycparser.noarch 0:2.14-1.el7 will be installed
--> Processing Dependency: python-ply for package: python-pycparser-2.14-1.el7.noarch
--> Running transaction check
---> Package ceph-iscsi-config.noarch 0:2.6-23.el7cp will be obsoleted
--> Processing Dependency: ceph-iscsi-config >= 2.3-1 for package: ceph-iscsi-tools-2.1-3.el7cp.noarch
---> Package python-ply.noarch 0:3.4-11.el7 will be installed
--> Finished Dependency Resolution
Error: Package: ceph-iscsi-tools-2.1-3.el7cp.noarch (@rhel-7-server-rhceph-3-tools-rpms)
           Requires: ceph-iscsi-config >= 2.3-1
           Removing: ceph-iscsi-config-2.6-23.el7cp.noarch (@rhel-7-server-rhceph-3-tools-rpms)
               ceph-iscsi-config = 2.6-23.el7cp
           Obsoleted By: ceph-iscsi-3.4-4.el7cp.noarch (lo-rhel-7-server-rhceph-4-tools-rpms)
               Not found
**********************************************************************
yum can be configured to try to resolve such errors by temporarily enabling
disabled repos and searching for missing dependencies.
To enable this functionality please set 'notify_only=0' in /etc/yum/pluginconf.d/search-disabled-repos.conf
**********************************************************************

Error: Package: ceph-iscsi-tools-2.1-3.el7cp.noarch (@rhel-7-server-rhceph-3-tools-rpms)
           Requires: ceph-iscsi-config >= 2.3-1
           Removing: ceph-iscsi-config-2.6-23.el7cp.noarch (@rhel-7-server-rhceph-3-tools-rpms)
               ceph-iscsi-config = 2.6-23.el7cp
           Obsoleted By: ceph-iscsi-3.4-4.el7cp.noarch (lo-rhel-7-server-rhceph-4-tools-rpms)
               Not found
 You could try using --skip-broken to work around the problem
 You could try running: rpm -Va --nofiles --nodigest



- Workaround - Manually remove ceph-iscsi-tools
~~~
[root@ceph72 ~]# rpm -e --nodeps ceph-iscsi-tools
~~~


[root@ceph72 ~]# yum update ceph-iscsi
Loaded plugins: product-id, search-disabled-repos, subscription-manager
Resolving Dependencies
--> Running transaction check
---> Package ceph-iscsi.noarch 0:3.4-4.el7cp will be obsoleting
--> Processing Dependency: python-cryptography for package: ceph-iscsi-3.4-4.el7cp.noarch
---> Package ceph-iscsi-cli.noarch 0:2.7-17.el7cp will be obsoleted
---> Package ceph-iscsi-config.noarch 0:2.6-23.el7cp will be obsoleted
--> Running transaction check
---> Package python2-cryptography.x86_64 0:1.7.2-2.el7 will be installed
--> Processing Dependency: python-pyasn1 >= 0.1.8 for package: python2-cryptography-1.7.2-2.el7.x86_64
--> Processing Dependency: python-idna >= 2.0 for package: python2-cryptography-1.7.2-2.el7.x86_64
--> Processing Dependency: python-cffi >= 1.4.1 for package: python2-cryptography-1.7.2-2.el7.x86_64
--> Processing Dependency: python-enum34 for package: python2-cryptography-1.7.2-2.el7.x86_64
--> Running transaction check
---> Package python-cffi.x86_64 0:1.6.0-5.el7 will be installed
--> Processing Dependency: python-pycparser for package: python-cffi-1.6.0-5.el7.x86_64
---> Package python-enum34.noarch 0:1.0.4-1.el7 will be installed
---> Package python-idna.noarch 0:2.4-1.el7 will be installed
---> Package python2-pyasn1.noarch 0:0.1.9-7.el7 will be installed
--> Running transaction check
---> Package python-pycparser.noarch 0:2.14-1.el7 will be installed
--> Processing Dependency: python-ply for package: python-pycparser-2.14-1.el7.noarch
--> Running transaction check
---> Package python-ply.noarch 0:3.4-11.el7 will be installed
--> Finished Dependency Resolution

Dependencies Resolved

========================================================================================================================================
 Package                           Arch                Version                  Repository                                         Size
========================================================================================================================================
Installing:
 ceph-iscsi                        noarch              3.4-4.el7cp              lo-rhel-7-server-rhceph-4-tools-rpms              204 k
     replacing  ceph-iscsi-cli.noarch 2.7-17.el7cp
     replacing  ceph-iscsi-config.noarch 2.6-23.el7cp
Installing for dependencies:
 python-cffi                       x86_64              1.6.0-5.el7              lo-rhel-7-server-rpms                             218 k
 python-enum34                     noarch              1.0.4-1.el7              lo-rhel-7-server-rpms                              52 k
 python-idna                       noarch              2.4-1.el7                lo-rhel-7-server-rpms                              94 k
 python-ply                        noarch              3.4-11.el7               lo-rhel-7-server-rpms                             123 k
 python-pycparser                  noarch              2.14-1.el7               lo-rhel-7-server-rpms                             105 k
 python2-cryptography              x86_64              1.7.2-2.el7              lo-rhel-7-server-rpms                             503 k
 python2-pyasn1                    noarch              0.1.9-7.el7              lo-rhel-7-server-rpms                             100 k

Transaction Summary
========================================================================================================================================
Install  1 Package (+7 Dependent packages)

Total download size: 1.4 M
Is this ok [y/d/N]: y
Downloading packages:
(1/8): ceph-iscsi-3.4-4.el7cp.noarch.rpm                                                                         | 204 kB  00:00:00     
(2/8): python-cffi-1.6.0-5.el7.x86_64.rpm                                                                        | 218 kB  00:00:00     
(3/8): python-idna-2.4-1.el7.noarch.rpm                                                                          |  94 kB  00:00:00     
(4/8): python-ply-3.4-11.el7.noarch.rpm                                                                          | 123 kB  00:00:00     
(5/8): python-enum34-1.0.4-1.el7.noarch.rpm                                                                      |  52 kB  00:00:00     
(6/8): python-pycparser-2.14-1.el7.noarch.rpm                                                                    | 105 kB  00:00:00     
(7/8): python2-pyasn1-0.1.9-7.el7.noarch.rpm                                                                     | 100 kB  00:00:00     
(8/8): python2-cryptography-1.7.2-2.el7.x86_64.rpm                                                               | 503 kB  00:00:00     
----------------------------------------------------------------------------------------------------------------------------------------
Total                                                                                                   8.2 MB/s | 1.4 MB  00:00:00     
Running transaction check
Running transaction test
Transaction test succeeded
Running transaction
Warning: RPMDB altered outside of yum.
  Installing : python2-pyasn1-0.1.9-7.el7.noarch                                                                                   1/10 
  Installing : python-enum34-1.0.4-1.el7.noarch                                                                                    2/10 
  Installing : python-ply-3.4-11.el7.noarch                                                                                        3/10 
  Installing : python-pycparser-2.14-1.el7.noarch                                                                                  4/10 
  Installing : python-cffi-1.6.0-5.el7.x86_64                                                                                      5/10 
  Installing : python-idna-2.4-1.el7.noarch                                                                                        6/10 
  Installing : python2-cryptography-1.7.2-2.el7.x86_64                                                                             7/10 
  Installing : ceph-iscsi-3.4-4.el7cp.noarch                                                                                       8/10 
  Erasing    : ceph-iscsi-cli-2.7-17.el7cp.noarch                                                                                  9/10 
  Erasing    : ceph-iscsi-config-2.6-23.el7cp.noarch                                                                              10/10 
  Verifying  : python-idna-2.4-1.el7.noarch                                                                                        1/10 
  Verifying  : python-pycparser-2.14-1.el7.noarch                                                                                  2/10 
  Verifying  : python-ply-3.4-11.el7.noarch                                                                                        3/10 
  Verifying  : python-cffi-1.6.0-5.el7.x86_64                                                                                      4/10 
  Verifying  : python-enum34-1.0.4-1.el7.noarch                                                                                    5/10 
  Verifying  : python2-pyasn1-0.1.9-7.el7.noarch                                                                                   6/10 
  Verifying  : ceph-iscsi-3.4-4.el7cp.noarch                                                                                       7/10 
  Verifying  : python2-cryptography-1.7.2-2.el7.x86_64                                                                             8/10 
  Verifying  : ceph-iscsi-cli-2.7-17.el7cp.noarch                                                                                  9/10 
  Verifying  : ceph-iscsi-config-2.6-23.el7cp.noarch                                                                              10/10 

Installed:
  ceph-iscsi.noarch 0:3.4-4.el7cp                                                                                                       

Dependency Installed:
  python-cffi.x86_64 0:1.6.0-5.el7          python-enum34.noarch 0:1.0.4-1.el7         python-idna.noarch 0:2.4-1.el7                 
  python-ply.noarch 0:3.4-11.el7            python-pycparser.noarch 0:2.14-1.el7       python2-cryptography.x86_64 0:1.7.2-2.el7      
  python2-pyasn1.noarch 0:0.1.9-7.el7      

Replaced:
  ceph-iscsi-cli.noarch 0:2.7-17.el7cp                              ceph-iscsi-config.noarch 0:2.6-23.el7cp                             

Complete!
~~~

Comment 10 Gopi 2022-01-29 11:54:36 UTC
Working with latest ceph-iscsi packages. Hence marking this bug as verified.

Before upgrade:
[root@ceph-pg-958pbh-node2 yum.repos.d]# ceph -v
ceph version 12.2.12-139.el7cp (e6f9ac0c7c2320bd7232cd8ef7154bb30216d9fc) luminous (stable)

[root@ceph-gpatta-rcir5u-node2 yum.repos.d]# gwcli ls
Warning: Could not load preferences file /root/.gwcli/prefs.bin.
o- / ......................................................................................................................... [...]
  o- clusters ........................................................................................................ [Clusters: 1]
  | o- ceph .......................................................................................................... [HEALTH_WARN]
  |   o- pools .......................................................................................................... [Pools: 3]
  |   | o- cephfs_data ........................................................... [(x3), Commit: 0.00Y/40910628K (0%), Used: 0.00Y]
  |   | o- cephfs_metadata ....................................................... [(x3), Commit: 0.00Y/40910628K (0%), Used: 2246b]
  |   | o- rbd .................................................................... [(x3), Commit: 30G/40910628K (76%), Used: 4111b]
  |   o- topology ................................................................................................ [OSDs: 9,MONs: 3]
  o- disks ......................................................................................................... [30G, Disks: 1]
  | o- rbd.ansible1 ............................................................................................... [ansible1 (30G)]
  o- iscsi-target ..................................................................................................... [Targets: 1]
    o- iqn.2003-01.com.redhat.iscsi-gw:ceph-igw ...................................................................... [Gateways: 2]
      o- gateways ............................................................................................ [Up: 2/2, Portals: 2]
      | o- ceph-gpatta-rcir5u-node2 ............................................................................. [10.0.210.96 (UP)]
      | o- ceph-gpatta-rcir5u-node3 ............................................................................ [10.0.209.254 (UP)]
      o- host-groups .................................................................................................. [Groups : 0]
      o- hosts .............................................................................................. [Hosts: 1: Auth: CHAP]
        o- iqn.1994-05.com.redhat:rh7-iscsi-client ..................................................... [Auth: CHAP, Disks: 1(30G)]
          o- lun 0 ............................................................ [rbd.ansible1(30G), Owner: ceph-gpatta-rcir5u-node2]

[root@ceph-gpatta-rcir5u-node3 yum.repos.d]# rpm -qa |grep ceph-iscsi
ceph-iscsi-cli-2.7-17.el7cp.noarch
ceph-iscsi-tools-2.1-3.el7cp.noarch
ceph-iscsi-config-2.6-23.el7cp.noarch


After upgrade:
--------------
[root@ceph-gpatta-rcir5u-node2 yum.repos.d]# gwcli ls
o- / ......................................................................................................................... [...]
  o- cluster ......................................................................................................... [Clusters: 1]
  | o- ceph .......................................................................................................... [HEALTH_WARN]
  |   o- pools .......................................................................................................... [Pools: 3]
  |   | o- cephfs_data ........................................................... [(x3), Commit: 0.00Y/56544804K (0%), Used: 0.00Y]
  |   | o- cephfs_metadata ....................................................... [(x3), Commit: 0.00Y/56544804K (0%), Used: 5487b]
  |   | o- rbd .................................................................... [(x3), Commit: 30G/56544804K (55%), Used: 5584b]
  |   o- topology ................................................................................................ [OSDs: 9,MONs: 3]
  o- disks ......................................................................................................... [30G, Disks: 1]
  | o- rbd ............................................................................................................. [rbd (30G)]
  |   o- ansible1 ............................................................................................. [rbd/ansible1 (30G)]
  o- iscsi-targets ............................................................................... [DiscoveryAuth: None, Targets: 1]
    o- iqn.2003-01.com.redhat.iscsi-gw:ceph-igw .......................................................... [Auth: None, Gateways: 2]
      o- disks .......................................................................................................... [Disks: 1]
      | o- rbd/ansible1 .................................................................. [Owner: ceph-gpatta-rcir5u-node2, Lun: 0]
      o- gateways ............................................................................................ [Up: 2/2, Portals: 2]
      | o- ceph-gpatta-rcir5u-node2 ............................................................................. [10.0.210.96 (UP)]
      | o- ceph-gpatta-rcir5u-node3 ............................................................................ [10.0.209.254 (UP)]
      o- host-groups .................................................................................................. [Groups : 0]
      o- hosts ....................................................................................... [Auth: ACL_ENABLED, Hosts: 1]
        o- iqn.1994-05.com.redhat:rh7-iscsi-client ..................................................... [Auth: CHAP, Disks: 1(30G)]
          o- lun 0 ............................................................ [rbd/ansible1(30G), Owner: ceph-gpatta-rcir5u-node2]

[root@ceph-gpatta-rcir5u-node2 yum.repos.d]# rpm -qa | grep ceph-iscsi
ceph-iscsi-3.4-5.el7cp.noarch
[root@ceph-gpatta-rcir5u-node2 yum.repos.d]# yum install ceph-iscsi-tools
Loaded plugins: product-id, search-disabled-repos, subscription-manager
Package ceph-iscsi-tools-2.1-3.el7cp.noarch is obsoleted by ceph-iscsi-3.4-5.el7cp.noarch which is already installed
Nothing to do

Comment 15 errata-xmlrpc 2022-02-08 13:01:20 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (Red Hat Ceph Storage 5.0 Bug Fix update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2022:0466


Note You need to log in before you can comment on or make changes to this bug.