Note: This bug is displayed in read-only format because the product is no longer active in Red Hat Bugzilla.
RHEL Engineering is moving the tracking of its product development work on RHEL 6 through RHEL 9 to Red Hat Jira (issues.redhat.com). If you're a Red Hat customer, please continue to file support cases via the Red Hat customer portal. If you're not, please head to the "RHEL project" in Red Hat Jira and file new tickets here. Individual Bugzilla bugs in the statuses "NEW", "ASSIGNED", and "POST" are being migrated throughout September 2023. Bugs of Red Hat partners with an assigned Engineering Partner Manager (EPM) are migrated in late September as per pre-agreed dates. Bugs against components "kernel", "kernel-rt", and "kpatch" are only migrated if still in "NEW" or "ASSIGNED". If you cannot log in to RH Jira, please consult article #7032570. That failing, please send an e-mail to the RH Jira admins at rh-issues@redhat.com to troubleshoot your issue as a user management inquiry. The email creates a ServiceNow ticket with Red Hat. Individual Bugzilla bugs that are migrated will be moved to status "CLOSED", resolution "MIGRATED", and set with "MigratedToJIRA" in "Keywords". The link to the successor Jira issue will be found under "Links", have a little "two-footprint" icon next to it, and direct you to the "RHEL project" in Red Hat Jira (issue links are of type "https://issues.redhat.com/browse/RHEL-XXXX", where "X" is a digit). This same link will be available in a blue banner at the top of the page informing you that that bug has been migrated.

Bug 2024522

Summary: [RFE] Provide a way to add a scsi **mpath** fencing device to a cluster without requiring a restart of all cluster resources
Product: Red Hat Enterprise Linux 9 Reporter: Tomas Jelinek <tojeline>
Component: pcsAssignee: Miroslav Lisik <mlisik>
Status: CLOSED ERRATA QA Contact: cluster-qe <cluster-qe>
Severity: urgent Docs Contact: Steven J. Levine <slevine>
Priority: urgent    
Version: 9.0CC: cluster-maint, cluster-qe, idevat, kgaillot, kmalyjur, mlisik, mmazoure, mpospisi, nhostako, omular, sbradley, slevine, tojeline
Target Milestone: rcKeywords: FutureFeature, Triaged
Target Release: 9.1Flags: pm-rhel: mirror+
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: pcs-0.11.2-1.el9 Doc Type: Enhancement
Doc Text:
.`pcs` supports updating multipath SCSI devices without requiring a system restart You can now update multipath SCSI devices with the `pcs stonith update-scsi-devices` command. This command updates SCSI devices without causing a restart of other cluster resources running on the same node.
 Story Points: ---
Clone Of: 2023845 Environment:
Last Closed: 2022-11-15 09:48:38 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On: 1872376, 1872378, 2023845    
Bug Blocks:    

Comment 2 Miroslav Lisik 2022-03-10 17:43:31 UTC 
Upstream commit: https://github.com/ClusterLabs/pcs/commit/bcf5526cb887be9987ae177ac494097f4532a949

Updated command:
pcs stonith update-scsi-devices

Test:
# pcs stonith update-scsi-devices <mpath-fence-device> set <device-path>...
# pcs stonith update-scsi-devices <mpath-fence-device> add <device-path>... remove <device-path>...

Devices of fence_mpath instance should be unfenced and updated and no resources should be restarted.
Comment 4 Miroslav Lisik 2022-05-19 17:17:06 UTC 
DevTestResults:

[root@r91-1 ~]# rpm -q pcs
pcs-0.11.2-1.el9.x86_64

### shortcuts for disk paths

[root@r91-1 ~]# for d in $mdisk{1..4}; do echo $d; done
/dev/disk/by-id/dm-uuid-mpath-36001405ae93fea8ee14438c843cb9e87
/dev/disk/by-id/dm-uuid-mpath-36001405b2eb733c07d34dbc87d393464
/dev/disk/by-id/dm-uuid-mpath-36001405eafcac8fb5004997b02bacbc7
/dev/disk/by-id/dm-uuid-mpath-360014053573a41da4a24ab7acde57014

### multipath configuration

[root@r91-1 ~]# multipath -ll
mpatha (36001405ae93fea8ee14438c843cb9e87) dm-0 LIO-ORG,r91-mdisk-01
size=100M features='0' hwhandler='1 alua' wp=rw
|-+- policy='service-time 0' prio=50 status=active
| `- 10:0:0:0 sdd 8:48  active ready running
`-+- policy='service-time 0' prio=50 status=enabled
  `- 15:0:0:0 sdf 8:80  active ready running
mpathb (36001405b2eb733c07d34dbc87d393464) dm-1 LIO-ORG,r91-mdisk-02
size=100M features='0' hwhandler='1 alua' wp=rw
|-+- policy='service-time 0' prio=50 status=active
| `- 10:0:0:1 sdk 8:160 active ready running
`-+- policy='service-time 0' prio=50 status=enabled
  `- 15:0:0:1 sdl 8:176 active ready running
mpathc (36001405eafcac8fb5004997b02bacbc7) dm-2 LIO-ORG,r91-mdisk-03
size=100M features='0' hwhandler='1 alua' wp=rw
|-+- policy='service-time 0' prio=50 status=active
| `- 10:0:0:2 sdh 8:112 active ready running
`-+- policy='service-time 0' prio=50 status=enabled
  `- 15:0:0:2 sdj 8:144 active ready running
mpathd (360014053573a41da4a24ab7acde57014) dm-3 LIO-ORG,r91-mdisk-04
size=100M features='0' hwhandler='1 alua' wp=rw
|-+- policy='service-time 0' prio=50 status=active
| `- 10:0:0:3 sdg 8:96  active ready running
`-+- policy='service-time 0' prio=50 status=enabled
  `- 15:0:0:3 sdi 8:128 active ready running

### list of reservation keys

[root@r91-1 ~]# for d in $mdisk{1..4}; do mpathpersist -i -k -d $d ; done
  PR generation=0x5,    0 registered reservation key.
  PR generation=0x0,    0 registered reservation key.
  PR generation=0x0,    0 registered reservation key.
  PR generation=0x0,    0 registered reservation key.

#### create fence-mpath stonith device and check the configuration

[root@r91-1 ~]# pcs stonith create mpath-fence-device fence_mpath devices="$mdisk1" pcmk_host_check="static-list" pcmk_host_list="r91-1 r91-2" pcmk_host_map="r91-1:1;r91-2:2" pcmk_host_argument="key" pcmk_reboot_action="off" meta provides="unfencing"
[root@r91-1 ~]# pcs stonith config
Resource: mpath-fence-device (class=stonith type=fence_mpath)
  Attributes: mpath-fence-device-instance_attributes
    devices=/dev/disk/by-id/dm-uuid-mpath-36001405ae93fea8ee14438c843cb9e87
    pcmk_host_argument=key
    pcmk_host_check=static-list
    pcmk_host_list="r91-1 r91-2"
    pcmk_host_map=r91-1:1;r91-2:2
    pcmk_reboot_action=off
  Meta Attributes: mpath-fence-device-meta_attributes
    provides=unfencing
  Operations:
    monitor: mpath-fence-device-monitor-interval-60s
      interval=60s
[root@r91-1 ~]# pcs stonith
  * mpath-fence-device  (stonith:fence_mpath):   Started r91-1
[root@r91-1 ~]# for d in $mdisk{1..4}; do mpathpersist -i -k -d $d ; done
  PR generation=0x9,    4 registered reservation keys follow:
    0x1
    0x1
    0x2
    0x2
  PR generation=0x0,    0 registered reservation key.
  PR generation=0x0,    0 registered reservation key.
  PR generation=0x0,    0 registered reservation key.


### create some resources for restart detection checks

[root@r91-1 ~]# for i in $(seq -w 01 04); do pcs resource create d-$i ocf:pacemaker:Dummy; done
[root@r91-1 ~]# pcs resource
  * d-01        (ocf:pacemaker:Dummy):   Started r91-2
  * d-02        (ocf:pacemaker:Dummy):   Started r91-1
  * d-03        (ocf:pacemaker:Dummy):   Started r91-2
  * d-04        (ocf:pacemaker:Dummy):   Started r91-1

### update fence-mpath device configuration and check if the resources restarted

[root@r91-1 ~]# pcs stonith update-scsi-devices mpath-fence-device set $mdisk1 $mdisk2

[root@r91-1 ~]# journalctl -n 0 -f
May 16 19:22:16 r91-1 kernel: sd 15:0:0:1: reservation conflict
May 16 19:22:16 r91-1 kernel: sd 10:0:0:1: reservation conflict
May 16 19:22:17 r91-1 pacemaker-fenced[4262]:  notice: Added 'mpath-fence-device' to device list (1 active device)

[root@r91-2 ~]# journalctl -n 0 -f
May 16 19:22:16 r91-2 kernel: sd 7:0:0:1: reservation conflict
May 16 19:22:16 r91-2 kernel: sd 8:0:0:1: reservation conflict
May 16 19:22:17 r91-2 pacemaker-controld[3915]:  notice: State transition S_IDLE -> S_POLICY_ENGINE
May 16 19:22:17 r91-2 pacemaker-fenced[3911]:  notice: Added 'mpath-fence-device' to device list (1 active device)
May 16 19:22:17 r91-2 pacemaker-schedulerd[3914]:  notice: Calculated transition 10, saving inputs in /var/lib/pacemaker/pengine/pe-input-29.bz2
May 16 19:22:17 r91-2 pacemaker-controld[3915]:  notice: Transition 10 (Complete=0, Pending=0, Fired=0, Skipped=0, Incomplete=0, Source=/var/lib/pacemaker/pengine/pe-input-29.bz2): Complete
May 16 19:22:17 r91-2 pacemaker-controld[3915]:  notice: State transition S_TRANSITION_ENGINE -> S_IDLE

[root@r91-1 ~]# for d in $mdisk{1..4}; do mpathpersist -i -k -d $d ; done
  PR generation=0x9,    4 registered reservation keys follow:
    0x1
    0x1
    0x2
    0x2
  PR generation=0x4,    4 registered reservation keys follow:
    0x1
    0x1
    0x2
    0x2


[root@r91-1 ~]# pcs stonith update-scsi-devices mpath-fence-device add $mdisk3 $mdisk4 remove $mdisk2
[root@r91-1 ~]# pcs stonith config
Resource: mpath-fence-device (class=stonith type=fence_mpath)
  Attributes: mpath-fence-device-instance_attributes
    devices=/dev/disk/by-id/dm-uuid-mpath-360014053573a41da4a24ab7acde57014,/dev/disk/by-id/dm-uuid-mpath-36001405ae93fea8ee14438c843cb9e87,/dev/disk/by-id/dm-uuid-mpath-36001405eafcac8fb5004997b02bacbc7
    pcmk_host_argument=key
    pcmk_host_check=static-list
    pcmk_host_list="r91-1 r91-2"
    pcmk_host_map=r91-1:1;r91-2:2
    pcmk_reboot_action=off
  Meta Attributes: mpath-fence-device-meta_attributes
    provides=unfencing
  Operations:
    monitor: mpath-fence-device-monitor-interval-60s
      interval=60s

[root@r91-1 ~]# journalctl -n 0 -f
May 16 19:32:37 r91-1 pacemaker-fenced[4262]:  notice: Added 'mpath-fence-device' to device list (1 active device)

[root@r91-2 ~]# journalctl -n 0 -f
May 16 19:32:37 r91-2 kernel: sd 7:0:0:3: reservation conflict
May 16 19:32:37 r91-2 pacemaker-controld[3915]:  notice: State transition S_IDLE -> S_POLICY_ENGINE
May 16 19:32:37 r91-2 pacemaker-fenced[3911]:  notice: Added 'mpath-fence-device' to device list (1 active device)
May 16 19:32:37 r91-2 pacemaker-schedulerd[3914]:  notice: Calculated transition 11, saving inputs in /var/lib/pacemaker/pengine/pe-input-30.bz2
May 16 19:32:37 r91-2 pacemaker-controld[3915]:  notice: Transition 11 (Complete=0, Pending=0, Fired=0, Skipped=0, Incomplete=0, Source=/var/lib/pacemaker/pengine/pe-input-30.bz2): Complete
May 16 19:32:37 r91-2 pacemaker-controld[3915]:  notice: State transition S_TRANSITION_ENGINE -> S_IDLE

[root@r91-1 ~]# for d in $mdisk{1..4}; do mpathpersist -i -k -d $d ; done
  PR generation=0x9,    4 registered reservation keys follow:
    0x1
    0x1
    0x2
    0x2
  PR generation=0x4,    4 registered reservation keys follow:
    0x1
    0x1
    0x2
    0x2
  PR generation=0x4,    4 registered reservation keys follow:
    0x1
    0x1
    0x2
    0x2
  PR generation=0x4,    4 registered reservation keys follow:
    0x1
    0x1
    0x2
    0x2
Comment 14 errata-xmlrpc 2022-11-15 09:48:38 UTC 
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (Moderate: pcs security, bug fix, and enhancement update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2022:7935