Bug 1575010

Summary:

'pcs cluster stop' results in unexpected fencing

Product:

Red Hat Enterprise Linux 7

Reporter:

michal novacek <mnovacek>

Component:

resource-agents

Assignee:

Oyvind Albrigtsen <oalbrigt>

Status:

CLOSED WONTFIX

QA Contact:

cluster-qe <cluster-qe>

Severity:

unspecified

Docs Contact:

Priority:

unspecified

Version:

7.5

CC:

agk, cfeist, cluster-maint, fdinitto, jkortus, mnovacek, oalbrigt, teigland

Target Milestone:

Target Release:

---

Hardware:

Unspecified

OS:

Unspecified

Whiteboard:

Fixed In Version:

Doc Type:

If docs needed, set a value

Doc Text:

Story Points:

---

Clone Of:

Environment:

Last Closed:

2021-02-15 07:38:46 UTC

Type:

Bug

Regression:

---

Mount Type:

---

Documentation:

---

CRM:

Verified Versions:

Category:

---

oVirt Team:

---

RHEL 7.3 requirements from Atomic Host:

Cloudforms Team:

---

Target Upstream Version:

Embargoed:

Attachments:

Description	Flags
pcs cluster stop	none
pcs cluster report	none

Description michal novacek 2018-05-04 14:12:18 UTC

Created attachment 1431379 [details]
pcs cluster stop

Description of problem:
We have two node cluster with configured fencing and a lot of resources
including clvmd, clustered vgs and gfs (2).  What I want to achieve is to stop
cluster on one node, join it back to cluster, stop cluster on the other node
and rejoin it to cluster all this while providing the service. Also I do not
want the nodes to be rebooted by fencing at any time.

Version-Release number of selected component (if applicable):
pacemaker-1.1.18-11.el7.x86_64
resource-agents-3.9.5-124.el7.x86_64
pcs-0.9.162-5.el7_5.1.x86_64

How reproducible: most of the times

Steps to Reproduce:
1. setup pcs cluster (2) (1)
2. node1$ pcs cluster stop node2
3. <wait until settled>
4. node1$ pcs cluster start node2
5. <wait until settled>
6. node2$ pcs cluster stop node1
7. <wait until settled>
8. node2$ pcs cluster start node1

Actual results: I'm unable to get past point six without getting fenced

Expected results: Perform all steps without no fencing.

Additional info:
I believe this is clvmd (resource agent?) fault but cannot find out why. I'm
leaving all resources and constraints in the cluster because it seems to work
with only LVM and Filesystem resource.

Attached 'pcs cluster report' was taken after fencing in step 2.

----

(1) pcs status
[root@virt-136 ~]# pcs config
Cluster Name: el-cluster
Corosync Nodes:
 virt-136 virt-138
Pacemaker Nodes:
 virt-136 virt-138

Resources:
 Clone: dlm-clone
  Resource: dlm (class=ocf provider=pacemaker type=controld)
   Operations: monitor interval=10 start-delay=0 timeout=20 (dlm-monitor-interval-10)
               start interval=0s timeout=90 (dlm-start-interval-0s)
               stop interval=0s timeout=100 (dlm-stop-interval-0s)
 Clone: clvmd-clone
  Meta Attrs: with_cmirrord=1 
  Resource: clvmd (class=ocf provider=heartbeat type=clvm)
   Operations: monitor interval=30 timeout=90 (clvmd-monitor-interval-30)
               start interval=0s timeout=90 (clvmd-start-interval-0s)
               stop interval=0s timeout=90 (clvmd-stop-interval-0s)
 Group: mysql-g-stage
  Resource: db-stage-vip (class=ocf provider=heartbeat type=IPaddr)
   Attributes: cidr_netmask=22 ip=10.37.165.6
   Operations: monitor interval=10s timeout=20s (db-stage-vip-monitor-interval-10s)
               start interval=0s timeout=20s (db-stage-vip-start-interval-0s)
               stop interval=0s timeout=20s (db-stage-vip-stop-interval-0s)
  Resource: db-stage-lvm (class=ocf provider=heartbeat type=LVM)
   Attributes: exclusive=true volgrpname=stagedbvg
   Operations: methods interval=0s timeout=5 (db-stage-lvm-methods-interval-0s)
               monitor interval=10 timeout=30 (db-stage-lvm-monitor-interval-10)
               start interval=0s timeout=30 (db-stage-lvm-start-interval-0s)
               stop interval=0s timeout=30 (db-stage-lvm-stop-interval-0s)
  Resource: db-stage-fs (class=ocf provider=heartbeat type=Filesystem)
   Attributes: device=/dev/stagedbvg/stagedblv directory=/var/lib/mysql-stage fstype=ext4
   Operations: monitor interval=20 timeout=40 (db-stage-fs-monitor-interval-20)
               notify interval=0s timeout=60 (db-stage-fs-notify-interval-0s)
               start interval=0s timeout=60 (db-stage-fs-start-interval-0s)
               stop interval=0s timeout=60 (db-stage-fs-stop-interval-0s)
  Resource: mysql-stage (class=ocf provider=heartbeat type=mysql)
   Attributes: additional_parameters=--bind-address=10.37.165.6 datadir=/var/lib/mysql-stage log=/var/log/mariadb/mariadb-stage.log pid=/var/lib/mysql-stage/mysql.pid socket=/var/lib/mysql-stage/mysql.sock
   Operations: demote interval=0s timeout=120 (mysql-stage-demote-interval-0s)
               monitor interval=20 timeout=30 (mysql-stage-monitor-interval-20)
               monitor interval=10 role=Master timeout=30 (mysql-stage-monitor-interval-10)
               monitor interval=30 role=Slave timeout=30 (mysql-stage-monitor-interval-30)
               notify interval=0s timeout=90 (mysql-stage-notify-interval-0s)
               promote interval=0s timeout=120 (mysql-stage-promote-interval-0s)
               start interval=0s timeout=120 (mysql-stage-start-interval-0s)
               stop interval=0s timeout=120 (mysql-stage-stop-interval-0s)
 Group: mysql-g-live
  Resource: db-live-vip (class=ocf provider=heartbeat type=IPaddr)
   Attributes: cidr_netmask=22 ip=10.37.165.3
   Operations: monitor interval=10s timeout=20s (db-live-vip-monitor-interval-10s)
               start interval=0s timeout=20s (db-live-vip-start-interval-0s)
               stop interval=0s timeout=20s (db-live-vip-stop-interval-0s)
  Resource: db-live-lvm (class=ocf provider=heartbeat type=LVM)
   Attributes: exclusive=true volgrpname=livedbvg
   Operations: methods interval=0s timeout=5 (db-live-lvm-methods-interval-0s)
               monitor interval=10 timeout=30 (db-live-lvm-monitor-interval-10)
               start interval=0s timeout=30 (db-live-lvm-start-interval-0s)
               stop interval=0s timeout=30 (db-live-lvm-stop-interval-0s)
  Resource: db-live-fs (class=ocf provider=heartbeat type=Filesystem)
   Attributes: device=/dev/livedbvg/livedblv directory=/var/lib/mysql-live fstype=ext4
   Operations: monitor interval=20 timeout=40 (db-live-fs-monitor-interval-20)
               notify interval=0s timeout=60 (db-live-fs-notify-interval-0s)
               start interval=0s timeout=60 (db-live-fs-start-interval-0s)
               stop interval=0s timeout=60 (db-live-fs-stop-interval-0s)
  Resource: mysql-live (class=ocf provider=heartbeat type=mysql)
   Attributes: additional_parameters=--bind-address=10.37.165.3 datadir=/var/lib/mysql-live log=/var/log/mariadb/mariadb-live.log pid=/var/lib/mysql-live/mysql.pid socket=/var/lib/mysql-live/mysql.sock
   Operations: demote interval=0s timeout=120 (mysql-live-demote-interval-0s)
               monitor interval=20 timeout=30 (mysql-live-monitor-interval-20)
               monitor interval=10 role=Master timeout=30 (mysql-live-monitor-interval-10)
               monitor interval=30 role=Slave timeout=30 (mysql-live-monitor-interval-30)
               notify interval=0s timeout=90 (mysql-live-notify-interval-0s)
               promote interval=0s timeout=120 (mysql-live-promote-interval-0s)
               start interval=0s timeout=120 (mysql-live-start-interval-0s)
               stop interval=0s timeout=120 (mysql-live-stop-interval-0s)
 Clone: dockerd-clone
  Resource: dockerd (class=systemd type=docker)
   Operations: monitor interval=60 timeout=100 (dockerd-monitor-interval-60)
               start interval=0s timeout=100 (dockerd-start-interval-0s)
               stop interval=0s timeout=100 (dockerd-stop-interval-0s)
 Clone: container-logs-clone
  Resource: container-logs (class=ocf provider=heartbeat type=Filesystem)
   Attributes: device=/dev/vg-logs/lv-logs directory=/var/log/containers fstype=gfs2
   Operations: monitor interval=20 timeout=40 (container-logs-monitor-interval-20)
               notify interval=0s timeout=60 (container-logs-notify-interval-0s)
               start interval=0s timeout=60 (container-logs-start-interval-0s)
               stop interval=0s timeout=60 (container-logs-stop-interval-0s)
 Group: cqe-frontend-stage
  Resource: frontend-stage-vip (class=ocf provider=heartbeat type=IPaddr)
   Attributes: cidr_netmask=22 ip=10.37.165.34
   Operations: monitor interval=10s timeout=20s (frontend-stage-vip-monitor-interval-10s)
               start interval=0s timeout=20s (frontend-stage-vip-start-interval-0s)
               stop interval=0s timeout=20s (frontend-stage-vip-stop-interval-0s)
  Resource: frontend-stage (class=ocf provider=heartbeat type=docker)
   Attributes: image=docker-registry.engineering.redhat.com/cqe/frontend:latest-stage monitor_cmd="curl http://localhost:8080/clusterqe/" run_opts="--env-file=/var/log/containers/container-variables-stage --volume /var/log/containers/frontend-stage:/var/log --publish 8081:8080"
   Operations: monitor interval=30 timeout=30 (frontend-stage-monitor-interval-30)
               start interval=0s timeout=90 (frontend-stage-start-interval-0s)
               stop interval=0s timeout=90 (frontend-stage-stop-interval-0s)
 Group: cqe-frontend-live
  Resource: frontend-live-vip (class=ocf provider=heartbeat type=IPaddr)
   Attributes: cidr_netmask=22 ip=10.37.165.44
   Operations: monitor interval=10s timeout=20s (frontend-live-vip-monitor-interval-10s)
               start interval=0s timeout=20s (frontend-live-vip-start-interval-0s)
               stop interval=0s timeout=20s (frontend-live-vip-stop-interval-0s)
  Resource: frontend-live (class=ocf provider=heartbeat type=docker)
   Attributes: image=docker-registry.engineering.redhat.com/cqe/frontend:latest-live monitor_cmd="curl http://localhost:8080/clusterqe/" run_opts="--env-file=/var/log/containers/container-variables-live --volume /var/log/containers/frontend-live:/var/log --publish 8080:8080"
   Operations: monitor interval=30 timeout=30 (frontend-live-monitor-interval-30)
               start interval=0s timeout=90 (frontend-live-start-interval-0s)
               stop interval=0s timeout=90 (frontend-live-stop-interval-0s)
 Resource: cqe-dispatcher-live (class=ocf provider=heartbeat type=docker)
  Attributes: image=docker-registry.engineering.redhat.com/cqe/dispatcher:latest-live run_opts="--env-file=/var/log/containers/container-variables-live --volume /var/log/containers/dispatcher-live:/var/log"
  Operations: monitor interval=30 timeout=30 (cqe-dispatcher-live-monitor-interval-30)
              start interval=0s timeout=90 (cqe-dispatcher-live-start-interval-0s)
              stop interval=0s timeout=90 (cqe-dispatcher-live-stop-interval-0s)
 Resource: cqe-dispatcher-stage (class=ocf provider=heartbeat type=docker)
  Attributes: image=docker-registry.engineering.redhat.com/cqe/dispatcher:latest-stage run_opts="--env-file=/var/log/containers/container-variables-stage --volume /var/log/containers/dispatcher-stage:/var/log"
  Operations: monitor interval=30 timeout=30 (cqe-dispatcher-stage-monitor-interval-30)
              start interval=0s timeout=90 (cqe-dispatcher-stage-start-interval-0s)
              stop interval=0s timeout=90 (cqe-dispatcher-stage-stop-interval-0s)

Stonith Devices:
 Resource: fence-virt-136 (class=stonith type=fence_xvm)
  Attributes: delay=5 pcmk_host_check=static-list pcmk_host_list=virt-136 pcmk_host_map=virt-136:virt-136.cluster-qe.lab.eng.brq.redhat.com
  Operations: monitor interval=60s (fence-virt-136-monitor-interval-60s)
 Resource: fence-virt-138 (class=stonith type=fence_xvm)
  Attributes: delay=5 pcmk_host_check=static-list pcmk_host_list=virt-138 pcmk_host_map=virt-138:virt-138.cluster-qe.lab.eng.brq.redhat.com
  Operations: monitor interval=60s (fence-virt-138-monitor-interval-60s)
Fencing Levels:

Location Constraints:
  Resource: fence-virt-136
    Disabled on: virt-136 (score:-10) (id:location-fence-virt-136-virt-136--10)
  Resource: fence-virt-138
    Disabled on: virt-138 (score:-10) (id:location-fence-virt-138-virt-138--10)
Ordering Constraints:
  start dlm-clone then start clvmd-clone (kind:Mandatory)
  start clvmd-clone then start mysql-g-stage (kind:Mandatory)
  start clvmd-clone then start mysql-g-live (kind:Mandatory)
  start dlm-clone then start container-logs-clone (kind:Mandatory)
  start container-logs-clone then start cqe-frontend-stage (kind:Mandatory)
  start dockerd-clone then start cqe-frontend-stage (kind:Mandatory)
  start mysql-g-stage then start cqe-frontend-stage (kind:Mandatory)
  start container-logs-clone then start cqe-frontend-live (kind:Mandatory)
  start dockerd-clone then start cqe-frontend-live (kind:Mandatory)
  start mysql-g-live then start cqe-frontend-live (kind:Mandatory)
  start cqe-frontend-live then start cqe-dispatcher-live (kind:Mandatory)
  start cqe-frontend-stage then start cqe-dispatcher-stage (kind:Mandatory)
Colocation Constraints:
  clvmd-clone with dlm-clone (score:INFINITY)
  mysql-g-stage with clvmd-clone (score:INFINITY)
  mysql-g-live with clvmd-clone (score:INFINITY)
  mysql-g-live with mysql-g-stage (score:-10)
  container-logs-clone with dlm-clone (score:INFINITY)
  cqe-frontend-stage with container-logs-clone (score:INFINITY)
  cqe-frontend-stage with dockerd-clone (score:INFINITY)
  cqe-frontend-live with container-logs-clone (score:INFINITY)
  cqe-frontend-live with dockerd-clone (score:INFINITY)
  cqe-dispatcher-live with container-logs-clone (score:INFINITY)
  cqe-dispatcher-live with dockerd-clone (score:INFINITY)
  cqe-dispatcher-stage with container-logs-clone (score:INFINITY)
  cqe-dispatcher-stage with dockerd-clone (score:INFINITY)
Ticket Constraints:

Alerts:
 No alerts defined

Resources Defaults:
 No defaults set
Operations Defaults:
 No defaults set

Cluster Properties:
 cluster-infrastructure: corosync
 cluster-name: el-cluster
 dc-version: 1.1.18-11.el7-2b07d5c5a9
 have-watchdog: false
 last-lrm-refresh: 1525441101

Quorum:
  Options:
    auto_tie_breaker: 0

(2) pcs config
[root@virt-136 ~]# pcs config
Cluster Name: el-cluster
Corosync Nodes:
 virt-136 virt-138
Pacemaker Nodes:
 virt-136 virt-138

Resources:
 Clone: dlm-clone
  Resource: dlm (class=ocf provider=pacemaker type=controld)
   Operations: monitor interval=10 start-delay=0 timeout=20 (dlm-monitor-interval-10)
               start interval=0s timeout=90 (dlm-start-interval-0s)
               stop interval=0s timeout=100 (dlm-stop-interval-0s)
 Clone: clvmd-clone
  Meta Attrs: with_cmirrord=1 
  Resource: clvmd (class=ocf provider=heartbeat type=clvm)
   Operations: monitor interval=30 timeout=90 (clvmd-monitor-interval-30)
               start interval=0s timeout=90 (clvmd-start-interval-0s)
               stop interval=0s timeout=90 (clvmd-stop-interval-0s)
 Group: mysql-g-stage
  Resource: db-stage-vip (class=ocf provider=heartbeat type=IPaddr)
   Attributes: cidr_netmask=22 ip=10.37.165.6
   Operations: monitor interval=10s timeout=20s (db-stage-vip-monitor-interval-10s)
               start interval=0s timeout=20s (db-stage-vip-start-interval-0s)
               stop interval=0s timeout=20s (db-stage-vip-stop-interval-0s)
  Resource: db-stage-lvm (class=ocf provider=heartbeat type=LVM)
   Attributes: exclusive=true volgrpname=stagedbvg
   Operations: methods interval=0s timeout=5 (db-stage-lvm-methods-interval-0s)
               monitor interval=10 timeout=30 (db-stage-lvm-monitor-interval-10)
               start interval=0s timeout=30 (db-stage-lvm-start-interval-0s)
               stop interval=0s timeout=30 (db-stage-lvm-stop-interval-0s)
  Resource: db-stage-fs (class=ocf provider=heartbeat type=Filesystem)
   Attributes: device=/dev/stagedbvg/stagedblv directory=/var/lib/mysql-stage fstype=ext4
   Operations: monitor interval=20 timeout=40 (db-stage-fs-monitor-interval-20)
               notify interval=0s timeout=60 (db-stage-fs-notify-interval-0s)
               start interval=0s timeout=60 (db-stage-fs-start-interval-0s)
               stop interval=0s timeout=60 (db-stage-fs-stop-interval-0s)
  Resource: mysql-stage (class=ocf provider=heartbeat type=mysql)
   Attributes: additional_parameters=--bind-address=10.37.165.6 datadir=/var/lib/mysql-stage log=/var/log/mariadb/mariadb-stage.log pid=/var/lib/mysql-stage/mysql.pid socket=/var/lib/mysql-stage/mysql.sock
   Operations: demote interval=0s timeout=120 (mysql-stage-demote-interval-0s)
               monitor interval=20 timeout=30 (mysql-stage-monitor-interval-20)
               monitor interval=10 role=Master timeout=30 (mysql-stage-monitor-interval-10)
               monitor interval=30 role=Slave timeout=30 (mysql-stage-monitor-interval-30)
               notify interval=0s timeout=90 (mysql-stage-notify-interval-0s)
               promote interval=0s timeout=120 (mysql-stage-promote-interval-0s)
               start interval=0s timeout=120 (mysql-stage-start-interval-0s)
               stop interval=0s timeout=120 (mysql-stage-stop-interval-0s)
 Group: mysql-g-live
  Resource: db-live-vip (class=ocf provider=heartbeat type=IPaddr)
   Attributes: cidr_netmask=22 ip=10.37.165.3
   Operations: monitor interval=10s timeout=20s (db-live-vip-monitor-interval-10s)
               start interval=0s timeout=20s (db-live-vip-start-interval-0s)
               stop interval=0s timeout=20s (db-live-vip-stop-interval-0s)
  Resource: db-live-lvm (class=ocf provider=heartbeat type=LVM)
   Attributes: exclusive=true volgrpname=livedbvg
   Operations: methods interval=0s timeout=5 (db-live-lvm-methods-interval-0s)
               monitor interval=10 timeout=30 (db-live-lvm-monitor-interval-10)
               start interval=0s timeout=30 (db-live-lvm-start-interval-0s)
               stop interval=0s timeout=30 (db-live-lvm-stop-interval-0s)
  Resource: db-live-fs (class=ocf provider=heartbeat type=Filesystem)
   Attributes: device=/dev/livedbvg/livedblv directory=/var/lib/mysql-live fstype=ext4
   Operations: monitor interval=20 timeout=40 (db-live-fs-monitor-interval-20)
               notify interval=0s timeout=60 (db-live-fs-notify-interval-0s)
               start interval=0s timeout=60 (db-live-fs-start-interval-0s)
               stop interval=0s timeout=60 (db-live-fs-stop-interval-0s)
  Resource: mysql-live (class=ocf provider=heartbeat type=mysql)
   Attributes: additional_parameters=--bind-address=10.37.165.3 datadir=/var/lib/mysql-live log=/var/log/mariadb/mariadb-live.log pid=/var/lib/mysql-live/mysql.pid socket=/var/lib/mysql-live/mysql.sock
   Operations: demote interval=0s timeout=120 (mysql-live-demote-interval-0s)
               monitor interval=20 timeout=30 (mysql-live-monitor-interval-20)
               monitor interval=10 role=Master timeout=30 (mysql-live-monitor-interval-10)
               monitor interval=30 role=Slave timeout=30 (mysql-live-monitor-interval-30)
               notify interval=0s timeout=90 (mysql-live-notify-interval-0s)
               promote interval=0s timeout=120 (mysql-live-promote-interval-0s)
               start interval=0s timeout=120 (mysql-live-start-interval-0s)
               stop interval=0s timeout=120 (mysql-live-stop-interval-0s)
 Clone: dockerd-clone
  Resource: dockerd (class=systemd type=docker)
   Operations: monitor interval=60 timeout=100 (dockerd-monitor-interval-60)
               start interval=0s timeout=100 (dockerd-start-interval-0s)
               stop interval=0s timeout=100 (dockerd-stop-interval-0s)
 Clone: container-logs-clone
  Resource: container-logs (class=ocf provider=heartbeat type=Filesystem)
   Attributes: device=/dev/vg-logs/lv-logs directory=/var/log/containers fstype=gfs2
   Operations: monitor interval=20 timeout=40 (container-logs-monitor-interval-20)
               notify interval=0s timeout=60 (container-logs-notify-interval-0s)
               start interval=0s timeout=60 (container-logs-start-interval-0s)
               stop interval=0s timeout=60 (container-logs-stop-interval-0s)
 Group: cqe-frontend-stage
  Resource: frontend-stage-vip (class=ocf provider=heartbeat type=IPaddr)
   Attributes: cidr_netmask=22 ip=10.37.165.34
   Operations: monitor interval=10s timeout=20s (frontend-stage-vip-monitor-interval-10s)
               start interval=0s timeout=20s (frontend-stage-vip-start-interval-0s)
               stop interval=0s timeout=20s (frontend-stage-vip-stop-interval-0s)
  Resource: frontend-stage (class=ocf provider=heartbeat type=docker)
   Attributes: image=docker-registry.engineering.redhat.com/cqe/frontend:latest-stage monitor_cmd="curl http://localhost:8080/clusterqe/" run_opts="--env-file=/var/log/containers/container-variables-stage --volume /var/log/containers/frontend-stage:/var/log --publish 8081:8080"
   Operations: monitor interval=30 timeout=30 (frontend-stage-monitor-interval-30)
               start interval=0s timeout=90 (frontend-stage-start-interval-0s)
               stop interval=0s timeout=90 (frontend-stage-stop-interval-0s)
 Group: cqe-frontend-live
  Resource: frontend-live-vip (class=ocf provider=heartbeat type=IPaddr)
   Attributes: cidr_netmask=22 ip=10.37.165.44
   Operations: monitor interval=10s timeout=20s (frontend-live-vip-monitor-interval-10s)
               start interval=0s timeout=20s (frontend-live-vip-start-interval-0s)
               stop interval=0s timeout=20s (frontend-live-vip-stop-interval-0s)
  Resource: frontend-live (class=ocf provider=heartbeat type=docker)
   Attributes: image=docker-registry.engineering.redhat.com/cqe/frontend:latest-live monitor_cmd="curl http://localhost:8080/clusterqe/" run_opts="--env-file=/var/log/containers/container-variables-live --volume /var/log/containers/frontend-live:/var/log --publish 8080:8080"
   Operations: monitor interval=30 timeout=30 (frontend-live-monitor-interval-30)
               start interval=0s timeout=90 (frontend-live-start-interval-0s)
               stop interval=0s timeout=90 (frontend-live-stop-interval-0s)
 Resource: cqe-dispatcher-live (class=ocf provider=heartbeat type=docker)
  Attributes: image=docker-registry.engineering.redhat.com/cqe/dispatcher:latest-live run_opts="--env-file=/var/log/containers/container-variables-live --volume /var/log/containers/dispatcher-live:/var/log"
  Operations: monitor interval=30 timeout=30 (cqe-dispatcher-live-monitor-interval-30)
              start interval=0s timeout=90 (cqe-dispatcher-live-start-interval-0s)
              stop interval=0s timeout=90 (cqe-dispatcher-live-stop-interval-0s)
 Resource: cqe-dispatcher-stage (class=ocf provider=heartbeat type=docker)
  Attributes: image=docker-registry.engineering.redhat.com/cqe/dispatcher:latest-stage run_opts="--env-file=/var/log/containers/container-variables-stage --volume /var/log/containers/dispatcher-stage:/var/log"
  Operations: monitor interval=30 timeout=30 (cqe-dispatcher-stage-monitor-interval-30)
              start interval=0s timeout=90 (cqe-dispatcher-stage-start-interval-0s)
              stop interval=0s timeout=90 (cqe-dispatcher-stage-stop-interval-0s)

Stonith Devices:
 Resource: fence-virt-136 (class=stonith type=fence_xvm)
  Attributes: delay=5 pcmk_host_check=static-list pcmk_host_list=virt-136 pcmk_host_map=virt-136:virt-136.cluster-qe.lab.eng.brq.redhat.com
  Operations: monitor interval=60s (fence-virt-136-monitor-interval-60s)
 Resource: fence-virt-138 (class=stonith type=fence_xvm)
  Attributes: delay=5 pcmk_host_check=static-list pcmk_host_list=virt-138 pcmk_host_map=virt-138:virt-138.cluster-qe.lab.eng.brq.redhat.com
  Operations: monitor interval=60s (fence-virt-138-monitor-interval-60s)
Fencing Levels:

Location Constraints:
  Resource: fence-virt-136
    Disabled on: virt-136 (score:-10) (id:location-fence-virt-136-virt-136--10)
  Resource: fence-virt-138
    Disabled on: virt-138 (score:-10) (id:location-fence-virt-138-virt-138--10)
Ordering Constraints:
  start dlm-clone then start clvmd-clone (kind:Mandatory)
  start clvmd-clone then start mysql-g-stage (kind:Mandatory)
  start clvmd-clone then start mysql-g-live (kind:Mandatory)
  start dlm-clone then start container-logs-clone (kind:Mandatory)
  start container-logs-clone then start cqe-frontend-stage (kind:Mandatory)
  start dockerd-clone then start cqe-frontend-stage (kind:Mandatory)
  start mysql-g-stage then start cqe-frontend-stage (kind:Mandatory)
  start container-logs-clone then start cqe-frontend-live (kind:Mandatory)
  start dockerd-clone then start cqe-frontend-live (kind:Mandatory)
  start mysql-g-live then start cqe-frontend-live (kind:Mandatory)
  start cqe-frontend-live then start cqe-dispatcher-live (kind:Mandatory)
  start cqe-frontend-stage then start cqe-dispatcher-stage (kind:Mandatory)
Colocation Constraints:
  clvmd-clone with dlm-clone (score:INFINITY)
  mysql-g-stage with clvmd-clone (score:INFINITY)
  mysql-g-live with clvmd-clone (score:INFINITY)
  mysql-g-live with mysql-g-stage (score:-10)
  container-logs-clone with dlm-clone (score:INFINITY)
  cqe-frontend-stage with container-logs-clone (score:INFINITY)
  cqe-frontend-stage with dockerd-clone (score:INFINITY)
  cqe-frontend-live with container-logs-clone (score:INFINITY)
  cqe-frontend-live with dockerd-clone (score:INFINITY)
  cqe-dispatcher-live with container-logs-clone (score:INFINITY)
  cqe-dispatcher-live with dockerd-clone (score:INFINITY)
  cqe-dispatcher-stage with container-logs-clone (score:INFINITY)
  cqe-dispatcher-stage with dockerd-clone (score:INFINITY)
Ticket Constraints:

Alerts:
 No alerts defined

Resources Defaults:
 No defaults set
Operations Defaults:
 No defaults set

Cluster Properties:
 cluster-infrastructure: corosync
 cluster-name: el-cluster
 dc-version: 1.1.18-11.el7-2b07d5c5a9
 have-watchdog: false
 last-lrm-refresh: 1525441101

Quorum:
  Options:
    auto_tie_breaker: 0

Comment 7 David Teigland 2018-05-16 15:21:07 UTC

In general yes, but clvm was not designed to do exclusive activation, and it was only added later, so it has never fit very naturally, and there can still be issues with it.  I would suggest only using gfs in clvm cluster VGs for gfs, and not doing exclusive activation in them.  However, if you provide the output of the failing command, then we could give some better advice about this; it's possible that it's some other issue entirely.

Comment 15 michal novacek 2018-09-11 08:58:46 UTC

Created attachment 1482316 [details]
pcs cluster report

Comment 19 RHEL Program Management 2021-02-15 07:38:46 UTC

After evaluating this issue, there are no plans to address it further or fix it in an upcoming release.  Therefore, it is being closed.  If plans change such that this issue will be fixed in an upcoming release, then the bug can be reopened.