Bug 1400068 - [GANESHA] Adding a node to cluster failed to allocate resource-agents to new node.
Summary: [GANESHA] Adding a node to cluster failed to allocate resource-agents to new ...
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat Gluster Storage
Classification: Red Hat Storage
Component: common-ha
Version: rhgs-3.2
Hardware: Unspecified
OS: Unspecified
urgent
unspecified
Target Milestone: ---
: RHGS 3.2.0
Assignee: Soumya Koduri
QA Contact: Manisha Saini
URL:
Whiteboard:
Depends On: 1403130 1404133
Blocks: 1351528
TreeView+ depends on / blocked
 
Reported: 2016-11-30 12:00 UTC by Manisha Saini
Modified: 2017-03-23 05:53 UTC (History)
12 users (show)

Fixed In Version: glusterfs-3.8.4-9
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
: 1403130 (view as bug list)
Environment:
Last Closed: 2017-03-23 05:53:16 UTC
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Red Hat Product Errata RHSA-2017:0486 0 normal SHIPPED_LIVE Moderate: Red Hat Gluster Storage 3.2.0 security, bug fix, and enhancement update 2017-03-23 09:18:45 UTC

Description Manisha Saini 2016-11-30 12:00:11 UTC
Description of problem:

Adding a node to cluster does not show shared shorage on new node even when cluster is running on new node

No Simlinks are created from
ganesha.conf -> /var/run/gluster/shared_storage/nfs-ganesha/ganesha.conf 

Pacemaker and corosync services are started on new node.


Version-Release number of selected component (if applicable):

glusterfs-ganesha-3.8.4-5.el7rhgs.x86_64

How reproducible:


Steps to Reproduce:
1.Create a 4 node cluster and configure ganesha on it.

2.Do prerequisite to run ganesha on new node like passwordless-ssh from one of the node in cluster to new node

# systemctl disable NetworkManager
# systemctl stop NetworkManager
# systemctl enable network
# systemctl start network
# systemctl enable pacemaker.service
# systemctl start pcsd

3.From one of the node in cluster do-

pcs cluster auth dhcp46-165.lab.eng.blr.redhat.com dhcp47-3.lab.eng.blr.redhat.com dhcp46-241.lab.eng.blr.redhat.com dhcp46-219.lab.eng.blr.redhat.com

4.Perform following steps on new node

# sed -i '/STATD_PORT/s/^#//' /etc/sysconfig/nfs
# systemctl restart nfs-config
# systemctl restart rpc-statd

Edit '/etc/sysconfig/nfs' using following commands:

# sed -i '/STATD_PORT/s/^#//' /etc/sysconfig/nfs
# sed -i '/LOCKD_TCPPORT/s/^#//' /etc/sysconfig/nfs
# sed -i '/LOCKD_UDPPORT/s/^#//' /etc/sysconfig/nfs
# sed -i '/MOUNTD_PORT/s/^#//' /etc/sysconfig/nfs

# systemctl restart nfs-config
# systemctl restart rpc-statd
# systemctl restart nfs-mountd
# systemctl restart nfslock

4.From one of the node in cluster run-
# /usr/libexec/ganesha/ganesha-ha.sh --add /var/run/gluster/shared_storage/nfs-ganesha/ dhcp46-165.lab.eng.blr.redhat.com 10.70.44.157


Logs:
# /usr/libexec/ganesha/ganesha-ha.sh --add /var/run/gluster/shared_storage/nfs-ganesha/ dhcp46-165.lab.eng.blr.redhat.com 10.70.44.157

Permission denied (publickey,gssapi-keyex,gssapi-with-mic,password).
Permission denied (publickey,gssapi-keyex,gssapi-with-mic,password).
Disabling SBD service...
dhcp46-165.lab.eng.blr.redhat.com: sbd disabled
dhcp46-241.lab.eng.blr.redhat.com: Corosync updated
dhcp46-219.lab.eng.blr.redhat.com: Corosync updated
dhcp47-3.lab.eng.blr.redhat.com: Corosync updated
dhcp47-159.lab.eng.blr.redhat.com: Corosync updated
Setting up corosync...
dhcp46-165.lab.eng.blr.redhat.com: Succeeded
Synchronizing pcsd certificates on nodes dhcp46-165.lab.eng.blr.redhat.com...
dhcp46-165.lab.eng.blr.redhat.com: Success

Restarting pcsd on the nodes in order to reload the certificates...
dhcp46-165.lab.eng.blr.redhat.com: Success
dhcp46-165.lab.eng.blr.redhat.com: Starting Cluster...
Removing Constraint - order-nfs-grace-clone-dhcp46-219.lab.eng.blr.redhat.com-cluster_ip-1-mandatory
Deleting Resource - dhcp46-219.lab.eng.blr.redhat.com-cluster_ip-1
Removing Constraint - order-nfs-grace-clone-dhcp46-241.lab.eng.blr.redhat.com-cluster_ip-1-mandatory
Deleting Resource - dhcp46-241.lab.eng.blr.redhat.com-cluster_ip-1
Removing Constraint - order-nfs-grace-clone-dhcp47-159.lab.eng.blr.redhat.com-cluster_ip-1-mandatory
Deleting Resource - dhcp47-159.lab.eng.blr.redhat.com-cluster_ip-1
Removing Constraint - order-nfs-grace-clone-dhcp47-3.lab.eng.blr.redhat.com-cluster_ip-1-mandatory
Deleting Resource - dhcp47-3.lab.eng.blr.redhat.com-cluster_ip-1
Adding nfs-grace-clone dhcp46-219.lab.eng.blr.redhat.com-cluster_ip-1 (kind: Mandatory) (Options: first-action=start then-action=start)
Adding nfs-grace-clone dhcp46-241.lab.eng.blr.redhat.com-cluster_ip-1 (kind: Mandatory) (Options: first-action=start then-action=start)
Adding nfs-grace-clone dhcp47-159.lab.eng.blr.redhat.com-cluster_ip-1 (kind: Mandatory) (Options: first-action=start then-action=start)
Adding nfs-grace-clone dhcp47-3.lab.eng.blr.redhat.com-cluster_ip-1 (kind: Mandatory) (Options: first-action=start then-action=start)
Adding nfs-grace-clone dhcp46-165.lab.eng.blr.redhat.com-cluster_ip-1 (kind: Mandatory) (Options: first-action=start then-action=start)
Error: duplicate constraint already exists, use --force to override
  Constraint: location-dhcp46-219.lab.eng.blr.redhat.com-group
    Rule: score=-INFINITY  (id:location-dhcp46-219.lab.eng.blr.redhat.com-group-rule)
      Expression: ganesha-active ne 1  (id:location-dhcp46-219.lab.eng.blr.redhat.com-group-rule-expr)
Error: duplicate constraint already exists, use --force to override
  Constraint: location-dhcp46-241.lab.eng.blr.redhat.com-group
    Rule: score=-INFINITY  (id:location-dhcp46-241.lab.eng.blr.redhat.com-group-rule)
      Expression: ganesha-active ne 1  (id:location-dhcp46-241.lab.eng.blr.redhat.com-group-rule-expr)
Error: duplicate constraint already exists, use --force to override
  Constraint: location-dhcp47-159.lab.eng.blr.redhat.com-group
    Rule: score=-INFINITY  (id:location-dhcp47-159.lab.eng.blr.redhat.com-group-rule)
      Expression: ganesha-active ne 1  (id:location-dhcp47-159.lab.eng.blr.redhat.com-group-rule-expr)
Error: duplicate constraint already exists, use --force to override
  Constraint: location-dhcp47-3.lab.eng.blr.redhat.com-group
    Rule: score=-INFINITY  (id:location-dhcp47-3.lab.eng.blr.redhat.com-group-rule)
      Expression: ganesha-active ne 1  (id:location-dhcp47-3.lab.eng.blr.redhat.com-group-rule-expr)
Error: Resource 'dhcp46-165.lab.eng.blr.redhat.com-group' does not exist
Error: Resource 'dhcp46-165.lab.eng.blr.redhat.com-group' does not exist
Error: Resource 'dhcp46-165.lab.eng.blr.redhat.com-group' does not exist
Error: Resource 'dhcp46-165.lab.eng.blr.redhat.com-group' does not exist
Error: Resource 'dhcp46-165.lab.eng.blr.redhat.com-group' does not exist
Error: Resource 'dhcp46-165.lab.eng.blr.redhat.com-group' does not exist
CIB updated


# pcs status
Cluster name: ganesha-ha-360
Stack: corosync
Current DC: dhcp46-241.lab.eng.blr.redhat.com (version 1.1.15-11.el7_3.2-e174ec8) - partition with quorum
Last updated: Wed Nov 30 17:23:42 2016		Last change: Wed Nov 30 16:47:30 2016 by root via crm_attribute on dhcp46-165.lab.eng.blr.redhat.com

5 nodes and 28 resources configured

Online: [ dhcp46-165.lab.eng.blr.redhat.com dhcp46-219.lab.eng.blr.redhat.com dhcp46-241.lab.eng.blr.redhat.com dhcp47-159.lab.eng.blr.redhat.com dhcp47-3.lab.eng.blr.redhat.com ]

Full list of resources:

 Clone Set: nfs_setup-clone [nfs_setup]
     Started: [ dhcp46-165.lab.eng.blr.redhat.com dhcp46-219.lab.eng.blr.redhat.com dhcp46-241.lab.eng.blr.redhat.com dhcp47-159.lab.eng.blr.redhat.com dhcp47-3.lab.eng.blr.redhat.com ]
 Clone Set: nfs-mon-clone [nfs-mon]
     Started: [ dhcp46-165.lab.eng.blr.redhat.com dhcp46-219.lab.eng.blr.redhat.com dhcp46-241.lab.eng.blr.redhat.com dhcp47-159.lab.eng.blr.redhat.com dhcp47-3.lab.eng.blr.redhat.com ]
 Clone Set: nfs-grace-clone [nfs-grace]
     Started: [ dhcp46-219.lab.eng.blr.redhat.com dhcp46-241.lab.eng.blr.redhat.com dhcp47-159.lab.eng.blr.redhat.com dhcp47-3.lab.eng.blr.redhat.com ]
     Stopped: [ dhcp46-165.lab.eng.blr.redhat.com ]
 Resource Group: dhcp46-241.lab.eng.blr.redhat.com-group
     dhcp46-241.lab.eng.blr.redhat.com-nfs_block	(ocf::heartbeat:portblock):	Started dhcp46-241.lab.eng.blr.redhat.com
     dhcp46-241.lab.eng.blr.redhat.com-nfs_unblock	(ocf::heartbeat:portblock):	Started dhcp46-241.lab.eng.blr.redhat.com
 Resource Group: dhcp46-219.lab.eng.blr.redhat.com-group
     dhcp46-219.lab.eng.blr.redhat.com-nfs_block	(ocf::heartbeat:portblock):	Started dhcp46-219.lab.eng.blr.redhat.com
     dhcp46-219.lab.eng.blr.redhat.com-nfs_unblock	(ocf::heartbeat:portblock):	Started dhcp46-219.lab.eng.blr.redhat.com
 Resource Group: dhcp47-3.lab.eng.blr.redhat.com-group
     dhcp47-3.lab.eng.blr.redhat.com-nfs_block	(ocf::heartbeat:portblock):	Started dhcp47-3.lab.eng.blr.redhat.com
     dhcp47-3.lab.eng.blr.redhat.com-nfs_unblock	(ocf::heartbeat:portblock):	Started dhcp47-3.lab.eng.blr.redhat.com
 Resource Group: dhcp47-159.lab.eng.blr.redhat.com-group
     dhcp47-159.lab.eng.blr.redhat.com-nfs_block	(ocf::heartbeat:portblock):	Started dhcp47-159.lab.eng.blr.redhat.com
     dhcp47-159.lab.eng.blr.redhat.com-nfs_unblock	(ocf::heartbeat:portblock):	Started dhcp47-159.lab.eng.blr.redhat.com
 dhcp46-219.lab.eng.blr.redhat.com-cluster_ip-1	(ocf::heartbeat:IPaddr):	Started dhcp46-165.lab.eng.blr.redhat.com
 dhcp46-241.lab.eng.blr.redhat.com-cluster_ip-1	(ocf::heartbeat:IPaddr):	Started dhcp46-165.lab.eng.blr.redhat.com
 dhcp47-159.lab.eng.blr.redhat.com-cluster_ip-1	(ocf::heartbeat:IPaddr):	Started dhcp46-165.lab.eng.blr.redhat.com
 dhcp47-3.lab.eng.blr.redhat.com-cluster_ip-1	(ocf::heartbeat:IPaddr):	Started dhcp47-3.lab.eng.blr.redhat.com
 dhcp46-165.lab.eng.blr.redhat.com-cluster_ip-1	(ocf::heartbeat:IPaddr):	Started dhcp46-165.lab.eng.blr.redhat.com

Failed Actions:
* dhcp46-241.lab.eng.blr.redhat.com-nfs_unblock_monitor_10000 on dhcp46-241.lab.eng.blr.redhat.com 'unknown error' (1): call=103, status=Timed Out, exitreason='none',
    last-rc-change='Tue Nov 29 18:20:25 2016', queued=0ms, exec=0ms
* dhcp47-3.lab.eng.blr.redhat.com-nfs_block_start_0 on dhcp46-241.lab.eng.blr.redhat.com 'insufficient privileges' (4): call=109, status=complete, exitreason='none',
    last-rc-change='Tue Nov 29 15:39:18 2016', queued=0ms, exec=56ms
* dhcp47-159.lab.eng.blr.redhat.com-nfs_unblock_monitor_10000 on dhcp47-159.lab.eng.blr.redhat.com 'unknown error' (1): call=91, status=Timed Out, exitreason='none',
    last-rc-change='Tue Nov 29 18:20:25 2016', queued=0ms, exec=0ms
* nfs-grace_monitor_0 on dhcp46-165.lab.eng.blr.redhat.com 'unknown error' (1): call=16, status=complete, exitreason='none',
    last-rc-change='Wed Nov 30 16:47:10 2016', queued=0ms, exec=102ms


Daemon Status:
  corosync: active/disabled
  pacemaker: active/disabled
  pcsd: active/enabled


Actual results:

New Node is added to cluster.But shared storage is not being reflected on new Node.
No Simlinks are created from
ganesha.conf -> /var/run/gluster/shared_storage/nfs-ganesha/ganesha.conf 


Expected results:

Shared storage should be reflected on new Node

Additional info:

Comment 2 Manisha Saini 2016-11-30 12:37:31 UTC
sosreport can be accessed at:

http://rhsqe-repo.lab.eng.blr.redhat.com/sosreports/1400068/

Comment 3 Manisha Saini 2016-12-01 07:43:42 UTC
According to Administration Guide, shared Volume should be mounted to new node after adding node to cluster.

So mount should be performed on newly added node manually and then shared_volume is reflected on new node.I missed this step while adding node to cluster for which i reported this bug.

But a new issue is also observed which is after adding node to cluster portmapper is not being assigned to newly added node in cluster.
Resources are not being allocated to new node in pcs status

Error: Resource 'dhcp47-3.lab.eng.blr.redhat.com-group' does not exist
Error: Resource 'dhcp47-3.lab.eng.blr.redhat.com-group' does not exist

So changing the summary of this Bug.


Logs-

Before Adding new node to cluster:

root@dhcp47-159 ganesha]# pcs status
Cluster name: ganesha-ha-360
Stack: corosync
Current DC: dhcp46-165.lab.eng.blr.redhat.com (version 1.1.15-11.el7_3.2-e174ec8) - partition with quorum
Last updated: Thu Dec  1 12:40:21 2016		Last change: Thu Dec  1 12:29:28 2016 by root via cibadmin on dhcp47-159.lab.eng.blr.redhat.com

4 nodes and 24 resources configured

Online: [ dhcp46-165.lab.eng.blr.redhat.com dhcp46-241.lab.eng.blr.redhat.com dhcp47-159.lab.eng.blr.redhat.com dhcp47-45.lab.eng.blr.redhat.com ]

Full list of resources:

 Clone Set: nfs_setup-clone [nfs_setup]
     Started: [ dhcp46-165.lab.eng.blr.redhat.com dhcp46-241.lab.eng.blr.redhat.com dhcp47-159.lab.eng.blr.redhat.com dhcp47-45.lab.eng.blr.redhat.com ]
 Clone Set: nfs-mon-clone [nfs-mon]
     Started: [ dhcp46-165.lab.eng.blr.redhat.com dhcp46-241.lab.eng.blr.redhat.com dhcp47-159.lab.eng.blr.redhat.com dhcp47-45.lab.eng.blr.redhat.com ]
 Clone Set: nfs-grace-clone [nfs-grace]
     Started: [ dhcp46-165.lab.eng.blr.redhat.com dhcp46-241.lab.eng.blr.redhat.com dhcp47-159.lab.eng.blr.redhat.com dhcp47-45.lab.eng.blr.redhat.com ]
 Resource Group: dhcp46-241.lab.eng.blr.redhat.com-group
     dhcp46-241.lab.eng.blr.redhat.com-nfs_block	(ocf::heartbeat:portblock):	Started dhcp46-241.lab.eng.blr.redhat.com
     dhcp46-241.lab.eng.blr.redhat.com-cluster_ip-1	(ocf::heartbeat:IPaddr):	Started dhcp46-241.lab.eng.blr.redhat.com
     dhcp46-241.lab.eng.blr.redhat.com-nfs_unblock	(ocf::heartbeat:portblock):	Started dhcp46-241.lab.eng.blr.redhat.com
 Resource Group: dhcp47-159.lab.eng.blr.redhat.com-group
     dhcp47-159.lab.eng.blr.redhat.com-nfs_block	(ocf::heartbeat:portblock):	Started dhcp47-159.lab.eng.blr.redhat.com
     dhcp47-159.lab.eng.blr.redhat.com-cluster_ip-1	(ocf::heartbeat:IPaddr):	Started dhcp47-159.lab.eng.blr.redhat.com
     dhcp47-159.lab.eng.blr.redhat.com-nfs_unblock	(ocf::heartbeat:portblock):	Started dhcp47-159.lab.eng.blr.redhat.com
 Resource Group: dhcp46-165.lab.eng.blr.redhat.com-group
     dhcp46-165.lab.eng.blr.redhat.com-nfs_block	(ocf::heartbeat:portblock):	Started dhcp46-165.lab.eng.blr.redhat.com
     dhcp46-165.lab.eng.blr.redhat.com-cluster_ip-1	(ocf::heartbeat:IPaddr):	Started dhcp46-165.lab.eng.blr.redhat.com
     dhcp46-165.lab.eng.blr.redhat.com-nfs_unblock	(ocf::heartbeat:portblock):	Started dhcp46-165.lab.eng.blr.redhat.com
 Resource Group: dhcp47-45.lab.eng.blr.redhat.com-group
     dhcp47-45.lab.eng.blr.redhat.com-nfs_block	(ocf::heartbeat:portblock):	Started dhcp47-45.lab.eng.blr.redhat.com
     dhcp47-45.lab.eng.blr.redhat.com-cluster_ip-1	(ocf::heartbeat:IPaddr):	Started dhcp47-45.lab.eng.blr.redhat.com
     dhcp47-45.lab.eng.blr.redhat.com-nfs_unblock	(ocf::heartbeat:portblock):	Started dhcp47-45.lab.eng.blr.redhat.com

Daemon Status:
  corosync: active/disabled
  pacemaker: active/disabled
  pcsd: active/enabled
[root@dhcp47-159 ganesha]# hostname
dhcp47-159.lab.eng.blr.redhat.com


After adding new node to cluster:

[root@dhcp46-241 ganesha]# /usr/libexec/ganesha/ganesha-ha.sh --add /var/run/gluster/shared_storage/nfs-ganesha/ dhcp47-3.lab.eng.blr.redhat.com 10.70.44.157
Disabling SBD service...
dhcp47-3.lab.eng.blr.redhat.com: sbd disabled
dhcp46-241.lab.eng.blr.redhat.com: Corosync updated
dhcp47-159.lab.eng.blr.redhat.com: Corosync updated
dhcp46-165.lab.eng.blr.redhat.com: Corosync updated
dhcp47-45.lab.eng.blr.redhat.com: Corosync updated
Setting up corosync...
dhcp47-3.lab.eng.blr.redhat.com: Succeeded
Synchronizing pcsd certificates on nodes dhcp47-3.lab.eng.blr.redhat.com...
dhcp47-3.lab.eng.blr.redhat.com: Success

Restarting pcsd on the nodes in order to reload the certificates...
dhcp47-3.lab.eng.blr.redhat.com: Success
dhcp47-3.lab.eng.blr.redhat.com: Starting Cluster...
Removing Constraint - order-nfs-grace-clone-dhcp46-165.lab.eng.blr.redhat.com-cluster_ip-1-mandatory
Deleting Resource - dhcp46-165.lab.eng.blr.redhat.com-cluster_ip-1
Removing Constraint - order-nfs-grace-clone-dhcp46-241.lab.eng.blr.redhat.com-cluster_ip-1-mandatory
Deleting Resource - dhcp46-241.lab.eng.blr.redhat.com-cluster_ip-1
Removing Constraint - order-nfs-grace-clone-dhcp47-159.lab.eng.blr.redhat.com-cluster_ip-1-mandatory
Deleting Resource - dhcp47-159.lab.eng.blr.redhat.com-cluster_ip-1
Removing Constraint - order-nfs-grace-clone-dhcp47-45.lab.eng.blr.redhat.com-cluster_ip-1-mandatory
Deleting Resource - dhcp47-45.lab.eng.blr.redhat.com-cluster_ip-1
Adding nfs-grace-clone dhcp46-165.lab.eng.blr.redhat.com-cluster_ip-1 (kind: Mandatory) (Options: first-action=start then-action=start)
Adding nfs-grace-clone dhcp46-241.lab.eng.blr.redhat.com-cluster_ip-1 (kind: Mandatory) (Options: first-action=start then-action=start)
Adding nfs-grace-clone dhcp47-159.lab.eng.blr.redhat.com-cluster_ip-1 (kind: Mandatory) (Options: first-action=start then-action=start)
Adding nfs-grace-clone dhcp47-45.lab.eng.blr.redhat.com-cluster_ip-1 (kind: Mandatory) (Options: first-action=start then-action=start)
Adding nfs-grace-clone dhcp47-3.lab.eng.blr.redhat.com-cluster_ip-1 (kind: Mandatory) (Options: first-action=start then-action=start)
Error: duplicate constraint already exists, use --force to override
  Constraint: location-dhcp46-165.lab.eng.blr.redhat.com-group
    Rule: score=-INFINITY  (id:location-dhcp46-165.lab.eng.blr.redhat.com-group-rule)
      Expression: ganesha-active ne 1  (id:location-dhcp46-165.lab.eng.blr.redhat.com-group-rule-expr)
Error: duplicate constraint already exists, use --force to override
  Constraint: location-dhcp46-241.lab.eng.blr.redhat.com-group
    Rule: score=-INFINITY  (id:location-dhcp46-241.lab.eng.blr.redhat.com-group-rule)
      Expression: ganesha-active ne 1  (id:location-dhcp46-241.lab.eng.blr.redhat.com-group-rule-expr)
Error: duplicate constraint already exists, use --force to override
  Constraint: location-dhcp47-159.lab.eng.blr.redhat.com-group
    Rule: score=-INFINITY  (id:location-dhcp47-159.lab.eng.blr.redhat.com-group-rule)
      Expression: ganesha-active ne 1  (id:location-dhcp47-159.lab.eng.blr.redhat.com-group-rule-expr)
Error: duplicate constraint already exists, use --force to override
  Constraint: location-dhcp47-45.lab.eng.blr.redhat.com-group
    Rule: score=-INFINITY  (id:location-dhcp47-45.lab.eng.blr.redhat.com-group-rule)
      Expression: ganesha-active ne 1  (id:location-dhcp47-45.lab.eng.blr.redhat.com-group-rule-expr)
Error: Resource 'dhcp47-3.lab.eng.blr.redhat.com-group' does not exist
Error: Resource 'dhcp47-3.lab.eng.blr.redhat.com-group' does not exist
Error: Resource 'dhcp47-3.lab.eng.blr.redhat.com-group' does not exist
Error: Resource 'dhcp47-3.lab.eng.blr.redhat.com-group' does not exist
Error: Resource 'dhcp47-3.lab.eng.blr.redhat.com-group' does not exist
Error: Resource 'dhcp47-3.lab.eng.blr.redhat.com-group' does not exist
CIB updated



===

[root@dhcp47-45 ganesha]# pcs status
Cluster name: ganesha-ha-360
Stack: corosync
Current DC: dhcp46-165.lab.eng.blr.redhat.com (version 1.1.15-11.el7_3.2-e174ec8) - partition with quorum
Last updated: Thu Dec  1 13:10:14 2016		Last change: Thu Dec  1 12:47:30 2016 by root via crm_attribute on dhcp47-3.lab.eng.blr.redhat.com

5 nodes and 28 resources configured

Online: [ dhcp46-165.lab.eng.blr.redhat.com dhcp46-241.lab.eng.blr.redhat.com dhcp47-159.lab.eng.blr.redhat.com dhcp47-3.lab.eng.blr.redhat.com dhcp47-45.lab.eng.blr.redhat.com ]

Full list of resources:

 Clone Set: nfs_setup-clone [nfs_setup]
     Started: [ dhcp46-165.lab.eng.blr.redhat.com dhcp46-241.lab.eng.blr.redhat.com dhcp47-159.lab.eng.blr.redhat.com dhcp47-3.lab.eng.blr.redhat.com dhcp47-45.lab.eng.blr.redhat.com ]
 Clone Set: nfs-mon-clone [nfs-mon]
     Started: [ dhcp46-165.lab.eng.blr.redhat.com dhcp46-241.lab.eng.blr.redhat.com dhcp47-159.lab.eng.blr.redhat.com dhcp47-3.lab.eng.blr.redhat.com dhcp47-45.lab.eng.blr.redhat.com ]
 Clone Set: nfs-grace-clone [nfs-grace]
     Started: [ dhcp46-165.lab.eng.blr.redhat.com dhcp46-241.lab.eng.blr.redhat.com dhcp47-159.lab.eng.blr.redhat.com dhcp47-3.lab.eng.blr.redhat.com dhcp47-45.lab.eng.blr.redhat.com ]
 Resource Group: dhcp46-241.lab.eng.blr.redhat.com-group
     dhcp46-241.lab.eng.blr.redhat.com-nfs_block	(ocf::heartbeat:portblock):	Started dhcp46-241.lab.eng.blr.redhat.com
     dhcp46-241.lab.eng.blr.redhat.com-nfs_unblock	(ocf::heartbeat:portblock):	Started dhcp46-241.lab.eng.blr.redhat.com
 Resource Group: dhcp47-159.lab.eng.blr.redhat.com-group
     dhcp47-159.lab.eng.blr.redhat.com-nfs_block	(ocf::heartbeat:portblock):	Started dhcp47-159.lab.eng.blr.redhat.com
     dhcp47-159.lab.eng.blr.redhat.com-nfs_unblock	(ocf::heartbeat:portblock):	Started dhcp47-159.lab.eng.blr.redhat.com
 Resource Group: dhcp46-165.lab.eng.blr.redhat.com-group
     dhcp46-165.lab.eng.blr.redhat.com-nfs_block	(ocf::heartbeat:portblock):	Started dhcp46-165.lab.eng.blr.redhat.com
     dhcp46-165.lab.eng.blr.redhat.com-nfs_unblock	(ocf::heartbeat:portblock):	Started dhcp46-165.lab.eng.blr.redhat.com
 Resource Group: dhcp47-45.lab.eng.blr.redhat.com-group
     dhcp47-45.lab.eng.blr.redhat.com-nfs_block	(ocf::heartbeat:portblock):	Started dhcp47-45.lab.eng.blr.redhat.com
     dhcp47-45.lab.eng.blr.redhat.com-nfs_unblock	(ocf::heartbeat:portblock):	Started dhcp47-45.lab.eng.blr.redhat.com
 dhcp46-165.lab.eng.blr.redhat.com-cluster_ip-1	(ocf::heartbeat:IPaddr):	Started dhcp47-3.lab.eng.blr.redhat.com
 dhcp46-241.lab.eng.blr.redhat.com-cluster_ip-1	(ocf::heartbeat:IPaddr):	Started dhcp47-3.lab.eng.blr.redhat.com
 dhcp47-159.lab.eng.blr.redhat.com-cluster_ip-1	(ocf::heartbeat:IPaddr):	Started dhcp47-159.lab.eng.blr.redhat.com
 dhcp47-45.lab.eng.blr.redhat.com-cluster_ip-1	(ocf::heartbeat:IPaddr):	Started dhcp47-45.lab.eng.blr.redhat.com
 dhcp47-3.lab.eng.blr.redhat.com-cluster_ip-1	(ocf::heartbeat:IPaddr):	Started dhcp46-165.lab.eng.blr.redhat.com

Failed Actions:
* nfs-grace_monitor_5000 on dhcp47-3.lab.eng.blr.redhat.com 'unknown error' (1): call=18, status=complete, exitreason='none',
    last-rc-change='Thu Dec  1 12:47:25 2016', queued=0ms, exec=0ms


Daemon Status:
  corosync: active/disabled
  pacemaker: active/disabled
  pcsd: active/enabled

Comment 7 Soumya Koduri 2016-12-09 06:56:30 UTC
Posted patch upstream for review -

http://review.gluster.org/16089

Comment 8 Atin Mukherjee 2016-12-13 09:16:31 UTC
upstream 3.9 : http://review.gluster.org/16115
downstream patch : https://code.engineering.redhat.com/gerrit/#/c/92793/

Comment 12 Manisha Saini 2016-12-20 06:43:23 UTC
Verified this bug on

# rpm -qa | grep ganesha
nfs-ganesha-2.4.1-3.el7rhgs.x86_64
glusterfs-ganesha-3.8.4-9.el7rhgs.x86_64
nfs-ganesha-gluster-2.4.1-3.el7rhgs.x86_64

As the issue is no more observed,hence marking this bug as verified.

Comment 19 errata-xmlrpc 2017-03-23 05:53:16 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://rhn.redhat.com/errata/RHSA-2017-0486.html


Note You need to log in before you can comment on or make changes to this bug.