Attachment 827169 Details for Bug 1033016 – commands run on cluster node explaining more in detail setup and what happened wrong

commands run on cluster node explaining more in detail setup and what happened wrong

ha-nfsserver.commands (text/plain), 8.36 KB, created by michal novacek on 2013-11-21 12:28:12 UTC

(hide)

Description:

Filename:

MIME Type:

Creator: michal novacek

Created: 2013-11-21 12:28:12 UTC

Size: 8.36 KB

patch

obsolete

>
>host-055$ pcs resource show vip havg nfs-shared-fs nfs-export nfs-server
> Resource: vip (class=ocf provider=heartbeat type=IPaddr2)
>  Attributes: ip=10.15.112.227 cidr_netmask=24 
>  Operations: monitor interval=30s (vip-monitor-interval-30s)
> Resource: havg (class=ocf provider=heartbeat type=LVM)
>  Attributes: volgrpname=shared exclusive=true 
>  Operations: monitor interval=60s (havg-monitor-interval-60s)
> Resource: nfs-shared-fs (class=ocf provider=heartbeat type=Filesystem)
>  Attributes: device=/dev/shared/shared0 directory=/mnt/shared0 fstype=ext4 options= 
>  Operations: monitor interval=30s (nfs-shared-fs-monitor-interval-30s)
> Resource: nfs-export (class=ocf provider=heartbeat type=exportfs)
>  Attributes: directory=/mnt/shared0 clientspec=* options=rw,async,no_all_squash fsid=290 
>  Operations: monitor interval=60s (nfs-export-monitor-interval-60s)
> Resource: nfs-server (class=ocf provider=heartbeat type=nfsserver)
>  Attributes: nfs_ip=10.15.112.227 nfs_shared_infodir=/mnt/shared0/nfs nfs_notify_cmd=/usr/sbin/sm-notify 
>  Operations: monitor interval=30s (nfs-server-monitor-interval-30s)
>
>host-055$  pcs status
>Cluster name: STSRHTS28726
>Last updated: Thu Nov 21 06:09:39 2013
>Last change: Thu Nov 21 05:51:07 2013 via crm_resource on host-055
>Stack: corosync
>Current DC: host-055 (2) - partition with quorum
>Version: 1.1.10-20.el7-368c726
>3 Nodes configured
>12 Resources configured
>
>Online: [ host-054 host-055 host-056 ]
>
>Full list of resources:
>
> virt-fencing   (stonith:fence_xvm):    Started host-054 
> Clone Set: dlm-clone [dlm]
>     Started: [ host-054 host-055 host-056 ]
> Clone Set: clvmd-clone [clvmd]
>     Started: [ host-054 host-055 host-056 ]
> Resource Group: ha-nfsserver
>     vip        (ocf::heartbeat:IPaddr2):       Started on host-055 
>     havg       (ocf::heartbeat:LVM):   Started on host-055 
>     nfs-shared-fs      (ocf::heartbeat:Filesystem):    Started on host-055  
>     nfs-server (ocf::heartbeat:nfsserver):     Stopped 
>     nfs-export (ocf::heartbeat:exportfs):      Stopped 
>
>
>host-055$ mount 
>...
>/dev/mapper/shared-shared0 on /mnt/shared0 type ext4 (rw,relatime,seclabel,data=ordered)
>
>host-055$ ls -l /mnt/shared0/nfs/
>total 12
>drwxr-x--x. 2 root    root    4096 Nov 21 05:37 rpc_pipefs
>drwxr-x--x. 5 rpcuser rpcuser 4096 Nov 21 05:37 statd
>drwxr-x--x. 2 root    root    4096 Nov 21 05:37 v4recovery
>
>host-055$ pcs resource debug-start nfs-server
>Operation start for nfs-server (ocf:heartbeat:nfsserver) returned 1
> >  stderr: DEBUG: nfs-lock.service - NFS file locking service. Loaded: loaded (/usr/lib/systemd/system/nfs-lock.service; disabled) Active: inactive (dead) since Thu 2013-11-21 05:44:02 CST; 1min 15s ago Process: 29062 ExecStopPost=/sbin/sysctl -w fs.nfs.nlm_tcpport=0 fs.nfs.nlm_udpport=0 (code=exited, status=0/SUCCESS) Process: 29013 ExecStart=/sbin/rpc.statd $STATDARG (code=exited, status=0/SUCCESS) Process: 29011 ExecStartPre=/usr/libexec/nfs-utils/scripts/nfs-lock.preconfig (code=exited, status=0/SUCCESS) Main PID: 29016 (code=exited, status=0/SUCCESS) Nov 21 05:44:02 host-055 systemd[1]: Starting NFS file locking service.... Nov 21 05:44:02 host-055 rpc.statd[29016]: Version 1.2.9 starting Nov 21 05:44:02 host-055 systemd[1]: Started NFS file locking service.. Nov 21 05:44:02 host-055 systemd[1]: Stopping NFS file locking service.... Nov 21 05:44:02 host-055 sysctl[29062]: fs.nfs.nlm_tcpport = 0 Nov 21 05:44:02 host-055 sysctl[29062]: fs.nfs.nlm_udpport = 0 Nov 21 05:44:02 host-055 systemd[1]: Stopped NFS file locking service.. nfs-server.service - NFS Server Loaded: loaded (/usr/lib/systemd/system/nfs-server.service; disabled) Active: inactive (dead) Nov 21 05:37:43 host-055 systemd[1]: Stopped NFS Server. Nov 21 05:37:51 host-055 systemd[1]: Dependency failed for NFS Server. Nov 21 05:37:51 host-055 systemd[1]: Stopped NFS Server. Nov 21 05:43:54 host-055 systemd[1]: Stopped NFS Server. Nov 21 05:44:02 host-055 systemd[1]: Dependency failed for NFS Server. Nov 21 05:44:02 host-055 systemd[1]: Stopped NFS Server.
> >  stderr: INFO: Starting NFS server ...
> >  stderr: DEBUG: A dependency job for nfs-server.service failed. See 'journalctl -xn' for details. ERROR: Systemd services nfs-lock and nfs-server are not in the same state after attempting start command
> >  stderr: ERROR: Failed to start NFS server
>
>host-055$ journalctl -xn
>-- Logs begin at Wed 2013-11-20 11:05:12 CST, end at Thu 2013-11-21 05:45:18 CST. --
>Nov 21 05:45:18 host-055 rpc.mountd[30007]: couldn't open /var/lib/nfs/etab
>Nov 21 05:45:18 host-055 systemd[1]: nfs-mountd.service: control process exited, code=exited status=1
>Nov 21 05:45:18 host-055 systemd[1]: Failed to start NFS Mount Daemon.
>-- Subject: Unit nfs-mountd.service has failed
>-- Defined-By: systemd
>-- Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel
>-- Documentation: http://www.freedesktop.org/wiki/Software/systemd/catalog/be02cf6855d2428ba40df7e9d022f03d
>-- 
>-- Unit nfs-mountd.service has failed.
>-- 
>-- The result is failed.
>Nov 21 05:45:18 host-055 systemd[1]: Dependency failed for NFS Server.
>-- Subject: Unit nfs-server.service has failed
>-- Defined-By: systemd
>-- Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel
>-- Documentation: http://www.freedesktop.org/wiki/Software/systemd/catalog/be02cf6855d2428ba40df7e9d022f03d
>-- 
>-- Unit nfs-server.service has failed.
>-- 
>-- The result is dependency.
>Nov 21 05:45:18 host-055 systemd[1]: Dependency failed for NFS Remote Quota Server.
>-- Subject: Unit nfs-rquotad.service has failed
>-- Defined-By: systemd
>-- Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel
>-- Documentation: http://www.freedesktop.org/wiki/Software/systemd/catalog/be02cf6855d2428ba40df7e9d022f03d
>-- 
>-- Unit nfs-rquotad.service has failed.
>-- 
>-- The result is dependency.
>Nov 21 05:45:18 host-055 systemd[1]: Dependency failed for NFSv4 ID-name mapping daemon.
>-- Subject: Unit nfs-idmap.service has failed
>-- Defined-By: systemd
>-- Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel
>-- Documentation: http://www.freedesktop.org/wiki/Software/systemd/catalog/be02cf6855d2428ba40df7e9d022f03d
>-- 
>-- Unit nfs-idmap.service has failed.
>-- 
>-- The result is dependency.
>Nov 21 05:45:18 host-055 systemd[1]: Unit nfs-mountd.service entered failed state.
>
>host-055$ ls -l /var/lib/nfs /mnt/shared0/nfs
>/mnt/shared0/nfs:
>total 12
>drwxr-x--x. 2 root    root    4096 Nov 21 05:37 rpc_pipefs
>drwxr-x--x. 5 rpcuser rpcuser 4096 Nov 21 05:37 statd
>drwxr-x--x. 2 root    root    4096 Nov 21 05:37 v4recovery
>
>/var/lib/nfs:
>total 12
>drwxr-x--x. 2 root    root    4096 Nov 21 05:37 rpc_pipefs
>drwxr-x--x. 5 rpcuser rpcuser 4096 Nov 21 05:37 statd
>drwxr-x--x. 2 root    root    4096 Nov 21 05:37 v4recovery
>
>host-055$ touch /var/lib/nfs/etab
>
>host-055$ pcs resource cleanup nfs-server
>Error: Unable to cleanup resource: nfs-server
>Waiting for 4 replies from the CRMd...No messages received in 60 seconds.. aborting
>Cleaning up nfs-server on host-054
>Cleaning up nfs-server on host-055
>Cleaning up nfs-server on host-056
>
>$ pcs status
>Cluster name: STSRHTS28726
>Last updated: Thu Nov 21 06:20:38 2013
>Last change: Thu Nov 21 06:20:23 2013 via crm_resource on host-054
>Stack: corosync
>Current DC: host-055 (2) - partition with quorum
>Version: 1.1.10-20.el7-368c726
>3 Nodes configured
>12 Resources configured
>
>
>Online: [ host-054 host-055 host-056 ]
>
>Full list of resources:
>
> virt-fencing   (stonith:fence_xvm):    Started host-054 
> Clone Set: dlm-clone [dlm]
>     Started: [ host-054 host-055 host-056 ]
> Clone Set: clvmd-clone [clvmd]
>     Started: [ host-054 host-055 host-056 ]
> Resource Group: ha-nfsserver
>     vip        (ocf::heartbeat:IPaddr2):       Started host-055 
>     havg       (ocf::heartbeat:LVM):   Started host-055 
>     nfs-shared-fs      (ocf::heartbeat:Filesystem):    Started host-055 
>     nfs-server (ocf::heartbeat:nfsserver):     Started host-055 
>     nfs-export (ocf::heartbeat:exportfs):      Started host-055 
>
>Failed actions:
>
>host-055$ pcs resource disable ha-nfsserver
>host-055$ mount | grep /var/lib/nfs
>host-055$ ls -l /var/lib/nfs/
>total 0
>-rw-r--r--.  1 root    root     0 Nov 12 12:47 etab
>-rw-r--r--.  1 root    root     0 Nov 12 12:47 rmtab
>dr-xr-xr-x. 10 root    root     0 Nov 20 11:05 rpc_pipefs
>drwx------.  4 rpcuser rpcuser 40 Nov 20 10:34 statd
>-rw-r--r--.  1 rpcuser rpcuser  0 Nov 12 12:47 state
>drwxr-xr-x.  2 root    root     6 Nov 12 12:47 v4recovery
>-rw-r--r--.  1 root    root     0 Nov 12 12:47 xtab
>host-055$ rpm -qf /var/lib/nfs/etab
>nfs-utils-1.2.9-0.3.el7.x86_64
>

Actions: View

Attachments on bug 1033016: 827169