Bug 1676892

Summary: ctdb tries to delete files in RO filesystem (var-lib-nfs-rpc_pipefs.mount)
Product: [Community] GlusterFS Reporter: Strahil Nikolov <hunter86_bg>
Component: gluster-smbAssignee: Guenther Deschner <gdeschner>
Status: CLOSED UPSTREAM QA Contact:
Severity: medium Docs Contact:
Priority: unspecified    
Version: mainlineCC: amitkuma, anoopcs, asn, bugs, dpal, gdeschner, jarrpa, jstephen, madam, pasik, rhs-smb, sheggodu, vdas
Target Milestone: ---Keywords: ZStream
Target Release: ---   
Hardware: x86_64   
OS: Linux   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2020-03-17 03:35:48 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Strahil Nikolov 2019-02-13 14:14:08 UTC
Description of problem:
ctdb tries to delete files in /var/lib/nfs/rpc_pipefs/. It dies with error - permission denied and fails to start.

Version-Release number of selected component (if applicable):
ctdb-4.8.3-4.el7.x86_64

How reproducible:
Always (tried on 3 gluster nodes)

Steps to Reproduce:
1.Install storhaug , ctdb & nfs-ganesha
2.Deploy via storhaug (https://github.com/gluster/storhaug/wiki):
- yum -y install storhaug-nfs
- ssh-keygen -f /etc/sysconfig/storhaug.d/secret.pem
- ssh-copy-id -i /etc/sysconfig/storhaug.d/secret.pem.pub root@nodeX
Repeat for all nodes
- populate /etc/ctdb/nodes and /etc/ctdb/public_addresses
- Configure /etc/ctdb/ctdbd.conf:
 grep -Ev "^#|^$" /etc/ctdb/ctdbd.conf
CTDB_RECOVERY_LOCK="/run/gluster/shared_storage/lock"
CTDB_NODES=/etc/ctdb/nodes
CTDB_PUBLIC_ADDRESSES=/etc/ctdb/public_addresses
CTDB_MANAGES_NFS=yes
CTDB_NFS_CALLOUT=/etc/ctdb/nfs-ganesha-callout
CTDB_NFS_STATE_FS_TYPE=glusterfs
CTDB_NFS_STATE_MNT=/run/gluster/shared_storage
CTDB_NFS_SKIP_SHARE_CHECK=yes
NFS_HOSTNAME=localhost
CTDB_DEBUG_LEVEL=DEBUG

-touch /etc/ganesha/ganesha.conf
-gluster volume set all cluster.enable-shared-storage enable
-storhaug setup

3.Start the ctdb.service (if not running)

Actual results:
CTDB defines in the log that it tries to cleanup /var/lib/nfs/rpc_pipefs and fails.

Expected results:
CTDB service file should contain:
[Unit]
Conflicts=var-lib-nfs-rpc_pipefs.mount

Additional info:
If the var-lib-nfs-rpc_pipefs.mount is not running - the ctdb.service starts properly.
My current ctdb service is:
[root@ovirt1 ~s]# systemctl cat ctdb.service
# /etc/systemd/system/ctdb.service
[Unit]
Description=CTDB
Documentation=man:ctdbd(1) man:ctdb(7)
After=network-online.target time-sync.target
Conflicts=var-lib-nfs-rpc_pipefs.mount

[Service]
Type=forking
LimitCORE=infinity
PIDFile=/run/ctdb/ctdbd.pid
ExecStartPre=/bin/bash -c "sleep 2; if [ -f /sys/fs/cgroup/cpu/system.slice/cpu.rt_runtime_us ]; then echo 10000 > /sys/fs/cgroup/cpu/system.slice/cpu.rt_runtime_us; fi"
ExecStart=/usr/sbin/ctdbd_wrapper /run/ctdb/ctdbd.pid start
ExecStop=/usr/sbin/ctdbd_wrapper /run/ctdb/ctdbd.pid stop
KillMode=control-group
Restart=no

[Install]
WantedBy=multi-user.target

Comment 1 Sumit Bose 2019-02-13 17:25:36 UTC
ctdb is now build as part of the samba package.

Comment 4 Sunil Kumar Acharya 2020-02-19 14:04:25 UTC
Moving the BZ to upstream

Comment 5 Worker Ant 2020-03-17 03:35:48 UTC
This bug is moved to https://github.com/gluster/glusterfs/issues/1117, and will be tracked there from now on. Visit GitHub issues URL for further details