Bug 701309

Summary: rpc_pipefs is not re-mounted properly when using nfs root.
Product: Red Hat Enterprise Linux 6 Reporter: Ondrej Valousek <ondrejv>
Component: dracutAssignee: Harald Hoyer <harald>
Status: CLOSED ERRATA QA Contact: Release Test Team <release-test-team-automation>
Severity: low Docs Contact:
Priority: medium    
Version: 6.0CC: atodorov, pknirsch, rwilliam
Target Milestone: rc   
Target Release: ---   
Hardware: x86_64   
OS: Linux   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
If the system was booted with root on NFS and a separate /var partition, the rpc pipefs was not accessible anymore. The dracut initramfs mounted the rpc pipefs on /var/lib/nfs/rpc_pipefs and the system later on mounts the /var partition over /var, making the original not accessible anymore. The dracut initramfs now mounts entries in /etc/fstab.sys, which provides a way for the administrator to mount /var on the root filesystem before /var/lib/nfs/rpc_pipefs is mounted.
Story Points: ---
Clone Of: Environment:
Last Closed: 2011-12-06 16:45:44 UTC Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Ondrej Valousek 2011-05-02 14:37:55 UTC
Description of problem:
I am booting a diskless RHEL-6 environment using dracut this way:
[victim@data ~]$ cat /proc/cmdline
initrd=rhel6-ws-boot/initramfs.img console=ttyS0,115200 console=tty0
root=nfs4:192.168.60.50:/vol/vol1/OS/rhel6/root crashkernel=256M@16M nomodeset
vesa BOOT_IMAGE=rhel6-ws-boot/vmlinuz
The system boots up correctly with / mounted via nfs4 but rpc_pipefs is not mounted correctly to /var/lib/nfs/rpc_pipefs.

Strangely enough, it "looks and feels" that it is mounted:

[root@data ~]# mount | grep rpc
sunrpc on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw,relatime)

... but is not really:

[root@data ~]# umount /var/lib/nfs/rpc_pipefs
umount: /var/lib/nfs/rpc_pipefs not mounted

and no NFS4 ID mapping is working as well. After re-mounting manually this way:

[root@data ~]# mount -t rpc_pipefs sunrpc /var/lib/nfs/rpc_pipefs

and restarting rpc.idmapd, the ID mapping starts working again.

Version-Release number of selected component (if applicable):
[ondrejv@data ~]$ rpm -qi dracut
Name        : dracut                       Relocations: (not relocatable)
Version     : 004                               Vendor: Red Hat, Inc.
Release     : 33.2.el6_0                    Build Date: 

How reproducible:
always

Comment 3 Harald Hoyer 2011-05-03 07:39:04 UTC
(In reply to comment #0)
> Strangely enough, it "looks and feels" that it is mounted:
> 
> [root@data ~]# mount | grep rpc
> sunrpc on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw,relatime)

Please check with "/proc/mounts" or "/proc/self/mountinfo" ... "/etc/mtab" could be out of sync.

Comment 4 Ondrej Valousek 2011-05-03 08:08:22 UTC
They are:
[root@data ~]# cat /proc/mounts | grep rpc
sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0
[root@data ~]# cat /etc/mtab | grep rpc
sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0
[root@data ~]# cat /proc/self/mounts | grep rpc
sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0
[root@data ~]# cat /proc/self/mountinfo | grep rpc
22 24 0:18 / /var/lib/nfs/rpc_pipefs rw,relatime - rpc_pipefs sunrpc rw

Looks like I am facing a bug similar to #583351 - can we get an updated dracut into RHEL-6 as well?

Comment 5 RHEL Program Management 2011-05-13 23:09:04 UTC
This request was evaluated by Red Hat Product Management for inclusion
in a Red Hat Enterprise Linux maintenance release. Product Management has 
requested further review of this request by Red Hat Engineering, for potential
inclusion in a Red Hat Enterprise Linux Update release for currently deployed 
products. This request is not yet committed for inclusion in an Update release.

Comment 8 Harald Hoyer 2011-08-10 13:29:04 UTC
    Technical note added. If any revisions are required, please edit the "Technical Notes" field
    accordingly. All revisions will be proofread by the Engineering Content Services team.
    
    New Contents:
If the system was booted with root on NFS and a separate /var partition, the rpc pipefs was not accessible anymore. The dracut initramfs mounted the rpc pipefs on /var/lib/nfs/rpc_pipefs and the system later on mounts the /var partition over /var, making the original not accessible anymore. The dracut initramfs now mounts entries in /etc/fstab.sys, which provides a way for the administrator to mount /var on the root filesystem before /var/lib/nfs/rpc_pipefs is mounted.

Comment 10 Ondrej Valousek 2011-10-20 09:43:24 UTC
Ok I agree this is not a problem with dracut but rather with /etc/rc.d/rc.sysinit.
I have filed a new bug 747552 against the initscripts.

Comment 11 Alexander Todorov 2011-11-03 13:58:49 UTC
Setup procedure with KVM guest: 

1) Prepare guest with 2 disk images
2) Make /boot, swap and /var on the first disk
3) Make / on the second disk
4) Install dracut-network package and re-create initrd so that it includes the necessary nfs dracut modules

5) Edit grub.conf and add an entry for boot with NFS root using the new initrd and specify root=nfs4:IPADDR:/share
6) Edit /etc/fstab and remove the line for /.

7) Power off the guest and remove the second disk (where / is)

8) Mount the root partition from 2nd disk on the host and export via NFS - This gives us a ready to go nfsroot (e.g. use kpartx if the disk image is a file).
/mnt/root	*(sync,rw,no_root_squash)

9) Boot the virtual guest (now having only one local disk) and select the boot entry which mounts / over nfs. 

10) When booted inspect /var/lib/nfs/rpc_pipefs.

Comment 12 Alexander Todorov 2011-11-03 14:50:47 UTC
(In reply to comment #11)

> 6) Edit /etc/fstab and remove the line for /.

Don't remove but edit like so:

nfs:10.10.9.126:/mnt/root		  /                       ext4    defaults        0 0


Also in /etc/sysconfig/network-scripts/ifcfg-eth0 set:
ONBOOT=no


Then I see at several places:
effective UID is -2

(16,47,35) harald: " effective UID is -2" that's _the_ bug :)

Comment 13 Alexander Todorov 2011-11-03 15:38:31 UTC
Next step:

1) Upgraded to latest packages in snap #4
2) Created /etc/fstab.sys file to mount /var
3) Recreated the initramfs
4) Boot into runlevel 1

On the console I see:

dracut: Mounted root filesystem 10.10.9.126:/mnt/root/
dracut: Mounting from /sysroot/etc/fstab.sys
EXT4-fs (vda3): recovery complete
EXT4-fs (vda3): mounted filesystem with ordered data mode. Opts: 
dracut: /dev/vda3 on /sysroot/var type ext4 (rw)

Later: 
# ls -l /var/lib/nfs/rpc_pipefs/
total 0
dr-xr-xr-x. 3 root root 0 Nov  3 11:36 cache
dr-xr-xr-x. 2 root root 0 Nov  3 11:36 lockd
dr-xr-xr-x. 2 root root 0 Nov  3 11:36 mount
dr-xr-xr-x. 4 root root 0 Nov  3 11:36 nfs
dr-xr-xr-x. 2 root root 0 Nov  3 11:36 nfsd4_cb
dr-xr-xr-x. 2 root root 0 Nov  3 11:36 portmap
dr-xr-xr-x. 2 root root 0 Nov  3 11:36 statd


According to Harald this indicated that the dracut bug is fixed.

Comment 14 errata-xmlrpc 2011-12-06 16:45:44 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

http://rhn.redhat.com/errata/RHBA-2011-1521.html