Bug 968023

Summary: NFSv3 mount of installation repository fails unless 'nolock' option is passed
Product: [Fedora] Fedora Reporter: Adam Williamson <awilliam>
Component: anacondaAssignee: Chris Lumens <clumens>
Status: CLOSED ERRATA QA Contact: Fedora Extras Quality Assurance <extras-qa>
Severity: high Docs Contact:
Priority: unspecified    
Version: 19CC: anaconda-maint-list, clumens, dshea, g.kaviyarasu, jonathan, mkolman, orion, sbueno, vanmeeuwen+fedora
Target Milestone: ---Keywords: CommonBugs
Target Release: ---   
Hardware: All   
OS: All   
Whiteboard: AcceptedFreezeException https://fedoraproject.org/wiki/Common_F19_bugs#nfsv3-nolock-fail
Fixed In Version: anaconda-19.30.3-1.fc19 Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2013-06-12 03:39:01 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---
Bug Depends On:    
Bug Blocks: 834091    
Attachments:
Description Flags
anaconda.log
none
anaconda.program.log
none
syslog none

Description Adam Williamson 2013-05-28 19:41:12 UTC
Wow, so this one confused me for a while.

So I think it works like this. "mount.nfs", if you call it with no nfsvers= option, will first try an NFSv4 mount, and then fall back to doing an NFSv3 mount if that fails. If you try to configure an NFS repo - either by passing inst.repo=nfs:foo (so long as there's also an inst.root or inst.stage2 parameter and 811242 doesn't come into play) or by using the Installation Source spoke - anaconda will just go ahead and call out to 'mount.nfs' , so it ought to get this behaviour, and people's NFS repos ought to work whether they're v3 or v4.

However, it looks like NFSv3 mounts don't actually work unless rpc.statd is running or the 'nolock' option is passed.

To reproduce - I set up an NFS server capable of both NFSv3 and NFSv4, containing a Fedora ISO in the directory /export/fedora , and with this /etc/exports:

/export 192.168.1.0/24(ro,sync,insecure,root_squash,no_subtree_check,fsid=0)
/export/fedora 192.168.1.0/24(rw,nohide,sync,insecure,root_squash,no_subtree_check)

Now, if I pass 'inst.repo=nfs:192.168.1.105:/fedora' - which will succeed as an NFSv4 mount - or enter '192.168.1.105:/fedora' as an NFS repo on Installation Destination, it works fine.

If I pass 'inst.repo=nfs:192.168.1.105:/export/fedora' - which would succeed only as an NFSv3 mount - it should try to mount that as NFSv4, fail, and fall back to mounting it as NFSv3. Indeed, it actually tries to. However, it fails, and I get this output in program.log:

19:23:03,215 INFO program: Running... mount -t nfs -o  192.168.1.105:/export/fedora /mnt/install/source
19:23:03,418 INFO program: mount.nfs: rpc.statd is not running but is required for remote locking.
19:23:03,419 INFO program: mount.nfs: Either use '-o nolock' to keep locks local, or start statd.
19:23:03,419 INFO program: mount.nfs: mounting 192.168.1.105:/export/fedora failed, reason given by server: No such file or directory

Now that stuff about statd and nolock is apparently important. Because if I pass 'inst.repo=nfs:nolock:192.168.1.105:/export/fedora' - note, I have still not specified an NFS protocol version explicitly, but I have passed 'nolock' - it works! The mount succeeds (with no output but a 0 return code) and the repo is happily accepted by anaconda. So it seems like mount.nfs' mechanism for v4->v3 fallback is working fine, but by not starting rpc.statd or using the 'nolock' option for NFS mounts, anaconda is not allowing v3 mounts to work without the user doing it instead.

So, can we do something here? Start rpc.statd if an NFS repo is being configured? Pass in 'nolock' for NFS mounts? Or are both of those a problem somehow and we should document the use of 'nolock' manually for NFSv3 repos? Thanks!

Setting severity 'high' as this behaviour is extremely non-obvious and could cause considerable frustration for users with NFSv3-only mounts.

Comment 1 Adam Williamson 2013-05-28 19:42:41 UTC
Nominating as final FE as anaconda has their 'freeze' in place; if we change this it would be beneficial to do it for F19 final.

Comment 2 Adam Williamson 2013-05-28 22:53:56 UTC
https://lists.fedorahosted.org/pipermail/anaconda-patches/2013-May/004028.html should fix this, but was not yet ack'ed and committed. I will test it with an updates.img .

Comment 3 Adam Williamson 2013-05-28 23:12:51 UTC
http://www.happyassassin.net/extras/updates-968023.img (including clumens' patch from c#2) fixes this for me.

Comment 4 Adam Williamson 2013-05-30 19:03:40 UTC
also confirmed that NFSv4 mounts still work fine with the update in place.

Comment 5 Adam Williamson 2013-05-30 19:08:51 UTC
Discussed at 2013-05-30 freeze exception review meeting: http://meetbot.fedoraproject.org/fedora-blocker-review/2013-05-30/f19final-blocker-review-1.1.2013-05-30-16.02.log.txt . Accepted as a freeze exception issue as this will significantly complicate the use of NFS repos in Final if not fixed.

Comment 6 Adam Williamson 2013-06-04 00:49:20 UTC
The patch for this has been stuck in review since May 3, it looks like:

https://lists.fedorahosted.org/pipermail/anaconda-patches/2013-May/004032.html

was the last message in the thread, Brian never followed up after that. Chris, do you maybe want to give this one a kick to get it reviewed and applied? It'd be really good to have this one fixed soon.

Comment 7 Fedora Update System 2013-06-06 18:37:52 UTC
anaconda-19.30.3-1.fc19 has been submitted as an update for Fedora 19.
https://admin.fedoraproject.org/updates/anaconda-19.30.3-1.fc19

Comment 8 Fedora Update System 2013-06-07 15:42:28 UTC
Package anaconda-19.30.3-1.fc19:
* should fix your issue,
* was pushed to the Fedora 19 testing repository,
* should be available at your local mirror within two days.
Update it with:
# su -c 'yum update --enablerepo=updates-testing anaconda-19.30.3-1.fc19'
as soon as you are able to.
Please go to the following url:
https://admin.fedoraproject.org/updates/FEDORA-2013-10283/anaconda-19.30.3-1.fc19
then log in and leave karma (feedback).

Comment 9 Orion Poplawski 2013-06-11 03:03:18 UTC
Something appears to have broken mounting nfs4 mounts for me with 19.30.3, perhaps this fix broke it.  In my ks %post I'm now getting:

# Make sure the rpc_pipefs filesystem is available for rpc.idmapd
/bin/mount -t rpc_pipefs sunrpc /var/lib/nfs/rpc_pipefs
+ /bin/mount -t rpc_pipefs sunrpc /var/lib/nfs/rpc_pipefs
# Start rpc.idmapd for nfs4
rpc.idmapd -v
+ rpc.idmapd -v
rpc.idmapd: libnfsidmap: using (default) domain: cora.nwra.com
rpc.idmapd: libnfsidmap: Realms list: 'CORA.NWRA.COM'
rpc.idmapd: libnfsidmap: loaded plugin /lib64/libnfsidmap/nsswitch.so for method nsswitch

+ mount -v -t nfs4 saga:/backup /data/backup
mount.nfs4: mount(2): No such device
mount.nfs4: No such device
mount.nfs4: timeout set for Fri Jun  7 14:41:50 2013
mount.nfs4: trying text-based options 'addr=10.10.10.2,clientaddr=10.10.11.103'

I'm also doing a modprobe nfs in %pre.

Comment 10 Orion Poplawski 2013-06-11 03:04:08 UTC
Created attachment 759407 [details]
anaconda.log

Comment 11 Orion Poplawski 2013-06-11 03:04:53 UTC
Created attachment 759408 [details]
anaconda.program.log

Comment 12 Orion Poplawski 2013-06-11 03:16:12 UTC
Created attachment 759409 [details]
syslog

Comment 13 Orion Poplawski 2013-06-11 20:57:39 UTC
Okay, today's install went fine, so hopefully no worries.  I'll keep testing...

Comment 14 Fedora Update System 2013-06-12 03:39:01 UTC
anaconda-19.30.3-1.fc19 has been pushed to the Fedora 19 stable repository.  If problems still persist, please make note of it in this bug report.