Bug 866414
Summary: | Fedora18:Alpha: NFSV4 mount failed on client with Connection timed out | ||||||||||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
Product: | [Fedora] Fedora | Reporter: | IBM Bug Proxy <bugproxy> | ||||||||||||||||
Component: | kernel | Assignee: | Jeff Layton <jlayton> | ||||||||||||||||
Status: | CLOSED NOTABUG | QA Contact: | Fedora Extras Quality Assurance <extras-qa> | ||||||||||||||||
Severity: | urgent | Docs Contact: | |||||||||||||||||
Priority: | unspecified | ||||||||||||||||||
Version: | 18 | CC: | bfields, gansalmon, itamar, jkachuck, jlayton, jonathan, kernel-maint, madhu.chinakonda, steved, wgomerin | ||||||||||||||||
Target Milestone: | --- | ||||||||||||||||||
Target Release: | --- | ||||||||||||||||||
Hardware: | ppc64 | ||||||||||||||||||
OS: | All | ||||||||||||||||||
Whiteboard: | |||||||||||||||||||
Fixed In Version: | Doc Type: | Bug Fix | |||||||||||||||||
Doc Text: | Story Points: | --- | |||||||||||||||||
Clone Of: | Environment: | ||||||||||||||||||
Last Closed: | 2012-10-30 16:40:26 UTC | Type: | --- | ||||||||||||||||
Regression: | --- | Mount Type: | --- | ||||||||||||||||
Documentation: | --- | CRM: | |||||||||||||||||
Verified Versions: | Category: | --- | |||||||||||||||||
oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |||||||||||||||||
Cloudforms Team: | --- | Target Upstream Version: | |||||||||||||||||
Embargoed: | |||||||||||||||||||
Attachments: |
|
Description
IBM Bug Proxy
2012-10-15 10:21:21 UTC
Created attachment 627286 [details]
steps-to-reproduce.txt
Created attachment 627287 [details]
var-log-messages-nfsv4server.txt
Created attachment 627288 [details]
/dmesg-nfsv4server.txt
Created attachment 627289 [details]
var-log-messages-nfsv4client.txt
Created attachment 627290 [details]
dmesg-nfsv4client.txt
I'm a little confused by this: I see the following on the server side : ... Oct 12 08:58:02 miz12 rpc.mountd[1213]: Caught signal 15, un-registering and exiting. Oct 12 08:58:02 miz12 systemd[1]: nfs-mountd.service: main process exited, code=exited, status=1 Oct 12 08:58:02 miz12 systemd[1]: Unit nfs-mountd.service entered failed state. # ps -ef | egrep 'nfs|rpc|portmap|lockd' | grep -v grep ... root 4419 1 0 Oct12 ? 00:00:00 /usr/sbin/rpc.mountd Well, I guess I'll believe the logs and not ps (perhaps it was a failure from some earlier attempt?) Would it be possible to get a network trace showing the failure? (So, tcpdump -s0 -wtmp.pcap, then try the mount, then kill tcpdump and attach tmp.pcap.) ------- Comment From maknayak.com 2012-10-17 06:56 EDT------- (In reply to comment #10) > I'm a little confused by this: > > I see the following on the server side : > ... > Oct 12 08:58:02 miz12 rpc.mountd[1213]: Caught signal 15, un-registering and > exiting. > Oct 12 08:58:02 miz12 systemd[1]: nfs-mountd.service: main process exited, > code=exited, status=1 > Oct 12 08:58:02 miz12 systemd[1]: Unit nfs-mountd.service entered failed > state. > > # ps -ef | egrep 'nfs|rpc|portmap|lockd' | grep -v grep > ... > root 4419 1 0 Oct12 ? 00:00:00 /usr/sbin/rpc.mountd > > Well, I guess I'll believe the logs and not ps (perhaps it was a failure > from some earlier attempt?) > > Would it be possible to get a network trace showing the failure? (So, > tcpdump -s0 -wtmp.pcap, then try the mount, then kill tcpdump and attach > tmp.pcap.) Hello RedHat, while trying to mount nfsv4 mounts on client side , I captured tcpdump on NFSv4 client & server as in "nfsv4client-tmp.pcap" & nfsv4server-tmp.pcap respectively. I have attached the network trace logfile in nfsv4client-tmp.pcap to the bugzilla. Created attachment 628569 [details]
nfsv4server-tmp.pcap
------- Comment (attachment only) From maknayak.com 2012-10-17 06:57 EDT-------
Created attachment 628570 [details]
nfsv4client-tmp.pcap
------- Comment (attachment only) From maknayak.com 2012-10-17 06:58 EDT-------
Looks like a firewalling issue: 2005 2012-10-17 06:39:21.951176 9.3.110.195 9.3.110.107 TCP 723 > nfs [SYN] Seq=0 Win=14600 Len=0 MSS=1460 SACK_PERM=1 TSval=42441783 TSecr=0 WS=128 2006 2012-10-17 06:39:21.951224 9.3.110.107 9.3.110.195 ICMP Destination unreachable (Host administratively prohibited) The client is unable to open a socket to the server's NFS port, but the server is rejecting that with icmp-host-prohibited. Interestingly, I see the same issue if I try to mount a firewalled server over IPv4: # mount -v tlielax:/foo /mnt/foo -o vers=4,proto=tcp mount.nfs: timeout set for Wed Oct 17 06:35:39 2012 mount.nfs: trying text-based options 'vers=4,proto=tcp,addr=192.168.1.3,clientaddr=192.168.1.22' mount.nfs: mount(2): No route to host mount.nfs: trying text-based options 'vers=4,proto=tcp,addr=192.168.1.3,clientaddr=192.168.1.22' mount.nfs: mount(2): No route to host ...and it keeps repeating, I guess until it eventually times out. If I try ipv6 though, I get a different error and it fails immediately: # mount -v tlielax:/foo /mnt/foo -o vers=4,proto=tcp6 mount.nfs: timeout set for Wed Oct 17 06:37:27 2012 mount.nfs: trying text-based options 'vers=4,proto=tcp6,addr=2001:470:8:d63:3a60:77ff:fe93:a95d,clientaddr=2001:470:8:d63:5054:ff:fe9b:3976' mount.nfs: mount(2): Permission denied mount.nfs: access denied by server while mounting tlielax:/foo Ok, I think the difference there is due to the default firewall rules for the stock firewall in fedora. Either way, it certainly sounds like firewalling is the problem. Can you confirm whether that's the case? ------- Comment From maknayak.com 2012-10-26 02:49 EDT------- (In reply to comment #17) > Ok, I think the difference there is due to the default firewall rules for > the stock firewall in fedora. Either way, it certainly sounds like > firewalling is the problem. > > Can you confirm whether that's the case? While mounting NFSV4 directories ,both client & server had iptables services disabled. On NFSv4 Server: [root@miz12 ~]# systemctl status iptables iptables.service - IPv4 firewall with iptables Loaded: loaded (/usr/lib/systemd/system/iptables.service; disabled) Active: inactive (dead) CGroup: name=systemd:/system/iptables.service [root@miz12 ~]# systemctl status ip6tables ip6tables.service - IPv6 firewall with ip6tables Loaded: loaded (/usr/lib/systemd/system/ip6tables.service; disabled) Active: inactive (dead) CGroup: name=systemd:/system/ip6tables.service On NFSV4 Client: [root@miz09 ~]# systemctl status iptables.service iptables.service - IPv4 firewall with iptables Loaded: loaded (/usr/lib/systemd/system/iptables.service; disabled) Active: inactive (dead) CGroup: name=systemd:/system/iptables.service [root@miz09 ~]# [root@miz09 ~]# systemctl status ip6tables.service ip6tables.service - IPv6 firewall with ip6tables Loaded: loaded (/usr/lib/systemd/system/ip6tables.service; disabled) Active: inactive (dead) CGroup: name=systemd:/system/ip6tables.service Can you also run these commands on both hosts and paste the output here? # iptables -L ...also, are there any non-comment lines in /etc/hosts.allow or /etc/hosts.deny ? ------- Comment From maknayak.com 2012-10-30 14:35 EDT------- (In reply to comment #19) > Can you also run these commands on both hosts and paste the output here? > > # iptables -L > On server: [root@miz12 ~]# iptables -L Chain INPUT (policy ACCEPT) target prot opt source destination ACCEPT all -- anywhere anywhere ctstate RELATED,ESTABLISHED ACCEPT all -- anywhere anywhere INPUT_direct all -- anywhere anywhere INPUT_ZONES all -- anywhere anywhere ACCEPT icmp -- anywhere anywhere REJECT all -- anywhere anywhere reject-with icmp-host-prohibited Chain FORWARD (policy ACCEPT) target prot opt source destination ACCEPT all -- anywhere anywhere ctstate RELATED,ESTABLISHED ACCEPT all -- anywhere anywhere FORWARD_direct all -- anywhere anywhere FORWARD_ZONES all -- anywhere anywhere ACCEPT icmp -- anywhere anywhere REJECT all -- anywhere anywhere reject-with icmp-host-prohibited Chain OUTPUT (policy ACCEPT) target prot opt source destination OUTPUT_direct all -- anywhere anywhere Chain FORWARD_ZONES (1 references) target prot opt source destination FWDO_ZONE_public all -- anywhere anywhere FWDI_ZONE_public all -- anywhere anywhere Chain FORWARD_direct (1 references) target prot opt source destination Chain FWDI_ZONE_public (1 references) target prot opt source destination FWDI_ZONE_public_deny all -- anywhere anywhere FWDI_ZONE_public_allow all -- anywhere anywhere Chain FWDI_ZONE_public_allow (1 references) target prot opt source destination Chain FWDI_ZONE_public_deny (1 references) target prot opt source destination Chain FWDO_ZONE_external (0 references) target prot opt source destination FWDO_ZONE_external_deny all -- anywhere anywhere FWDO_ZONE_external_allow all -- anywhere anywhere Chain FWDO_ZONE_external_allow (1 references) target prot opt source destination ACCEPT all -- anywhere anywhere Chain FWDO_ZONE_external_deny (1 references) target prot opt source destination Chain FWDO_ZONE_public (1 references) target prot opt source destination FWDO_ZONE_public_deny all -- anywhere anywhere FWDO_ZONE_public_allow all -- anywhere anywhere Chain FWDO_ZONE_public_allow (1 references) target prot opt source destination Chain FWDO_ZONE_public_deny (1 references) target prot opt source destination Chain INPUT_ZONES (1 references) target prot opt source destination IN_ZONE_public all -- anywhere anywhere Chain INPUT_direct (1 references) target prot opt source destination Chain IN_ZONE_dmz (0 references) target prot opt source destination IN_ZONE_dmz_deny all -- anywhere anywhere IN_ZONE_dmz_allow all -- anywhere anywhere Chain IN_ZONE_dmz_allow (1 references) target prot opt source destination ACCEPT tcp -- anywhere anywhere tcp dpt:ssh ctstate NEW Chain IN_ZONE_dmz_deny (1 references) target prot opt source destination Chain IN_ZONE_external (0 references) target prot opt source destination IN_ZONE_external_deny all -- anywhere anywhere IN_ZONE_external_allow all -- anywhere anywhere Chain IN_ZONE_external_allow (1 references) target prot opt source destination ACCEPT tcp -- anywhere anywhere tcp dpt:ssh ctstate NEW Chain IN_ZONE_external_deny (1 references) target prot opt source destination Chain IN_ZONE_home (0 references) target prot opt source destination IN_ZONE_home_deny all -- anywhere anywhere IN_ZONE_home_allow all -- anywhere anywhere Chain IN_ZONE_home_allow (1 references) target prot opt source destination ACCEPT tcp -- anywhere anywhere tcp dpt:ssh ctstate NEW ACCEPT udp -- anywhere anywhere udp dpt:ipp ctstate NEW ACCEPT udp -- anywhere 224.0.0.251 udp dpt:mdns ctstate NEW ACCEPT udp -- anywhere anywhere udp dpt:netbios-ns ctstate NEW ACCEPT udp -- anywhere anywhere udp dpt:netbios-dgm ctstate NEW Chain IN_ZONE_home_deny (1 references) target prot opt source destination Chain IN_ZONE_internal (0 references) target prot opt source destination IN_ZONE_internal_deny all -- anywhere anywhere IN_ZONE_internal_allow all -- anywhere anywhere Chain IN_ZONE_internal_allow (1 references) target prot opt source destination ACCEPT tcp -- anywhere anywhere tcp dpt:ssh ctstate NEW ACCEPT udp -- anywhere anywhere udp dpt:ipp ctstate NEW ACCEPT udp -- anywhere 224.0.0.251 udp dpt:mdns ctstate NEW ACCEPT udp -- anywhere anywhere udp dpt:netbios-ns ctstate NEW ACCEPT udp -- anywhere anywhere udp dpt:netbios-dgm ctstate NEW Chain IN_ZONE_internal_deny (1 references) target prot opt source destination Chain IN_ZONE_public (1 references) target prot opt source destination IN_ZONE_public_deny all -- anywhere anywhere IN_ZONE_public_allow all -- anywhere anywhere Chain IN_ZONE_public_allow (1 references) target prot opt source destination ACCEPT tcp -- anywhere anywhere tcp dpt:ssh ctstate NEW ACCEPT udp -- anywhere 224.0.0.251 udp dpt:mdns ctstate NEW Chain IN_ZONE_public_deny (1 references) target prot opt source destination Chain IN_ZONE_work (0 references) target prot opt source destination IN_ZONE_work_deny all -- anywhere anywhere IN_ZONE_work_allow all -- anywhere anywhere Chain IN_ZONE_work_allow (1 references) target prot opt source destination ACCEPT tcp -- anywhere anywhere tcp dpt:ssh ctstate NEW ACCEPT udp -- anywhere 224.0.0.251 udp dpt:mdns ctstate NEW ACCEPT udp -- anywhere anywhere udp dpt:ipp ctstate NEW Chain IN_ZONE_work_deny (1 references) target prot opt source destination Chain OUTPUT_direct (1 references) target prot opt source destination > ...also, are there any non-comment lines in /etc/hosts.allow or > /etc/hosts.deny ? Nope there is no such non-comment lines. [root@miz12 ~]# cat /etc/hosts.allow # # hosts.allow This file contains access rules which are used to # allow or deny connections to network services that # either use the tcp_wrappers library or that have been # started through a tcp_wrappers-enabled xinetd. # # See 'man 5 hosts_options' and 'man 5 hosts_access' # for information on rule syntax. # See 'man tcpd' for information on tcp_wrappers # [root@miz12 ~]# cat /etc/hosts.deny # # hosts.deny This file contains access rules which are used to # deny connections to network services that either use # the tcp_wrappers library or that have been # started through a tcp_wrappers-enabled xinetd. # # The rules in this file can also be set up in # /etc/hosts.allow with a 'deny' option instead. # # See 'man 5 hosts_options' and 'man 5 hosts_access' # for information on rule syntax. # See 'man tcpd' for information on tcp_wrappers # Thanks... Manas The server clearly has some firewalling set up, even though iptables.service and ip6tables.service are disabled. I think you'll need to do some investigation to determine why that is and how to fix it to allow traffic to port 2049 in. At this point, I'm not seeing any evidence of a bug here. I'm going to go ahead and close this as NOTABUG. Please reopen if you find evidence to the contrary. |