Bug 1038959

Summary: samba browsing blocked even with the samba-client service enabled, works with connection/interface in 'trusted' zone
Product: [Fedora] Fedora Reporter: Adam Williamson <awilliam>
Component: NetworkManagerAssignee: Dan Williams <dcbw>
Status: CLOSED DUPLICATE QA Contact: Fedora Extras Quality Assurance <extras-qa>
Severity: high Docs Contact:
Priority: unspecified    
Version: 20CC: abokovoy, dcbw, gansalmon, garrett.mitchener, giallu, itamar, jlayton, jolsa, jonathan, jpopelka, kaffeetisch, kernel-maint, madhu.chinakonda, nhorman, remjg, sergio, twoerner, vash63
Target Milestone: ---Keywords: CommonBugs
Target Release: ---   
Hardware: x86_64   
OS: Linux   
See Also: https://bugzilla.redhat.com/show_bug.cgi?id=1297235
Whiteboard: https://fedoraproject.org/wiki/Common_F20_bugs#smb-browse-firewalld
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2014-01-10 08:29:58 EST Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Attachments:
Description Flags
stock.pcapng: wireshark capture of 'smbtree -N' attempt with completely stock settings
none
samba-client.pcapng: wireshark capture of 'smbtree -N' attempt with 'samba-client' service allowed in firewalld config
none
trusted.pcapng: wireshark capture of 'smbtree -N' attempt with interface in trusted zone none

Description Adam Williamson 2013-12-06 04:04:19 EST
So I've kinda been noticing this bug forever, but only decided to dig into it today...and it looks pretty weird.

The problem is samba browsing, and how it doesn't work. If I run 'smbtree -N' on any Fedora system on my network with firewalld enabled, I get either nothing or just the shares provided by the machine it's run on.

I've verified this in a completely fresh environment: a clean VM booting a Fedora 20 Final TC5 live image. Run 'smbtree -N', nada. Open nautilus and try browsing 'Windows Network', you get an error.

Okay, so firewalld has the 'samba-client' service. OK. Enable that for the appropriate zone, re-run smbtree -N, and...tada!...still nada.

Hmm.

Maybe we need 'samba' too? Nope. Doesn't help.

Only if I put the network connection in the 'trusted' zone does it work.

I'll attach wireshark captures of the mentioned configurations, to see if we can isolate what exactly is the difference. 192.168.1.13 is the test machine, 192.168.1.9 is my NAS (Thecus N5550) providing various shares, 192.168.1.2 is a Windows 7 machine on the network (SAMMY-PC). Anything else is likely noise.

I felt like for sure someone must've reported or at least noticed this before, but I just can't find any discussion of it. But it's certainly completely broken for me, and I have multiple SMB/CIFS hosts on my network running a variety of server software, and I'm pretty sure I saw the same thing when I took my laptop to the UK and connected to my family's network; I couldn't browse there either.
Comment 1 Adam Williamson 2013-12-06 04:13:00 EST
So the obvious smoking gun I see in the wireshark output: in every failure log I've seen, there is an attempt by the client (machine trying to browse) to ping (ICMP) a server that just responded to its NBNS broadcast query, and that ping failing. In the success log, I see neither a successful nor a failed ping. See packet #5 in the 'samba-client service allowed' log.
Comment 2 Adam Williamson 2013-12-06 04:13:46 EST
Created attachment 833479 [details]
stock.pcapng: wireshark capture of 'smbtree -N' attempt with completely stock settings
Comment 3 Adam Williamson 2013-12-06 04:14:27 EST
Created attachment 833480 [details]
samba-client.pcapng: wireshark capture of 'smbtree -N' attempt with 'samba-client' service allowed in firewalld config
Comment 4 Adam Williamson 2013-12-06 04:15:01 EST
Created attachment 833481 [details]
trusted.pcapng: wireshark capture of 'smbtree -N' attempt with interface in trusted zone
Comment 5 Adam Williamson 2013-12-06 04:20:53 EST
well, maybe it's not just a ping at that. I'm no expert at reading wireshark captures, but it looks like the ICMP packet has an NBNS packet inside it.
Comment 6 Adam Williamson 2013-12-06 04:30:20 EST
Ah, OK, I think I grok it better now. Compare packet #5 in 'samba-client' and #12 in 'trusted'. I think they're the same packet - some kind of NBNS setup stuff - getting incorrectly blocked in 'samba-client'. wireshark's identification of it as 'ICMP' in 'samba-client' is a result of it being blocked.
Comment 7 Adam Williamson 2013-12-06 04:46:49 EST
It looks like the packet is rejected as "Host administratively prohibited". I guess that lines up with the rules at the end of the INPUT and FORWARD chains of firewalld's iptables config:

Chain INPUT (policy ACCEPT)
target     prot opt source               destination         
ACCEPT     all  --  anywhere             anywhere             ctstate RELATED,ESTABLISHED
ACCEPT     all  --  anywhere             anywhere            
INPUT_direct  all  --  anywhere             anywhere            
INPUT_ZONES_SOURCE  all  --  anywhere             anywhere            
INPUT_ZONES  all  --  anywhere             anywhere            
ACCEPT     icmp --  anywhere             anywhere            
REJECT     all  --  anywhere             anywhere             reject-with icmp-host-prohibited

Chain FORWARD (policy ACCEPT)
target     prot opt source               destination         
ACCEPT     all  --  anywhere             anywhere             ctstate RELATED,ESTABLISHED
ACCEPT     all  --  anywhere             anywhere            
FORWARD_direct  all  --  anywhere             anywhere            
FORWARD_IN_ZONES_SOURCE  all  --  anywhere             anywhere            
FORWARD_IN_ZONES  all  --  anywhere             anywhere            
FORWARD_OUT_ZONES_SOURCE  all  --  anywhere             anywhere            
FORWARD_OUT_ZONES  all  --  anywhere             anywhere            
ACCEPT     icmp --  anywhere             anywhere            
REJECT     all  --  anywhere             anywhere             reject-with icmp-host-prohibited

those are the only "reject-with icmp-host-prohibited" rules I see in iptables...
Comment 8 Jiri Popelka 2013-12-06 09:13:03 EST
My guess is that there's some problem with the NetBIOS name service broadcast connection tracking helper - nf_conntrack_netbios_ns.

Adam, could you make sure it's loaded (lsmod | grep nf_conntrack_netbios_ns)
when you enable the samba-client service ?
Comment 9 Adam Williamson 2013-12-06 12:29:52 EST
The module appears to be loaded on boot - when I boot a live image it's loaded already. I don't see any kind of change in its status when I check/uncheck samba-client in firewall-config.
Comment 10 Sergio Monteiro Basto 2013-12-08 22:15:22 EST
Hi, 

From https://sourceforge.net/p/smb4k/wiki/TroubleShooting_EmptyBrowser/

Open Ports in Your Firewall

Open the ports 137 (TCP+UDP), 138 (UDP), 139 (TCP+UDP), and 445 (TCP
+UDP) in your firewall.

firewalld config don't have a couple of this rules from samba or
samba-client , I add some ports and protocols  in service samba
like 137 TCP 139 UDP 445 UDP , but no luck ...

after go to here: http://www.novell.com/coolsolutions/feature/11952.html
 
Set the firewall to allow ICMP (Internet Control Message Protocol) with following types.

    0 (Echo request)
    8 (Echo Reply)
    3 (Destination unreachable)(This one is optional but useful)

Allow Traffic for the Windows network by opening the following ports for SMB both TCP and UDP

Port Number Traffic Type
445 Microsoft-DS
135 DCE endpoint resolution
136
137 NETBIOS Name Service
138 NETBIOS Datagram Service
139 NETBIOS session service

but still just works with "trusted" zone 

lsmod | grep nf_conntrack_netbios_ns
nf_conntrack_netbios_ns    12665  0 
nf_conntrack_broadcast    12527  1 nf_conntrack_netbios_ns
nf_conntrack           86430  11 nf_conntrack_netbios_ns,ipt_MASQUERADE,nf_nat,nf_nat_ipv4,nf_nat_ipv6,xt_conntrack,ip6table_nat,nf_conntrack_broadcast,iptable_nat,nf_conntrack_ipv4,nf_conntrack_ipv6

The problem unless trusted zone , firewall blocks here :
11	13.536262000	10.0.2.15	192.168.1.253	ICMP	132	Destination unreachable (Host administratively prohibited)

I can I unblock it ?
Comment 11 Sergio Monteiro Basto 2013-12-08 22:18:00 EST
The problem is firewall blocks here :
11	13.536262000	10.0.2.15	192.168.1.253	ICMP	132	Destination unreachable (Host administratively prohibited)

just don't block in "trusted" zone

How I can unblock it ? for other zone
Comment 12 Adam Williamson 2013-12-11 17:20:54 EST
Thomas: is there anything more I can do to help with this? It seems like a significant bug and I was kinda happy to finally trace it out to some extent, I was hoping we could get it fixed fairly soon (for F20 final ideally, but I think that ship has sailed). Thanks!
Comment 13 Adam Williamson 2013-12-12 07:27:57 EST
So, we poked at this quite a bit in IRC today.

We're not to the bottom of it yet, but it's not totally straightforward. This seems to be specific to F20, in some way.

If I boot an F19 live image on a clean test system and run 'smbtree -N' - absolutely no configuration of anything required, not even the 'samba-client' firewalld service - I see my servers fine.

If I boot an F20 RC1 live image on the same system, same servers, same everything, and try the same thing, I see nothing.

Adding the network interface to the 'trusted' zone works fine every time, so it's definitely the firewall causing problems, but it's not as straightforward as some firewalld config being wrong. Right now we suspect that nf_conntrack_netbios_ns is behaving differently in F19 and F20 somehow...
Comment 14 Thomas Woerner 2013-12-12 10:01:47 EST
Adding a rule to accept all from souce port 137/udp makes it work again, but this is not a good solution.
Comment 15 Josh Boyer 2013-12-12 10:12:16 EST
(In reply to Adam Williamson from comment #13)
> So, we poked at this quite a bit in IRC today.
> 
> We're not to the bottom of it yet, but it's not totally straightforward.
> This seems to be specific to F20, in some way.
> 
> If I boot an F19 live image on a clean test system and run 'smbtree -N' -
> absolutely no configuration of anything required, not even the
> 'samba-client' firewalld service - I see my servers fine.
> 
> If I boot an F20 RC1 live image on the same system, same servers, same
> everything, and try the same thing, I see nothing.
> 
> Adding the network interface to the 'trusted' zone works fine every time, so
> it's definitely the firewall causing problems, but it's not as
> straightforward as some firewalld config being wrong. Right now we suspect
> that nf_conntrack_netbios_ns is behaving differently in F19 and F20
> somehow...

When you booted your "F19 live image", was that the original F19 GA live image?  Because that has a different kernel (much older) kernel than F20.

At this point F19 and F20 are on virtually identical kernels, so figuring out if this also impacts an updated F19 would be good.  If it doesn't it's likely something elsewhere that changed between F19 and F20.

If the kernel version does matter, then you can upgrade kernel major versions to see where the suspected change came in at.  Though looking at the git tree, there hasn't been a change to net/netfilter/nf_conntrack_netbios_ns.c since 2011 (2.6.39) so I'm not sure that's really the culprit.

Adding Neil, Jiri, and Jeff to CC.
Comment 16 Josh Boyer 2013-12-12 10:19:57 EST
Thomas mentioned on IRC that 3.12 from koji is working fine on F19.  He's going to try the F20 kernel on F19 as well.
Comment 17 Josh Boyer 2013-12-12 15:37:54 EST
Thomas mentioned the F20 kernel on F19 works as well.  This is looking like an issue in something other than the kernel.
Comment 18 Adam Williamson 2013-12-12 16:02:48 EST
jwb: when I tested a 19 live image it was GA, yeah, but a fully updated 19 install running 3.11.10-200.fc19 behaves the same (i.e. works). I'd tend to agree that as of now it doesn't look like the issue is in the kernel.
Comment 19 Adam Williamson 2013-12-12 19:10:48 EST
As this is actually a new bug in F20 and we didn't figure out a fix yet, we should document this...
Comment 20 Adam Williamson 2013-12-13 03:56:53 EST
I tested this in RHEL 7 public beta too, just for the heck of it. Bug seems to exist there too, nothing from 'smbtree -N' with OOTB config or 'samba-client' enabled, works if I stop firewalld.service (couldn't test putting it in 'restricted' zone easily because it seems incredibly RAM hungry, and when I boot it to GNOME the VM exhausts all RAM and freezes after like 2 minutes). Not sure how important this is to RHEL.
Comment 21 Alexander Bokovoy 2013-12-17 08:00:52 EST
(In reply to Thomas Woerner from comment #14)
> Adding a rule to accept all from souce port 137/udp makes it work again, but
> this is not a good solution.
Not sure why you are claiming it is not a good solution since it is a requirement: NetBT browsing requires 137/udp connectivity by design since 137/udp is the port used for name services in NetBIOS over TCP/IP.

See this TechNet article for more details:
http://technet.microsoft.com/en-us/library/cc940063.aspx
Comment 22 Michael Cronenworth 2013-12-19 13:18:21 EST
*** Bug 1023918 has been marked as a duplicate of this bug. ***
Comment 23 RĂ©mi G. 2013-12-27 09:08:54 EST
Samba browsing with Nautilus is working again for me since one week.
Comment 24 Thomas Woerner 2014-01-09 10:30:34 EST
I have verified this a bit more.

On a machine, where it is working, the incoming packets from port 137 are accepted with the RELATED,ESTABLISHED line in the firewall. On a system, where is not working the packet is not accepted by this line.

The firewall configuration is the same on both machines.

This is not a firewalld issue. It seems to be a problem somewhere in netfilter (kernel).

Reassigning to kernel.
Comment 25 Thomas Woerner 2014-01-10 06:42:53 EST
With the latest updates in F-20 (including kernel 3.12.6-300) samba browsing is working now as expected. There has not been an update of iptables or firewalld.
Comment 26 Thomas Woerner 2014-01-10 08:29:58 EST
This was a bug in NetworkManager. The broadcast address of the interfaces has been set to 0.0.0.0.
Using the latest NetworkManager update or fixing the broadcast address manually fixes the problem.

Reassigning to NetworkManager. Closing as duplicate for 1032819.

*** This bug has been marked as a duplicate of bug 1032819 ***