Bug 676439

+++ This bug was initially created as a clone of Bug #675761 +++

Description of problem:

I have an F14 server running samba. Configuration is mostly Fedora-default (security = user, passdb backend = tdbsam, no printers and a couple of simple shares - nothing fancy). When I try to use this share with Windows 7 everything works just fine.
When I try to mount the share with a F14 workstation, I get a mount error(13): Permission denied error. On the server, there are no log entries for this failed attempt.

I used wireshark both on the server and on the workstation and discovered that the workstation doesn't do any network traffic to the server. It does query the DNS server to resolve the hostname of the server, and it gets the right IP address, but it sends no traffic to the samba server at all.

The mount command is as follows:

# mount -vvv -t cifs -o credentials=/etc/mnt/server.cred //server/share /mnt/server/user2
mount: fstab path: "/etc/fstab"
mount: mtab path:  "/etc/mtab"
mount: lock path:  "/etc/mtab~"
mount: temp path:  "/etc/mtab.tmp"
mount: UID:        0
mount: eUID:       0
mount: spec:  "//server/share"
mount: node:  "/mnt/server/user2"
mount: types: "cifs"
mount: opts:  "credentials=/etc/mnt/server.cred"
mount: external mount: argv[0] = "/sbin/mount.cifs"
mount: external mount: argv[1] = "//server/share"
mount: external mount: argv[2] = "/mnt/server/user2"
mount: external mount: argv[3] = "-v"
mount: external mount: argv[4] = "-o"
mount: external mount: argv[5] = "rw,credentials=/etc/mnt/server.cred"
domain=foobar
mount.cifs kernel mount options: ip=192.168.1.8,unc=\\server\share,credentials=/etc/mnt/server.cred,ver=1,user=user2,domain=foobar,pass=********
mount error(13): Permission denied
Refer to the mount.cifs(8) manual page (e.g. man mount.cifs)

I did an strace (-f) of this mount command, which resulted in the following:

http://pastebin.com/raw.php?i=HBey5Tpr

I can't make much of it, the only thing I can see is mount.cifs resolving the IP address of the server (server -> 192.168.1.8), but no traffic to the cifs server. Apparently it does call the kernel mount function, but that immediately returns -1 EACCESS, I don't know why.

I tried a couple of things that were suggested in the #fedora IRC channel, like temporarily disabling SELinux and fiddling with the ownership/permissions of the mountpoint, but to no avail. I also checked, the cifs kernel module was loaded.

Also I tried accessing the same share through konqueror with the smb://server/share syntax. This resulted in an authentication popup as expected, and subsequently I could access the share just fine. Also during this test I captured SMB-traffic between client and server, so I'm pretty sure that my wireshark was configured correctly and also there were no firewall issues. At least, I assume konqueror does roughly the same as mount.cifs would.

Version-Release number of selected component (if applicable):

Kernel: 2.6.35.10-74.fc14.x86_64 (both client and server)
cifs-utils-4.8.1-1.fc14.x86_64

How reproducible:

Always.

Steps to Reproduce:
1. Use mount.cifs on my workstation to mount any cifs share.
  
Actual results:

mount error(13): Permission denied

Expected results:

1. Access to the share.
2. At the very least I would have expected any network traffic to the server, or an error message explaining some client-side issue.

Additional info:

I don't know how to further diagnose. Please advice.

--- Additional comment from jlayton on 2011-02-07 16:22:20 EST ---

Most likely the authentication to the server is failing for some reason. Windows doesn't necessarily log anything on an auth failure. What would probably be a good first step is to turn up debugging and capture a log of the mount attempt. See the instructions here:

http://wiki.samba.org/index.php/LinuxCIFS_troubleshooting#Enabling_Debugging

--- Additional comment from erik on 2011-02-07 18:00:02 EST ---

Err.. well thanks for that wiki page, I'll try those troubleshooting hints.

However please note that there was no "Windows" involved. Nor was there any network traffic from the client to the server at all.

So I don't understand what you mean by those remarks.

--- Additional comment from erik on 2011-02-07 18:05:27 EST ---

Ah, enable debug messages by echoing 7 to /proc/fs/cifs/cifsFYI
Yes, I already tried that, it was also suggested in the #fedora channel. Unfortunately dmesg remains completely silent.

I do:

# echo 7 >  /proc/fs/cifs/cifsFYI
# cat /proc/fs/cifs/cifsFYI
7
# mount -t cifs -o credentials=/etc/mnt/server.cred //server/share
/mnt/server/user2
mount error(13): Permission denied
Refer to the mount.cifs(8) manual page (e.g. man mount.cifs)
# dmesg
(no new output)
# cat /proc/fs/cifs/cifsFYI
7
(debugging is still enabled)

/var/log/* don't pick up any activity either. Not a single line of logfile about the failed mount attempt.

--- Additional comment from jlayton on 2011-02-08 06:42:55 EST ---

Sorry, I misread your description earlier and thought the server was a windows machine...

If dmesg showed no info after turning up cifsFYI, then it sounds like something is being denied at a higher level before calling down into cifs.

--- Additional comment from jlayton on 2011-02-08 06:56:06 EST ---

I know that you said that /var/log/* doesn't have anything...

Does anything show up in /var/log/audit after a failed attempt?

--- Additional comment from jlayton on 2011-02-08 07:07:37 EST ---

Yes, if you're getting nothing from cFYI then the problem is almost certainly in a different layer. In 2.6.35, the first thing the VFS will do with cifs is call its get_sb routine. The first thing that cifs_get_sb does is this:

        cFYI(1, "Devname: %s flags: %d ", dev_name, flags);

...if that message isn't popping up then the mount is likely failing before the VFS ever touches cifs.ko.

--- Additional comment from jlayton on 2011-02-08 07:15:02 EST ---

Here's something you can try...

If you rmmod cifs.ko and then attempt the mount, does cifs.ko get loaded by the kernel?

--- Additional comment from erik on 2011-02-08 08:49:27 EST ---

Thanks for helping me further diagnose this problem.

/var/log/audit/audit.log only shows a succesful "su" action that I used to become root. The failed mount attempt does not show up.
Now I know that SELinux can deny access silently in certain situations, so I also tried temporarily disabling SELinux just to be sure. It didn't help.

I unloaded the cifs module and re-tried the mount. The cifs module did not get reloaded (it still failed with the same error message as before).
Do you have an idea what this tells us?

--- Additional comment from jlayton on 2011-02-08 09:01:13 EST ---

That means that the problem is (likely) not with cifs. All of the cifs code is contained within the cifs module. If it's not even getting as far as loading it then it sounds like something is failing before any symbols in the cifs code are called.

I'll have to look over the code when I get a chance. For now, I'll go ahead and move this to a kernel bug, and alter the title to better describe the problem.

--- Additional comment from erik on 2011-02-08 11:02:07 EST ---

Okay thanks, please let me know if you need me to provide any additional information.

--- Additional comment from jlayton on 2011-02-08 12:15:59 EST ---

Can you do this and paste in the output?

    # stat /mnt/server/user2

...note that you need execute permissions on a directory in order to be able to mount on it.

--- Additional comment from erik on 2011-02-08 13:01:38 EST ---

Wow.. something unexpected is going on here.

As I said earlier, I am issuing the mount command as root. The mount point is owned by the user that's supposed to use the share. Like this:

drwx------. 2 user2 users 4096 Feb  6 23:11 /mnt/server/user2

Now being root I assume I have no permission issues by definition. However, inspired by your last remark, I made the mount point world executable:

drwx-----x. 2 user2 users 4096 Feb  6 23:11 /mnt/server/user2

This way, the mount succeeds!

Please help me out here, am I being a complete noob for not understanding this? Is the root user actually supposed not to be able to issue mount on a directory owned by a regular user without making it world executable first?

For what it's worth:

# stat user2/
  File: `user2/'
  Size: 4096            Blocks: 8          IO Block: 4096   directory
Device: fd03h/64771d    Inode: 4194313     Links: 2
Access: (0701/drwx-----x)  Uid: ( 1000/    user2)   Gid: (  100/   users)
Access: 2011-02-06 23:21:44.333871250 +0100
Modify: 2011-02-06 23:11:00.368925860 +0100
Change: 2011-02-08 18:51:31.001305300 +0100

--- Additional comment from erik on 2011-02-08 13:47:11 EST ---

Additionally, please note that although explicit o+x permissions are apparently required on the mount point, it appears that this requirement does not apply for the parent directories of the mount point.

So without having explicit execute permissions on all the directories leading up to the mount point, I can positively mount, as long as the execute permissions on the mount point itself are explicit.

Is there a "|| uid == 0" check missing somewhere or is this intended behaviour?

--- Additional comment from jlayton on 2011-02-08 14:44:30 EST ---

No, I think I know what the problem is...

mount.cifs drops most capability flags before it performs a mount. One of those is CAP_DAC_READ_SEARCH, which is what bypasses the permissions check on directories. I think we just need to reinstate that flag before calling mount(2).

Setting back to cifs-utils since I understand the problem now, and that's where it is...