Bug 1389138

Summary: automount can't take wildcard entries from sssd
Product: Red Hat Enterprise Linux 7 Reporter: Ian Allison <iana>
Component: sssdAssignee: SSSD Maintainers <sssd-maint>
Status: CLOSED UPSTREAM QA Contact: Steeve Goveas <sgoveas>
Severity: medium Docs Contact:
Priority: unspecified    
Version: 7.2CC: grajaiya, iana, jhrozek, lslebodn, mkosek, mzidek, orion, pbrezina
Target Milestone: rc   
Target Release: ---   
Hardware: x86_64   
OS: Linux   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2018-04-18 14:54:10 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Attachments:
Description Flags
sssd_autofs log file
none
sssd_default.log file with debug_level = 10 none

Description Ian Allison 2016-10-27 01:15:02 UTC
Description of problem:

Wildcard autofs maps don't seem to work when the map is stored in LDAP and accessed by sssd. Using a map containing a wildcard works of autofs contacts LDAP directly, but not via sssd. Ordinary "fully specified" (non-wildcard) keys work fine via SSSD+LDAP.

Version-Release number of selected component (if applicable):

sssd-krb5-common-1.13.0-40.el7_2.12.x86_64
sssd-ldap-1.13.0-40.el7_2.12.x86_64
sssd-common-1.13.0-40.el7_2.12.x86_64
sssd-common-pac-1.13.0-40.el7_2.12.x86_64
sssd-krb5-1.13.0-40.el7_2.12.x86_64
sssd-proxy-1.13.0-40.el7_2.12.x86_64
sssd-ipa-1.13.0-40.el7_2.12.x86_64
python-sssdconfig-1.13.0-40.el7_2.12.noarch
sssd-client-1.13.0-40.el7_2.12.x86_64
sssd-ad-1.13.0-40.el7_2.12.x86_64
sssd-1.13.0-40.el7_2.12.x86_64
autofs-5.0.7-54.el7.x86_64


How reproducible:

Only tested on one client so far.

Steps to Reproduce:
1. Add an automount entry to /etc/auto.master with automount configured to access LDAP via sssd
```
$ cat /etc/auto.master
/nfs	auto.nfs

$ grep automount /etc/nsswitch.conf
automount:  files sss
```

2. Configure a wildcard entry in LDAP
```
dn: cn=/,nisMapName=auto.nfs,dc=example.com
changetype: add
objectClass: nisObject
cn: * 
nisMapEntry: -rw,hard,intr,rsize=8192,wsize=8192,nosuid,tcp   XXX.XXX.XXX.XXX:/t
 ank/HomeNet/Users/&
nisMapName: auto.nfs
```

3. Configure sssd to consult ldap for autofs
```
ldap_autofs_search_base=dc=example,dc=com
ldap_autofs_map_object_class=nisMap
ldap_autofs_map_name=nisMapName
ldap_autofs_entry_object_class=nisObject
ldap_autofs_entry_key=cn
ldap_autofs_entry_value=nisMapEntry
```

4. Clear sss_cache, restart sssd and automount


Actual results:

Mountpoints aren't mounted
```
ls /nfs/visitor
ls: cannot access /nfs/visitor: No such file or directory
```

Expected results:
Mountpoint get's automounted

Additional info:
Configuring automount to consult the same LDAP entry directly (avoiding sssd) works
```
$ cat /etc/sysconfig/autofs
LDAP_URI=ldap://r7-ldap1.example.com
SEARCH_BASE="dc=example,dc=com"
MAP_OBJECT_CLASS=nisMap
ENTRY_OBJECT_CLASS=nisObject
MAP_ATTRIBUTE=nisMapName
ENTRY_ATTRIBUTE=cn
VALUE_ATTRIBUTE="nisMapEntry"

$ grep automount /etc/nsswitch.conf
automount:  files ldap

$ systemctl restart autofs
$ ls /nfs/user1
```

Also, specifying the key explicitly works, e.g. an LDAP entry for
```
dn: cn=user1,nisMapName=auto.nfs,dc=example,dc=com
objectClass: nisObject
cn: user1
nisMapEntry: -rw,hard,intr,rsize=8192,wsize=8192,nosuid,tcp   XXX.XXX.XXX.XXX:/t
 ank/HomeNet/Users/user1
nisMapName: auto.nfs
```

Comment 2 Jakub Hrozek 2016-10-27 07:55:12 UTC
If automounter works with LDAP but not with SSSD, then I would agree we have a bug. Could you please increase debug_level in the [autofs] and [domain] sections of sssd.conf to 10, re-run the test and attach the log files from /var/log/sssd/*.log ?

Comment 3 Ian Allison 2016-10-27 17:08:27 UTC
I Jakub, I've set debug_level = 10 in both of those sections and run the tests. I'm attaching the sss_default.log and sss_autofs.log files (the others are empty).

When testing, I tried accessing the following directories
```
$ ls /nfs/iana
ls: cannot access /nfs/iana: No such file or directory

$ ls /nfs/37zfs1-iana
file
```

I'm expecting the first of these entries to match the following wildcard from LDAP
```
dn: cn=/,nisMapName=auto.nfs,dc=stat,dc=ubc,dc=ca
objectClass: nisObject
cn: *
cn: /
nisMapEntry: -rw,hard,intr,rsize=8192,wsize=8192,nosuid,tcp   142.103.37.18:/t
 ank/StatNet/Users/&
nisMapName: auto.nfs
```
This is slightly different from the tests in my initial report (it contains an extra cn attribute for /) but I've confirmed that the behaviour is still the same with automount talking directly to LDAP.


For the second (successful) test, I'm matching
```
dn: cn=37zfs1-iana,nisMapName=auto.nfs,dc=stat,dc=ubc,dc=ca
objectClass: nisObject
cn: 37zfs1-iana
nisMapEntry: -rw,hard,intr,rsize=8192,wsize=8192,nosuid,tcp   142.103.37.18:/t
 ank/StatNet/Users/iana
nisMapName: auto.nfs
```

In sss_autofs.log I see the following entries
[getautomntbyname_process] (0x0080): No key named [/] found
[getautomntbyname_process] (0x0080): No key named [*] found

Comment 4 Ian Allison 2016-10-27 17:11:27 UTC
Created attachment 1214677 [details]
sssd_autofs log file

Comment 5 Ian Allison 2016-10-27 17:12:01 UTC
Created attachment 1214678 [details]
sssd_default.log file with debug_level = 10

Comment 6 Jakub Hrozek 2016-10-31 10:05:45 UTC
Indeed, the root and asterisk maps are not found. Could you also take a look at how we store those in the cache? I know that in general we escape characters that have a special meaning in LDAP, like asterisk, but on searching these in the responder, we should unescape them back.

Can you install the ldb-tools package and run:
ldbsearch -H /var/lib/sss/db/cache_* 
and check the automount objects?

Comment 7 Ian Allison 2016-10-31 17:31:40 UTC
I can't see any entry for the root or asterisk maps in that database file. I can see the 37zfs1-iana map, but I don't think / is getting added for some reason.

```
$ ldbsearch -H /var/lib/sss/db/cache_default.ldb | grep automountKey
automountKey: 37zfs1-iana
```

I checked the number of DNs with auto.nfs against the results of ldapsearch just to be sure and there is indeed one missing.
```
$ ldbsearch -H /var/lib/sss/db/cache_default.ldb | grep -E '^dn:' | grep auto.nfs

asq: Unable to register control with rootdse!
dn: name=37zfs1-iana-rw\,hard\,intr\,rsize\3D8192\,wsize\3D8192\,nosuid\,tcp   XXX.XXX.XXX.XXX:/tank/StatNet/Users/iana,name=auto.nfs,cn=autofsmaps,cn=custom,cn=default,cn=sysdb
dn: name=auto.nfs,cn=autofsmaps,cn=custom,cn=default,cn=sysdb

$ ldapsearch -x -Z -b 'nisMapName=auto.nfs,dc=example,dc=com' -LLL 'dn' 

dn: nisMapName=auto.nfs,dc=example,dc=com
dn: cn=37zfs1-iana,nisMapName=auto.nfs,dc=example,dc=com
dn: cn=/,nisMapName=auto.nfs,dc=example,dc=com

```

Comment 8 Ian Allison 2016-11-04 01:30:32 UTC
I got a little further with this. Historically, my LDAP information came from NIS via the PADL migration tools, so starting from an entry like this
```
* -rw XXX.XXX.XXX.XXX:/tank/StatNet/Users/&
```
migrate_automount.pl, gives ldif something like
```
dn: nisMapName=autofs.dump,dc=example,dc=com
objectClass: top
objectClass: nisMap
nisMapName: autofs.dump

dn: cn=/,nisMapName=autofs.dump,dc=example,dc=com
objectClass: nisObject
cn: *
nisMapEntry: -rw XXX.XXX.XXX.XXX:/tank/StatNet/Users/&
nisMapName: autofs.dump
```

Adding that to OpenLDAP is no problem, but the cn attribute picks up two values: * and /
```
dn: cn=/,nisMapName=auto.nfs,dc=example,dc=com
objectClass: nisObject
cn:: KiA=
cn: /
nisMapEntry: -rw XXX.XXX.XXX.XXX:/tank/StatNet/Users/&
nisMapName: auto.nfs
```
The first entry is * base64 encoded. cn isn't SINGLE-VALUE (as far as I know) so there is no schema violation.

Next, get_autofs_entry_attr notices the two cn values but only takes the first (*)
```
if (el->num_values != 1) {
        DEBUG(SSSDBG_CRIT_FAILURE,
              "Expected one entry got %d\n", el->num_values);
        return NULL;
    }

    return (const char *)el->values[0].data;
```
The "Expected one entry got 2" is in my sssd_default.log. Finally, sdap_autofs_setautomntent_save, notices that the key and value are different (* vs /) and throws away the map entry with "Malformed entry, skipping" in sssd_default.log.

I haven't looked at the automounter code yet, so I'm not sure how it handles the two cn attributes, but this explains why it doesn't see the map in sssd.

Comment 9 Ian Allison 2016-11-04 04:28:24 UTC
OK, I took a look at the automount code. 

https://git.kernel.org/cgit/linux/storage/autofs/autofs.git/tree/modules/lookup_ldap.c#n2486

"""
/*
 * Keys should be unique so, in general, there shouldn't be
 * more than one attribute value. We make an exception for
 * wildcard entries as people may have values for '*' or
 * '/' for compaibility reasons. We use the '/' as the
 * wildcard in LDAP but allow '*' as well to allow for
 * people using older schemas that allow '*' as a key
 * value. Another case where there can be multiple key
 * values is when people have used the "%" hack to specify
 * case matching ctriteria in a case insensitive attribute.
 */


i.e. They handle it as a special case, checking for the existing of / or * (or both) and then using / internally. Would the same thing be appropriate in sssd?

Comment 10 Orion Poplawski 2017-04-13 16:41:35 UTC
This appears to be working fine for me with autofs maps in IPA and a wildcard entry.

sssd-1.14.0-43.el7_3.11.x86_64

Comment 11 Ian Allison 2017-04-13 17:16:59 UTC
Orion, I think in my case the problem was that my LDAP tree had entries for both * and / which seems to be allowed in LDAP, but not by sssd. 

It looks like get_autofs_entry_attr still contains the following block (https://github.com/SSSD/sssd/blob/4f2509f8d23d9e921f07b2ead63392ae82ad3a38/src/providers/ldap/sdap_async_autofs.c#L64-L68)
```
    if (el->num_values != 1) {
        DEBUG(SSSDBG_CRIT_FAILURE,
              "Expected one entry got %d\n", el->num_values);
        return NULL;
    }
```
so will find both entries in LDAP, and return neither of them. 

In my case having both entries was an artifact of the migration_tools, so I just adjusted the migration tools to make sure that there was only a single entry per wildcard in LDAP.

I called it a bug because the behavior here is inconsistent with thhat when autofs is configured to talk to LDAP directly. In that case, they handle having '*' and '/' present for a key as a special case (https://git.kernel.org/pub/scm/linux/storage/autofs/autofs.git/tree/modules/lookup_ldap.c#n2486).

Comment 13 Jakub Hrozek 2017-08-14 13:18:42 UTC
Upstream ticket:
https://pagure.io/SSSD/sssd/issue/3476

Comment 14 Jakub Hrozek 2018-04-18 14:54:10 UTC
Let's track this with the upstream ticket only instead of rolling over the BZ from one release to another, to another, ...