Description of problem: idmapper is not properly mapping uids when nfsv4(nfs-ganesh) is used in RHGS 3.2. The "Domain" mentioned in idmap.conf is same on both the client and the server. Version-Release number of selected component (if applicable): RHGS 3.2 libnfsidmap-0.25-15.el7.x86_64 How reproducible: Reproducible at customer environment Steps to Reproduce: 1. Setup a nfs-ganesha cluster 2. Kerberize nfs-ganesha 3. Mount the share on client and create files using a domain user Actual results: The uids getting squashed to an unknown uid Expected results: uids should be mapped correctly. Additional info: - We have collected NFS ganesha debug logs, configuration files and the tcpdump. I will attach these to bz. - A snip from the logs ~~~~ 29/06/2017 11:30:04 : epoch 54100000 : server.example.com : ganesha.nfsd-86527[work-46] nfs_req_creds :ID MAPPER :WARN :Could not map principal user1 to uid 29/06/2017 11:30:04 : epoch 54100000 : server.example.com : ganesha.nfsd-86527[work-46] xdr_encode_nfs4_princ :ID MAPPER :INFO :nfs4_uid_to_name failed with code -2. 29/06/2017 11:30:04 : epoch 54100000 : server.example.com : ganesha.nfsd-86527[work-46] xdr_encode_nfs4_princ :ID MAPPER :INFO :Lookup for -2 failed, using numeric owner 29/06/2017 11:30:04 : epoch 54100000 : server.example.com : ganesha.nfsd-86527[work-46] xdr_encode_nfs4_princ :ID MAPPER :INFO :nfs4_gid_to_name failed with code -2 ~~~~ - getent passwd user is properly giving uid from the client and the server. - "rpc.idmapd -f -vv" gives the same domain name on both the nodes.
What are the respective idmapd.conf files? These issues are almost always something not right in idmapd.conf. Is idmapd running on both client and server? Any error logs from idmapd? If absolutely necessary we can turn on Ganesha logging for idmapping, and also collect tcpdump of some simple operations, but most likely we can solve the problem by having the idmapd.conf files.
Hmm, I can't tell what credentials are in the kerberos ticket. That would be another thing to check since the ownership of a created file depends on the credentials the request comes in with (so that's actually before idmapping).
use_getpwnam is false, so principal2uid() will always fail. Try setting UseGetpwnam to true?
Hmmm... I think that check should only be used if NFSIDMAP isn't in use, but it appears to be in use all the time.
Did the customer try all settings being the same case?
need reproducer, unable to reproduce locally.
@Kaleb, I am trying to reproduce this issue , but facing problems in creating this environment only. Do we need any further info/logs from customer? Since it is getting very delayed, can we go on remote session with customer for live debugging if customer agrees , sometimes early next week?
I can join a call if necessary. Not sure I can add anything atm though. Mike Flannery and John Call have been exchanging emails with Frank Filz about idmapping. You need to install and run idmapd on all the systems and use the same config on the server and all the clients. Maybe Mike can shed some light on the subject after his recent experiments?
Hi Kaleb, not sure if this will help in this case, but it helped in my recent customer engagement... My main problem was identifying what names needed to be mapped, and what format the names were coming in on... I used journalctl to observe usernames that require mapping via # journalctl -l | grep "idmap" OR # nfsidmap -l Then I configured /etc/idmapd.conf similarly on both client and server Use # nfsidmap -c to clear the client mapping cache after making changes to /etc/idmapd.conf, or reboot. Use # systemctl restart nfs-ganesha to update mappings on the server Static mappings and ACLs can work around unknown UIDs -- See appendix Appendix: [root@client ~]# hostname client.not-cluster.net [root@client ~]# id mflanner uid=1001(mflanner) gid=1001(mflanner) groups=1001(mflanner),190(systemd-journal) [root@client ~]# egrep -v '^$|^#' /etc/idmapd.conf [General] Domain = cluster.net Local-Realms = msft.cluster.net [Mapping] Nobody-User = myuser Nobody-Group = mygroup [Translation] Method = nsswitch,static [Static] MSFT\gclark = mflanner
No, I don't have any update. I was not able to reproduce it, and neither were Jiffin or Frank. Reading through all the comments Frank seems fairly certain it must be some misconfiguration of K5 or uidmapper. Frank, did we get everything we need to be able to diagnose this?