RHEL Engineering is moving the tracking of its product development work on RHEL 6 through RHEL 9 to Red Hat Jira (issues.redhat.com). If you're a Red Hat customer, please continue to file support cases via the Red Hat customer portal. If you're not, please head to the "RHEL project" in Red Hat Jira and file new tickets here. Individual Bugzilla bugs in the statuses "NEW", "ASSIGNED", and "POST" are being migrated throughout September 2023. Bugs of Red Hat partners with an assigned Engineering Partner Manager (EPM) are migrated in late September as per pre-agreed dates. Bugs against components "kernel", "kernel-rt", and "kpatch" are only migrated if still in "NEW" or "ASSIGNED". If you cannot log in to RH Jira, please consult article #7032570. That failing, please send an e-mail to the RH Jira admins at rh-issues@redhat.com to troubleshoot your issue as a user management inquiry. The email creates a ServiceNow ticket with Red Hat. Individual Bugzilla bugs that are migrated will be moved to status "CLOSED", resolution "MIGRATED", and set with "MigratedToJIRA" in "Keywords". The link to the successor Jira issue will be found under "Links", have a little "two-footprint" icon next to it, and direct you to the "RHEL project" in Red Hat Jira (issue links are of type "https://issues.redhat.com/browse/RHEL-XXXX", where "X" is a digit). This same link will be available in a blue banner at the top of the page informing you that that bug has been migrated.
Bug 1559185 - Files created by root user is shown as nfsnobody as owner and group when NFS is mounted with sec=krb5
Summary: Files created by root user is shown as nfsnobody as owner and group when NFS ...
Keywords:
Status: CLOSED CURRENTRELEASE
Alias: None
Product: Red Hat Enterprise Linux 7
Classification: Red Hat
Component: doc-Storage_Administration_Guide
Version: 7.4
Hardware: x86_64
OS: Linux
high
medium
Target Milestone: rc
: ---
Assignee: Marek Suchánek
QA Contact: Yongcheng Yang
Marek Suchánek
URL:
Whiteboard:
Depends On:
Blocks: 1477664 1546181
TreeView+ depends on / blocked
 
Reported: 2018-03-21 23:34 UTC by Glen Babiano
Modified: 2021-12-10 15:50 UTC (History)
16 users (show)

Fixed In Version:
Doc Type: No Doc Update
Doc Text:
undefined
Clone Of:
Environment:
RHEL 7.4, all erratas applied including 2nd January 2018
Last Closed: 2019-04-08 15:21:00 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)

Description Glen Babiano 2018-03-21 23:34:54 UTC
Description of problem:
Files created by root user is shown as nfsnobody as owner and group when mounted with sec=krb5. 

Note that a root user principal is created in the kerberos database thus is able to get a tgt ticket as root.

Version-Release number of selected component (if applicable):
RHEL 7

How reproducible:
1. Server host
Install and configure krb5-server
Create a kerberos user principal root
Configure an NFS share with sec=krb5 
Export the newly created NFS share

2. Client host
Install krb5-workstation pam_krb5 and copy krb5.conf from server
Enable kerberos authentication with authconfig --update --enablekrb5
Mount the share with sec=krb5
Switch user to root
kinit root (acquiring tgt is successful) 
Touch a file on the mounted nfs
Check ownership of the new file

Actual results:
The file ownership and group shows nfsnobody:nfsnobody

Expected results:
The file should show root:root as owner and group

Additional info:

Comment 2 Glen Babiano 2018-03-26 06:10:47 UTC
Hi,

Do we have any update/requirement on this issue?

Thanks.

Comment 3 Dave Wysochanski 2018-03-27 14:22:42 UTC
steve please assess, and note that in the customer case the customer was exporting with 'no_root_squash' which is left out of the reproducer details.  Also what is left out was that 'sec=sys' works properly.  I'm not sure if there are other details left out of the reproducer but please try it and work with the filer of this bug to validate the reproducer and clarify if this is a bug or just a misconfiguration.

Comment 4 Glen Babiano 2018-03-28 22:59:39 UTC
Hi Dave and Steve,

Thanks for the update.

We have been trying different configurations on the reproducer so you may find some missing bits but please feel free to update or reach out to me if you have any questions.

Glen

Comment 5 Glen Babiano 2018-04-04 06:12:13 UTC
Hi Team,

Do we have any update on this? We need to provide feedback to the customer as it's been a while. Thanks.

Regards,

Glen

Comment 7 Glen Babiano 2018-04-10 22:47:34 UTC
Hi Team,

Have we had any progress on this?

Just following up.

Regards,

Glen

Comment 8 Glen Babiano 2018-04-16 01:46:00 UTC
Hi Team,

Do we have progress on this request?

Thanks.

Glen

Comment 9 Vincent L 2018-04-19 03:41:11 UTC
Hi Team,

There is the same problem on the CentOS.

Thanks.

Vincent

Comment 10 Vincent L 2018-04-19 11:12:00 UTC
When do kadmin on the client, the kadmind.log log this:

Apr 19 06:01:29 nfs-server.example.com kadmind[1133](Notice): Request: kadm5_init, root/admin, success, client=root/admin ......

It is "root/admin ",not "root".

Comment 11 Vincent L 2018-04-20 02:12:19 UTC
https://docs.oracle.com/cd/E23824_01/html/821-1456/setup-148.html#fgohx

This part of the documet introduce "How to Access a Kerberos Protected NFS File System as the root User". When i do as it, I get this:
[root@client01 tmp]# mount -t nfs4 -o sec=krb5,root=root/client01.example.com nfs-server.example.com:/nfs /tmp/client
mount.nfs4: an incorrect mount option was specified

Comment 12 Vincent L 2018-04-20 07:25:58 UTC
When googling it, I find someone who also suffered this problem.
https://www.spinics.net/lists/linux-nfs/msg43695.html

Comment 14 Vincent L 2018-04-27 03:37:28 UTC
(In reply to Glen Babiano from comment #8)
> Hi Team,
> 
> Do we have progress on this request?
> 
> Thanks.
> 
> Glen

Hi Glen,

Have U got some progress?

Comment 16 Steve Dickson 2018-05-07 15:04:04 UTC
Make sure bot the NFS server and Client are using the correct
domain name. 

Try to get some network traces between the NFS server, client and KDC
and set RPCIDMAPDARGS=-vvv in /etc/sysconfig/nfs to see if there
are any mapping errors.

Comment 17 amitkuma 2018-05-21 10:02:18 UTC
Hello Steve,

Customer provided 
a. rpc.idmapd debug logs for client,server.
b. packet captures of nfs client,server.

Files can be found at:
https://foobar.gsslab.pnq.redhat.com/02038494/

Thanks
Amit

Comment 18 Steve Dickson 2018-05-21 12:54:41 UTC
(In reply to amitkuma from comment #17)
> Hello Steve,
> 
> Customer provided 
> a. rpc.idmapd debug logs for client,server.
> b. packet captures of nfs client,server.
> 
> Files can be found at:
> https://foobar.gsslab.pnq.redhat.com/02038494/

Case Not Found

The case you are looking for must have been closed or deleted. Foobar do not keep more than 6 weeks old data if you are trying to pull attachment for old cases.

Comment 19 amitkuma 2018-05-21 12:57:59 UTC
ahh my bad. Please look at this link.
https://foobar.gsslab.pnq.redhat.com/02007264

Comment 20 Steve Whitehouse 2018-06-25 09:22:32 UTC
Steve D, please provide an update for this bug. It is on the 7.6 RPL so we need to make a decision as to whether to keep it on the list or move to 7.7

Justin, can you take a look too?

Comment 21 Alice Mitchell 2018-06-25 17:03:30 UTC
I believe the subject is covered in this email thread:
https://www.redhat.com/archives/freeipa-users/2014-June/msg00107.html

The root user has by default no mapping into the kerberos realm resulting in a permission denied behind the scenes and the results coming back as if it was the nobody user.

I was able to reproduce this and then fix it by enabling static mapping in the servers idmapd config and then mapping the clients host key to the local root user.

idmapd.conf:

[Translation]
Method = nsswitch static

[Static]
host/nfsclient.mydomain.com = root

Comment 23 Steve Whitehouse 2018-07-13 09:04:10 UTC
Justin, does that mean that we can fix this with just a config change? If so then we should just document how to do it and we can probably resolve the bug that way.

Comment 25 Steve Whitehouse 2018-07-19 10:39:09 UTC
That to me sounds like we should update our documentation to explicitly say that this will not work. I've added the Documentation keyword to this, but it probably needs to be reassigned to the relevant bit of the documentation.

I've set the doc type to the nearest thing to this issue. Justin, can you help draft some suitable text for the doc text field above? We should explain why it doesn't work, what the issues are and what we recommend as alternatives.

Comment 26 Marek Suchánek 2018-08-06 13:56:32 UTC
Hi, I'm the technical writer responsible for NFS documentation.

I'm not sure what documentation update is needed here exactly. Do we want to:

* describe this as a known issue or deprecated functionality in the 7.6 release notes,
* add a note in the NFS chapter of the Storage Administration Guide,
* or both?

I'm assigning the bug to myself in the meantime.

Comment 27 Alice Mitchell 2018-08-08 10:24:24 UTC
There are several aspects to this, and I am still working to untangle them, but it basically goes like this:

Enabling a root privileged user on kerberised nfs is very strongly advised against, it is not supposed to work, and the workarounds required create a lot of potential security holes.

If you setup the user identities correctly and put in the static overrides then you can get a user with root like permissions.

but, file creation by that user always comes out as nfsnobody, this looks to be enforced by the kernel somewhere as the identity never gets as far as being mapped to root.

If and when this ever worked I cannot say as yet nobody has provided a working example, and I am still in the process of trying to determine the exact combination of settings required, and thus to understand the root causes so they can be accurately documented.

Comment 28 Marek Suchánek 2018-09-07 17:51:42 UTC
Hello Justin,

Has there been any progress on figuring out this bug? I'm still ready to document this in the 7.6 release notes (or elsewhere) if needed.

Thanks,
Marek

Comment 29 Alice Mitchell 2018-09-10 15:19:43 UTC
Sorry for the delay, I have as yet been unable to make this feature work as suggested on 7.6, 7.4 or even 7.2. So still trying to figure out if and when it ever worked, and how to reliably enable it.

Comment 30 Marek Suchánek 2018-09-10 15:56:26 UTC
(In reply to Justin Mitchell from comment #29)
> Sorry for the delay, I have as yet been unable to make this feature work as
> suggested on 7.6, 7.4 or even 7.2. So still trying to figure out if and when
> it ever worked, and how to reliably enable it.

No problem. Let me know when you find out more then. I'm setting the "requires_doc_text-" flag in the meantime to signify we don't want to document it yet.

Comment 34 Alice Mitchell 2018-12-13 11:43:42 UTC
Thanks to Scott and Simo for providing the answer to this, I have replicated this on RHEL7.x and can confirm that the documentation should be updated now to reflect this.  If no_root_squash functionality is required, idmap is no longer relevant, the correct procedure is to add the following to /etc/krb5.conf on the NFS server host.

[realms]
  ...
  EXAMPLE.COM = {
    ...
    auth_to_local = RULE:[2:$1/$2@$0](host/nfsclient.example.com)s/.*/root/
    auth_to_local = DEFAULT
  }

The export requires sec=krb5 and no_root_squash and it enables root access for only the one host named here, e.g. nfsclient.example.com
If required, additional 'auth_to_lkocal = RULE:...' lines may be added, but the 'auth_to_local = DEFAULT' must be last.

Let me know if any further examples or info are required

Comment 35 Marek Suchánek 2019-03-18 17:14:32 UTC
Hi Justin, sorry to keep you waiting.

(In reply to Justin Mitchell from comment #34)
> Thanks to Scott and Simo for providing the answer to this, I have replicated
> this on RHEL7.x and can confirm that the documentation should be updated now
> to reflect this.  If no_root_squash functionality is required, idmap is no
> longer relevant, the correct procedure is to add the following to
> /etc/krb5.conf on the NFS server host.
> 
> [realms]
>   ...
>   EXAMPLE.COM = {
>     ...
>     auth_to_local =
> RULE:[2:$1/$2@$0](host/nfsclient.example.com)s/.*/root/
>     auth_to_local = DEFAULT
>   }
> 
> The export requires sec=krb5 and no_root_squash and it enables root access
> for only the one host named here, e.g. nfsclient.example.com
> If required, additional 'auth_to_lkocal = RULE:...' lines may be added, but
> the 'auth_to_local = DEFAULT' must be last.
> 
> Let me know if any further examples or info are required

Thanks for that information, it seems much clearer now.

Looking at our current NFS documentation, I'd say that this new bit should go into the "Securing NFS" section, which documents securing with Kerberos, among other things. Link:
https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/7/html/storage_administration_guide/s1-nfs-security

However, that section doesn't document the "no_root_squash" functionality yet, so context for it has to be added. Right now, I'm deciding between two ways to solve this:

* Add a new step to the "Configuring Kerberos" procedure, saying something like "if you want to write files as root on the NFS share and retain root ownership on these files, use the 'no_root_squash' option and add <code block> in the /etc/krb5.conf file on the server."

* If the step with all the necessary context turns out to be too long, I can create a separate procedure for it with its own heading, called e.g. "Reading and writing files as root on secured NFS". That would give me more space to explain all that's necessary.

Just to be sure: Am I going the right way with this? Can I assume that the config options solve strictly the use case when you want to write files to an NFS share as root and keep root permissions/ownership, or is there more to it?

If you think that my proposed solutions work, I'll prepare a documentation preview.

Comment 36 Alice Mitchell 2019-03-18 18:16:32 UTC
If my understanding of this is correct, someone please step in if i am mistaken, then this procedure is not recommended as a general practice and should be used only where this specific behaviour is absolutely required as it potentially weakens security by effectively mapping an unauthenticated local root user into the superuser of the NFS server. This risk is partly mitigated by fully qualifying which machine this applies to, as per the example, but should be avoided if at all possible.

I think you are right that you may need to break the documentation for this out into a new sub-section/procedure as it is likely to be too much info to included inline.

Comment 43 Marek Suchánek 2019-04-08 15:21:00 UTC
A KBase article documenting the use case has been published:
https://access.redhat.com/articles/4040141

I've added the link to the article in our RHEL 7 and RHEL 8 documentation on NFS. The RHEL 8 documentation update will be published with RHEL 8.0 GA, while the RHEL 7 version can be found here:
https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/7/html/storage_administration_guide/s1-nfs-security?lb_target=stage#s3-nfs-security-hosts-nfsv4

I believe I can close this bug now. If anybody believes that there's more to be done, feel free to reopen.


Note You need to log in before you can comment on or make changes to this bug.