Fedora Account System
Red Hat Associate
Red Hat Customer
It looks like this is very much intentional per [1] but I think this is a breaking change that will affect many users/scripts. The fact that useradd name no longer assigns subuids by default means any user/script depending on that is now broken. In my case we add users for the use with podman and now podman will fail as the user no longer has subuids/subgids assigned to them. I don't see any upstream change which disables it by default either so this seems like it got patched only downstream. I cannot find any discussions around chaining this behavior, is there some good explanation why this must be changed? Changing this for rawhide is one thing but I see the same chnage is being pushed out to f42 as well which I think this will break people on a "stable" release. Reproducible: Always Steps to Reproduce: 1. useradd test Actual Results: no subuids assigned in /etc/subuid Expected Results: it should assign 65536 subuids by default. Additional Information: [1] https://src.fedoraproject.org/rpms/shadow-utils/c/e1cfa31731cd68aa2d76ebfb3142037d16d3bb1e?branch=rawhide
Hello, This is a deliberate security-hardening measure to address a significant vulnerability. We understand this is a breaking change and apologize for the disruption to your workflows, but we believe the security benefits are necessary. The core reason for this change is that the automatic assignment of subordinate ID ranges was found to have serious security implications. A local user's subordinate ID range can overlap with the real UID/GID of another user, particularly on centrally managed systems. This creates an attack vector where a local user could effectively become the other user by leveraging their subuid, potentially gaining unauthorized read/write access to that user's files. Given this risk, the consensus from security analysis and community discussion (see https://github.com/shadow-maint/shadow/issues/1157) was that subordinate IDs should not be assigned by default. The system should be secure out of the box, and enabling this functionality should be an explicit, opt-in decision made by a system administrator who understands the environment. As you noted, this is not an upstream default in shadow-utils. The upstream project intentionally leaves the policy decision of whether to assign subuids by default to the distributions, and we have decided to adopt a "secure by default" stance. While we try to avoid such breaking changes in stable releases, the potential security impact of this vulnerability (tracked in https://bugzilla.redhat.com/show_bug.cgi?id=2334168), which could allow for privilege escalation, was significant enough to warrant backporting the change. Protecting users from potential privilege escalation took precedence over maintaining the previous behavior.
While I understand the "secure by default" stance, I think we need to balance security with not breaking existing functionalities. Running as rootless is an important feature of Podman/Buildah, and with this change, we're breaking it by default. By default a system is not part of a centralized source, so users are not affected by any issue. I've looked at the upstream issue and the bugzilla you've linked but I don't see any user being affected, it is only discussed as a theoretical scenario. Users who rely on a centralized source must take the additional step of ensuring no extra subids are allocated to users. This already happens when using FreeIPA, as the local sources (/etc/subuid, /etc/subgid) are not used. How would systemd work in such a configuration anyway? systemd uses different ranges (https://systemd.io/UIDS-GIDS/) that can interfere with a centralized source, is it going to be blocked too?
> We understand this is a breaking change and apologize for the disruption to your workflows This isn't about my workflows. Every single rootless podman, buildah, skopeo user that relied on the auto assignment (which I think are the vast majority) will be affected. If new users no longer have subuids all of the sudden *without* any communication (other than the rpm changelog) about such change it will cause a lot of confusion and thus a very high support load on us. At the very least this mean we must update documentation to highlight such change. The CVE was filled end of last year so why patch it only now and then backport into stable releases to break the users there (it could have been patched before f42 was released). I think it is fair to discuss changing the default but I have problems with breaking people on existing releases by backporting this.
Any system administrator that is setting up NFS homedirs already have this problem, where multiple users will likely share the same UID. I would argue that this is far more likely the defaulf of using UID 1000 on all systems for users being added is going to cause an overlap of users. Have you done anything to address this issue. IE assigning random UIDs when creating users with useradd, to attempt to make it less likely to overlap? When setting up centralized NFS homedirs you need to guarantee no UID overlap. Whether this is /etc/passwd or /etc/subuid and /etc/subgid I don't believe it is smart to treat /etc/passwd, /etc/subuid and /etc/subgid differently from a security point of view. The overlap needs to be heavily considered by users setting up networked home directories, and in almost all cases is handled via networked UIDs via something like ldap or FreeIPA. Has this been discussed with the Fedora Board?
This change stems from a directive by our security team. After a recent analysis of the potential risks, they determined that the automatic assignment of subordinate IDs posed an unacceptable risk for a default configuration. The principle being applied is that a default installation should not contain known vectors for privilege escalation, even if they require a non-default environment (like centrally managed IDs) to be exploited. While many users are not on centralized systems, a distribution like ours must be secure for those who are, especially in enterprise or academic settings. This unfortunately means that another valid and important feature—rootless containers—now requires an explicit, one-time administrative step to function. It's true that administrators of systems with networked home directories must be diligent about UID management. The perspective of the security team, however, is that a tool's default behavior shouldn't create a hidden pitfall for them. A primary UID in /etc/passwd is highly visible; a subuid range is less so, and this change is about reducing that default attack surface. The point about the lack of communication is fair. The decision to patch was made based on the new security assessment, and standard policy is to patch all supported releases to ensure a consistent security posture. To restore the previous behavior, particularly for scripted user creation, administrators can edit /etc/login.defs and set SUB_UID_COUNT and SUB_GID_COUNT to a non-zero value, such as 65536.
where is the analysis report? What are these major risks (AFAICS the CVE itself is disputed) that can justify breaking a major use case like rootless containers? Mixing local and remote IDs is always a bad idea. As Dan already pointed out there is no difference between /etc/passwd and /etc/subuid. In other words, this change doesn't solve the root issue, the only effect is that it breaks rootless Podman/Buildah.
(In reply to Iker Pedrosa from comment #5) > This change stems from a directive by our security team. After a recent > analysis of the potential risks ...which, I assume, accidentally missed the fact that this breaks rootless containers (otherwise, I'm sure, Podman developers would have been informed right away). Would it make sense to re-evaluate the outcome of this analysis, given this point? Would it be possible to look for another way to warn users and administrators if the chance of overlap might actually arise (say, as a question during the installation process, or when some specific NFS-related packages are installed, etc.)?
This change should have gone through FESCO and had a CHANGE request to go through this discussion. Who is the Security team? I have asked Lukas Vrabec to look into this, he seemed to not know about it.
FYI this is breaking rootless podman/buildah on CentOS Stream 9/10 [1][2] as well as the patch was already backported there. It was caught up in CoreOS pipeline. [1] https://gitlab.com/redhat/centos-stream/rpms/shadow-utils/-/commit/725c221b69609ca358e9f508e94fe9ce1bb56c1d [2] https://gitlab.com/redhat/centos-stream/rpms/shadow-utils/-/commit/c6102cdd4646d5a89353d6261cc836c3947c9741
I don't think "our security team" here refers to Fedora, I think it's an RH security team. "To restore the previous behavior, particularly for scripted user creation, administrators can edit /etc/login.defs and set SUB_UID_COUNT and SUB_GID_COUNT to a non-zero value, such as 65536." This does not solve the most common case: I install a perfectly normal Fedora on a perfectly normal system (not an enterprise network environment), create a user during install, then try to run podman as that user. This has worked for years. Now I get a cryptic error: potentially insufficient UIDs or GIDs available in user namespace (requested 0:12 for /var/spool/mail): Check /etc/subuid and /etc/subgid if configured locally and run "podman system migrate": lchown /var/spool/mail: invalid argument Changing /etc/login.defs is not going to fix this, I now have to figure out that I have to run: usermod --add-subuids 100000-165535 (username) usermod --add-subgids 100000-165535 (username) to fix it. It's not great.
This breaks podman left and right (we just ran into it in https://github.com/cockpit-project/cockpit-podman/issues/2203). The justification here feels too hand-wavy and broad to me, and not actionable. Doing such a change in a released distro without even documenting what to do instead feels like a too big hammer. But more importantly, *IF* assigning subuids by default is somehow unsafe, then we the documentation/recommendation CANNOT be "turn it back on in the config" or "run usermod --add-subuids 100000-165535". That's just an "unbreak my setup" knob, and it would have exactly the same potential problems as just doing that by default. A possible middle ground might be: - don't break existing systems on upgrades, only change the config on new installations (would still break our tests, but that we can cope with) - keep automatic subuids for local-only NSS (i.e. the majority of systems) - print a warning in `useradd` when not creating subuids and pointing to some documentation which explains the security implications and how to avoid them properly
Moving Version as this change is introduced into Fedora 42 in https://bodhi.fedoraproject.org/updates/FEDORA-2025-2c653c3a82
It's a half-baked idea, but is it possible for *something* to notice if there's a overlap between subuids/subgids and "real" uids/gids, and complain loudly and/or block the affect subuid/subgid ranges? It kinda feels like it ought to be.
(In reply to Giuseppe Scrivano from comment #6) > Mixing local and remote IDs is always a bad idea. (In reply to Adam Williamson from comment #13) > It's a half-baked idea, but is it possible for *something* to notice if > there's a overlap between subuids/subgids and "real" uids/gids, and complain > loudly and/or block the affect subuid/subgid ranges? It kinda feels like it > ought to be. I wonder if the problem here can be restated as, machines that are part of an enterprise network and get their group and user IDs from a centralized source don't always have `subid: sss` in `/etc/nsswitch.conf`? Here's a more verbose version: https://github.com/shadow-maint/shadow/issues/1157#issuecomment-3115041755
(In reply to Martin Pitt from comment #12) > Moving Version as this change is introduced into Fedora 42 in > https://bodhi.fedoraproject.org/updates/FEDORA-2025-2c653c3a82 Note: F42 update was unpushed.
While CVE is being disputed, per agreement with the team, I've reverted the patch in Rawhide - https://src.fedoraproject.org/rpms/shadow-utils/c/b75f74d3f1b086fd5e699e55f131ae9e1beca525?branch=rawhide and rebuilt - https://bodhi.fedoraproject.org/updates/FEDORA-2025-3b1a650972
When you say "CVE is being disputed", what does that mean? Is that process public? If not, can we please make it public? There should be nothing secret here, the CVE is published and the shadow-utils change was made upstream. I'm not sure the issue is that the CVE needs to be disputed. I think the CVE is correct: there clearly *is* a security issue here. The upstream reported demonstrated that. What I'd say we need is a more considered approach to how to best address the CVE.
(In reply to Adam Williamson from comment #17) > When you say "CVE is being disputed", what does that mean? Formally I meant "Tags: disputed" in https://www.cve.org/CVERecord?id=CVE-2024-56433 But maybe I misunderstand this. > the CVE is published and the shadow-utils change was made upstream. Was it? https://github.com/shadow-maint/shadow/blob/master/etc/login.defs#L241 -- ranges are delegated upstream. Or what do you mean? > I'm not sure the issue is that the CVE needs to be disputed. I think the CVE > is correct: there clearly *is* a security issue here. The upstream reported > demonstrated that. What I'd say we need is a more considered approach to how > to best address the CVE. I personally agree with both statements: - there is a security issue - https://src.fedoraproject.org/rpms/shadow-utils/c/e1cfa31731cd68aa2d76ebfb3142037d16d3bb1e wasn't a proper fix
> Formally I meant "Tags: disputed" in https://www.cve.org/CVERecord?id=CVE-2024-56433 But maybe I misunderstand this. No, that sounds fine, I wasn't aware of that process. However, it looks like that process doesn't have any kind of endpoint, per https://www.cve.org/ResourcesSupport/FAQs . So I think we still need to make a clear determination about what the most appropriate resolution to this would be, I'm not sure just turning subuid generation back on exactly as before is the right fix? > Was it? https://github.com/shadow-maint/shadow/blob/master/etc/login.defs#L241 -- ranges are delegated upstream. Or what do you mean? Oh, sorry, you're right - I misremembered that the change had been merged upstream when in fact we did it downstream. But the larger point stands - we did it downstream in Fedora, which is public, so there's nothing secret there. I just wanted to make sure we're not debating this in some internal RH channel that nobody else can see.
Dropping Fedora common bugs nomination as this is not currently an active issue. Has there been any movement on a long term plan here?
There is related, but not directly applicable, work in progress upstream in systemd to address this issue: https://cfp.all-systems-go.io/all-systems-go-2025/talk/E7FHPY/ It would need integration work in podman.
This package has changed maintainer in Fedora. Reassigning to the new maintainer of this component.
This bug appears to have been reported against 'rawhide' during the Fedora Linux 45 development cycle. Changing version to 45.