Bug 1255791 - RPM replaces file marked as configuration during new system installation via Anaconda
RPM replaces file marked as configuration during new system installation via ...
Status: ASSIGNED
Product: Red Hat Enterprise Linux 7
Classification: Red Hat
Component: rpm (Show other bugs)
7.1
Unspecified Unspecified
unspecified Severity high
: rc
: ---
Assigned To: packaging-team-maint
BaseOS QE Security Team
: Documentation
Depends On:
Blocks:
  Show dependency treegraph
 
Reported: 2015-08-21 10:46 EDT by Habig, Alec
Modified: 2018-01-31 00:13 EST (History)
9 users (show)

See Also:
Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed:
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---
lkimlick: needinfo-
lkimlick: needinfo-


Attachments (Terms of Use)

  None (edit)
Description Habig, Alec 2015-08-21 10:46:44 EDT
First caveat: working with Scientific Linux, after discussions on their mailing list it was recommended I report this upstream to you all, since the issue affects RHEL7 too.  And Fedora, although people probably aren't running larger clusters of Fedora boxes.

Second caveat: this seems to mostly be a documentation bug, in that the only official word I've been able to find on how to work around it is wrong.  Hopefully future people will at least find this bug to learn more.  There has to be an easier way, though, even if documentation is updated, so it's a bit of a feature request too.

On to the problem:

Added some new student workstations, running 7.1, into an ldap managed cluster consisting of 6.x machines.  7 wants system accounts numbered under 1000, 6 was happy with under 500.  Many users and countless files over a number of machines have uids between 500 and 1000: a global migration to the new scheme would be A Lot Of Work (yes, that's the right way long term: no, adding a few new client machines isn't a good time to trigger a server migration).  This fedora features proposal page:

  https://fedoraproject.org/wiki/Features/1000SystemAccounts

suggests dropping in a tweaked /etc/login.defs file in kickstart's %pre section for people in my situation.

Unfortunately, the filesystem doesn't exist yet in %pre, so that's too early to pull in a tweaked file.  In %post, all the system accounts are already made and many config files have pulled the UID min and max values from the default login.defs file already, so that's too late.

I worked around this by making my own install repo with a tweaked and rebuilt shadow-utils package containing the old uid numbering scheme.  Adding in a simpler rpm that drops in login.defs only early in the install process (triggering on filesystem probably) would have been a cleaner solution.  The good news is that starting with that login.defs file, all the other parts of the system now seem to actually be looking there for their uid ranges (good work, shadow-utils people!).

Another way of doing it would be to not have any partitioning and formatting stuff in the kickstart script, write that all up manually in %pre, and drop in /etc/login.defs at the end of %pre.  This, however, avoids using all the goodness you've built into kickstart at the expense of making fragile handmade scripts.

Features which would make this easier would have to be a joint kickstart/shadow-utils operation: either a kickstart option to set uid range, or a new "after filesystem creation, before rpms" scripting stage.

Kind of an odd bug report, but if I'd found this last week it would have saved me a lot of time.  One of those questions out there where there are discussion threads about the problem, but no proposed workarounds other than "just change all your existing uids already".
Comment 9 Martin Kolman 2017-06-29 06:41:26 EDT
(In reply to Lenka Kimlickova from comment #8)
> Hi Martin,
> 
> if possible, could you please provide information on how to manage IDs in
> kickstart and how to migrate at RHEL 7 install?
> 
> Or could you please refer to someone who can provide the information?
> 
> Thanks,
> 
> Lenka

Looking at the original bug description:

"Features which would make this easier would have to be a joint kickstart/shadow-utils operation: either a kickstart option to set uid range, or a new "after filesystem creation, before rpms" scripting stage."

The %pre-install scripts run after filesystem creation but before RPMs are installed:

http://pykickstart.readthedocs.io/en/latest/kickstart-docs.html#chapter-7-pre-installation-script

It's likely that %pre-install has not yet been implemented back when the issue was first reported.

All in all I think it should be possible to use %pre-install script to handle the issue described by the reporter & it seems to me like a better solution than to complicate kickstart syntax & handling by adding knowledge about UID/GID range assignment.
Comment 10 Habig, Alec 2017-06-29 11:24:25 EDT
a %pre-install sounds perfect.  As luck would have it, if the UPS tracking is to be believed I've got a new box showing up tomorrow I can try it out on, actually.  Stay tuned.
Comment 11 Martin Kolman 2017-06-29 12:16:16 EDT
(In reply to Habig, Alec from comment #10)
> a %pre-install sounds perfect.  As luck would have it, if the UPS tracking
> is to be believed I've got a new box showing up tomorrow I can try it out
> on, actually.  Stay tuned.
Nice timing! :)
Comment 12 Habig, Alec 2017-06-30 17:38:11 EDT
This worked great.  In my ks script, I said:

%pre-install
mkdir /etc
wget http://myLocalServer/login.defs -O /etc/login.defs
%end

where that copy of login.defs defines the uid/gid range I want.

This was preserved when all the packages landed, and I've got the right range of ID's on my new system.  The invention of %pre-install was just what (this) Dr. Ordered.  Thanks!
Comment 13 Martin Kolman 2017-06-30 18:06:55 EDT
(In reply to Habig, Alec from comment #12)
> This worked great.  In my ks script, I said:
> 
> %pre-install
> mkdir /etc
> wget http://myLocalServer/login.defs -O /etc/login.defs
> %end
> 
> where that copy of login.defs defines the uid/gid range I want.
> 
> This was preserved when all the packages landed, and I've got the right
> range of ID's on my new system.  The invention of %pre-install was just what
> (this) Dr. Ordered.  Thanks!

Nice! So problem solved I guess. :)
Comment 14 Habig, Alec 2017-07-06 11:22:58 EDT
Oops - didn't work after all.  At some point login.defs got over-written again anyway, and subsequent groups for daemons are now stepping on user uid space.

I've got some groups that obeyed my initial file, others that didn't, and I'm not sure at which point it went off the rails.

How can one reliably lock down a login.defs files so any shadow-utils updates don't overwrite it?  I though a .conf file that had been changed caused rpm to put any new copies in a .rpmnew file?
Comment 15 Miloslav Trmač 2017-07-06 13:31:17 EDT
Can you possibly correlate the timing, perhaps the change time of /etc/login.defs with the anaconda logs, to figure out whether it happened before/after/during package installation, and in particular whether it happened during the installation of shadow-utils?

This did work at some point (years?) in the past.  Perhaps RPM behavior has changed in the meantime?
Comment 16 Habig, Alec 2017-07-06 13:58:20 EDT
already (re)-dropped the proper login.defs over the wrong one, so lost that timestamp I'm afraid.

Reading the anaconda logs (there's a lot more in there than I thought!), the kickstarted in login.defs gets pulled in at the proper place (after fs creation, before yum goes to work).  Somewhere in there it created the local users with the proper uid/gid from this file: that's not logged, but it worked.  In my case, that started with 500 rather than 1000: so I guess that happened early in the install process?

Other rpm-created users and groups are logged as yum does its work.  shadow_utils is installed by yum ~1/3 of the way through the list, and after that the system daemon uid's in the 900's get made (and logged), presumably by the useradd -r command counting down from 1000 as expected.

So, even without the timestamp, it looks like the anaconda yum install of shadow_utils blew away the modified login.defs with a fresh version.

In the shadow_utils spec file, it is marked properly:

  %attr(0644,root,root)   %config(noreplace) %{_sysconfdir}/login.defs

However, reading this nice summary of what that actually means:

  http://people.ds.cam.ac.uk/jw35/docs/rpm_config.html

leads me to guess that such files are preserved on upgrade not on initial install.  There was no older package to be replaced, so rpm didn't check for "noreplace".

So: I've learned more, %pre-install is _almost_ the right workaround, but rpm's behavior means it doesn't actually work.

The "put a modified shadow_utils package in a local copy of the install repo" from earlier in this thread does work completely.  It's just more work for the installer.
Comment 17 Miloslav Trmač 2017-07-06 14:46:00 EDT
Yeah, let’s ask RPM experts.

tl;dr: It used to be the case around the time of Fedora 16 that when _first_ installing a RPM, a %config(noreplace) file from the RPM did not replace a preexisting file on the filesystem; now it seems that it does replace it.  Is there anything simple which can be done (either by the user before installing the RPM, or by the packager in the spec file) to prefer the preexisting file to the in-RPM one?
Comment 18 Florian Festi 2017-07-19 06:23:41 EDT
Config files are not replaced (but saved as .rpmnew) if the existing config file has been altered. So just edit the file. Adding white space is you really like the default config.
Comment 19 Habig, Alec 2017-07-19 08:46:24 EDT
Didn't work as advertised.  Empty filesystem other than a (modified) /etc/login.defs, and the anaconda-driven yum install of shadow-utils wiped out the existing file with the version in the rpm.
Comment 20 Tomáš Hozza 2017-08-01 05:36:03 EDT
It is a not really a nice solution, but if the filesystem you use supports extended file attributes, you could set the immutable attribute to the file. This way you can be sure it won't be overwritten.

You can set the bit by 'chattr +i <path>' and remove it by 'chattr -i <path>'.

Nevertheless it seems that if RPM replaces the file from the RPM, then can be either a bug in RPM, or you didn't follow the steps carefully.
Comment 22 Habig, Alec 2017-08-25 18:28:56 EDT
Put chattr +i /etc/login.defs in the %pre-install script after pulling in the hacked login.defs file I need.

It still got clobbered: after anaconda makes the requested initial accounts (which started at 500 as requested), and before rpm's are installed (making their own system level accounts starting at 1000).

Going back to the original solution of rebuilding the shadow-utils rpm from source (with uid's starting at 500 included), and putting that into my local repository.  That solution worked from end to end a year ago, hopefully still does.  Will let you know.
Comment 23 Habig, Alec 2017-08-31 10:20:17 EDT
... and, I see why the shadow-utils rpm is blasting the pre-existing login.defs.

Whomever implemented the change from 500->1000 dropped a new source file in there, 

Source1: shadow-utils.login.defs 

which has the "new" definitions of system vs. user IDs.  Then, in the %install stage, that file is blasted directly into place:

install -p -c -m 0644 %{SOURCE1} $RPM_BUILD_ROOT/%{_sysconfdir}/login.defs 

IIRC, the "-c" used to mean "config file", but the man page now says it's not implemented. 

Anyway, this bypasses rpm's handling of pre-existing conf files.  The proper thing to do is to patch the tarball's login.def's with the desired changes, and let rpm install (or not) the altered-at-build-time config file, rather than manually doing it at the %install step.  Or, heck, since shadow-utils is your package, why not just have what you want in the tarball source in the first place instead of trying to cope with it in the spec file?

So, this is a plain old bug in the spec file for shadow-utils.  You redhat people want to put this bug in, or should I?
Comment 24 Tomas Mraz 2017-08-31 10:26:20 EDT
Nope, your analysis is incorrect. The %install script is run only during the rpm build to install the file into the place where the rpmbuild gathers the files into the resulting rpm. It is not run when the package is installed on the target system.
Comment 25 Habig, Alec 2017-08-31 11:01:14 EDT
Doh!  You're right.

What was tripping me up in the rebuild of the rpm for my own purposes was that the install line is grabbing %{SOURCE1} instead of the actual login.defs file in the build tree.  What would that be, $RPM_BUILD_ROOT/%{_sysconfdir}/login.defs I think.  Ok, I guess that'll get simplified in a new shadow-utils release that rolls in the patches.

Sorry - was momentarily excited that I might have had a good reason why the existing config file was getting blasted.
Comment 26 Lenka Kimlickova 2017-10-10 07:09:42 EDT
Hi Alec,

I am doing my best to help you. Editing the config file should work as advertised in comment #18. Have you tried it since then? Does it work for you? If it does not, I would recommend you to open the engineering bug. Let me know if you need any further help.

Regards,

Lenka
Comment 27 Habig, Alec 2017-10-10 09:23:28 EDT
Thanks Lenka.  Unfortunately comment #18 didn't work.  A customized /etc/login.defs was dropped into place in %pre-install as suggested in comment #9 but was clobbered by the rpm install.  Replicated on several systems since.  The work-around is to install from a local repository where the customized login.defs file is included in a locally modified shadow-utils rpm.  This works.  That workaround is fine for me: it's just not the advertised behavior in the documentation referenced in comment #1, nor the way rpm is supposed to work (and, in fact, under all other circumstances rpm does leave conf files alone).
Comment 29 Tomáš Hozza 2017-10-18 10:49:56 EDT
Based on comment #27, this looks like as a bug in RPM. Maybe caused by the way Anaconda uses it.

Reassigning to RPM component for investigation.
Comment 30 Panu Matilainen 2017-10-19 04:29:20 EDT
This is all expected behavior from rpm POV: unpackaged pre-existing files are never left in place (but %config's are backed up) to ensure consistent installation.

Which makes sense when you think about a running system where content delivered otherwise is to be taken over by rpm. But none whatsoever in a situation like this where you intentionally pull in alternate config files in kickstart.

So while this is not a bug, I can see a legit RFE in it. Would be nice if the situation could be autodetected somehow but at least it should be possible to adde a transaction flag / config setting to leave pre-existing config files alone. Need to think about it some.

Note You need to log in before you can comment on or make changes to this bug.