Bug 464695

I'm pretty sure the platform should be changed to "All".  The behavior is the same on i386.  And the danger exists as long as the passwd.rpmnew file continues to exist, since at any time someone might do a search for *.rpmnew files, find /etc/passwd.rpmnew, and do the replacement without looking at it.  It would be nice to have another setup update that modifies the existing passwd file and deletes the existing passwd.rpmnew file, if it exists, to remove the danger.  Although with F10 coming out in a little over a month it might not be deemed worth the trouble.

As new maintainer of setup package, I guess it would be even more confusing to have message about creating /etc/passwd.rpmnew file (done by RPM) and /etc/passwd.rpmnew file deleted by %post scriptlet. I agree that modification of /etc/passwd should be prevented in "live" distributions and should be done only in rawhide - to prevent such confusing situations for new admins. Intention was (according to ChangeLog) to remove news user from default /etc/passwd - but because /etc/passwd is already modified when updating setup package from default installation, this change is not working properly at all. 

Anyway, I'm not convinced removal of .rpmnew is good solution - as rpm will report  warning message about .rpmnew file, only thing I can promise is to be more careful about updating config(noreplace) files like /etc/passwd in future. I guess no admin with at least basic knowledge of the Linux system will use that /etc/passwd.rpmnew file for replacing existing /etc/passwd - and common unexperienced admin will keep .rpmnew files as they are... so IMHO the risk is pretty minimal. 

Ok with closing CANTFIX? I don't like "removal of .rpmnew" solution...

(In reply to comment #2)
> As new maintainer of setup package, I guess it would be even more confusing to
> have message about creating /etc/passwd.rpmnew file (done by RPM) and
> /etc/passwd.rpmnew file deleted by %post scriptlet.

I'm not sure I agree with this, though it probably doesn't matter.  For me, what matters to me more than the messages printed out is the state that the filesystem is left in.  To my thinking, the worst case is to leave behind a broken /etc/passwd.rpmnew file.  I don't know for certain that the /etc/passwd.rpmnew file in my case would have broken my system, but I'm concerned that it would have -- and if that is accurate, it seems like a problem.  The question I'm still stuck on is this: Why is the RPM creating a broken /etc/passwd.rpmnew file in the first place?

> I agree that modification
> of /etc/passwd should be prevented in "live" distributions and should be done
> only in rawhide - to prevent such confusing situations for new admins.

Hmm.  I'm not sure who you are agreeing with here, as I don't think that either Andre Robatino or I proposed this.  In any case, speaking personally, I don't have any problem with live modification of /etc/passwd as long as it is done correctly, in a way that doesn't break my system.

> Intention was (according to ChangeLog) to remove news user from default
> /etc/passwd - but because /etc/passwd is already modified when updating setup
> package from default installation, this change is not working properly at all.

I see.  I'm still surprised this RPM converted ':x:' to ':*:' and deleting 31 other accounts as well, if the intent was just to delete the 'news' account.  That sounds like something didn't work out as intended/expected.

If the purpose is to delete one account, would it help to use 'userdel' to do so?  If I understand correctly, 'userdel' is unlikely to break your system (as long as the user account being removed is truly unused).  I probably don't understand all the constraints, so I'm just throwing out ideas here, though I realize they may not make sense.

By the way, I have INN installed and running on my system.  INN runs as user 'news', so removing the 'news' user would break a working system.  I hope and assume that this was only going to remove the 'news' user if there is no rpm installed that needs it.

> I guess no admin with at least basic knowledge of the Linux system will use
> that /etc/passwd.rpmnew file for replacing existing /etc/passwd

Hmm.  Well, I guess I have a different perspective on that.  I can see myself replacing /etc/passwd with /etc/passwd.rpmnew, on a bad day.  I consider it undesirable to leave lying around a /etc/passwd.rpmnew file that is so broken that it would break my system if I installed it.  To me that feels like a landmine waiting for the unwary to step on it -- it doesn't seem like a desirable user experience to me.
 
> Ok with closing CANTFIX? I don't like "removal of .rpmnew" solution...

I'm not sure what CANTFIX implies, and I'm not sure if this question is directed to me.

If you're asking me, I personally do not find it desirable to leave it unfixed.  Just my opinion.  That may not be the question you're asking, though ....

Let me share my viewpoint, as one user (keeping in mind that this is a sample size of one, and I may not be representative, so you will need to filter this through your own experience as well).  In my mind, there are two problems:

Problem 1. Some RPM left my filesystem in a dangerous state (i.e., one where there is a /etc/passwd.rpmnew file that might break my system if it were installed).

Problem 2. Something about the way that RPMs work, or the quality assurance process, allowed Problem 1 to occur.  (I'm fuzzy about the details here.)

To my thinking, Problem 1 is an unfortunate outcome.  It's not friendly to the sysadmin, and I think it's dangerous.  I'm not wedded to any particular solution to Problem 1.  However I would point out even if one candidate solution turns out to be unworkable, the problem still remains.  I don't have full knowledge, but I would have expected that Problem 1 ought to be fixable (i.e., this particular instance of the problematic RPM can be fixed to no longer do that).

I'm not sure what to say about Problem 2.  As a user, I would be disappointed to see this occur again in the future.  Of course I personally will be much more attuned to this issue in the future so it probably wont' be a problem fo rme.  Still, for other users, it seems like it might be nice to have some way to avoid this happening again, if possible -- whether that's a technical mechanism, or awareness that creating broken /etc/passwd.rpmnew files is unacceptable, or something else, I don't know.

I wonder if it would make sense to have a re-examination about the way that /etc/passwd files are handled.  Why is the RPM creating /etc/passwd.rpmnew files in the first place?  Why are modifications to the passwd file handled by creating a new file?  If modifications are needed, should they perhaps be handled by writing a script that modifies the passwd file in-place (after checking that all preconditions are established and that the passwd file is in a state where the script can be guaranteed to work safely)?  Would it make sense to completely forbid, either through technical or nontechnical means, any RPM from creating/modifying the /etc/passwd file?  Would it make sense to have a restriction that no RPM shall create any passwd.rpmnew file, if we can't be confident that the new file it creates will be safe to install?  Would that be a possible approach to handling this?  Should some check go into rpmlint?  I don't have answers to any of these questions.  I know it's easy for me to ask these questions, since I don't have to implement it, but I haven't understood yet why it is impossible to fix this.  If CANTFIX means that it is impossible to fix the problem, then I wonder whether that is really applicable.

I should also say this:

I recognize this is not my call.  I understand and respect that.  I appreciate that many/most Fedora folks work on this as a volunteer in your spare time.  I very much appreciate the time you've spent on this.  I have absolutely no right to demand any more of your time.  I know that I'm not paying a cent and so it's totally unreasonable of me to have any expectations.  I hope you won't take this bugzilla report as criticism of you or your dedication, because I very much appreciate the software product that you build.

My only purpose is to share my personal perspective as one user -- but I realize that other considerations may trump that, or that my perspective may not be shared by others, or I may have unreasonable expectations, or I may have misunderstood something.  If so, that's OK!  I will completely understand.  And I hope you won't find this message to be offensive or disrespectful or inappropriate in any way.

(In reply to comment #3)
> (In reply to comment #2)
> > I agree that modification
> > of /etc/passwd should be prevented in "live" distributions and should be done
> > only in rawhide - to prevent such confusing situations for new admins.
> 
> Hmm.  I'm not sure who you are agreeing with here, as I don't think that either
> Andre Robatino or I proposed this.  In any case, speaking personally, I don't
> have any problem with live modification of /etc/passwd as long as it is done
> correctly, in a way that doesn't break my system.

Ok, sorry, my bad wording - I just tried to say not modifying /etc/passwd directly by making new /etc/passwd in rpm package in "live" distribution. This causes this .rpmnew situation as the scenario is following:
1) You install clean system (and setup package) - with /etc/passwd file A
2) Many packages and ANYONE could and will add users to the system - and they will make /etc/passwd being file B
3) Any direct distribution of modified /etc/passd file C in setup package update causes .rpmnew file

So correct solution for any change for /etc/passwd in "live" distribution should be to modify /etc/passwd in %post scriptlet. This will create no .rpmnew ... therefore I have written that I agreee that modification of /etc/passwd should be prevented - because direct modification (different /etc/passwd file shipped by setup rpm) is useless and leads to dangerous .rpmnew file in /etc. Just bad wording from me.


> I see.  I'm still surprised this RPM converted ':x:' to ':*:' and deleting 31
> other accounts as well, if the intent was just to delete the 'news' account. 
> That sounds like something didn't work out as intended/expected.

It's caused by converted/unconverted version of /etc/passwd file, for details see http://linux.derkeiler.com/Mailing-Lists/Fedora/2008-09/msg03096.html - that's the thread where were complaints about setup update before.

> If the purpose is to delete one account, would it help to use 'userdel' to do
> so?  If I understand correctly, 'userdel' is unlikely to break your system (as
> long as the user account being removed is truly unused).  I probably don't
> understand all the constraints, so I'm just throwing out ideas here, though I
> realize they may not make sense.
> 
> By the way, I have INN installed and running on my system.  INN runs as user
> 'news', so removing the 'news' user would break a working system.  I hope and
> assume that this was only going to remove the 'news' user if there is no rpm
> installed that needs it.

No - the purpose was to move creating of the news user and group to inn directly. It was not supposed to change that in F-9, that change was thought to be done in Rawhide only (to make it for F-10). But it accidently happened... Userdel was not option - as the user/group news could have been in use on the system - as is in use (at least) by INN and leafnode packages.

> > Ok with closing CANTFIX? I don't like "removal of .rpmnew" solution...
> 
> I'm not sure what CANTFIX implies, and I'm not sure if this question is
> directed to me.

CANTFIX means in this case means, that the problem happened, but solving it somehow could make even more troubles. I have in general three options:
1) To keep it as it is and to be aware of such confusing things in future
2) To add rm -rf /etc/passwd.rpmnew to the %post section of rpm
3) To change /etc/passwd to old version in next update

Option 1 will lead to no /etc/passwd.rpmnew on system with updated F-9 installations (where /etc/passwd.rpmnew file was already created by previous package) after setup updates. Will create /etc/passwd.rpmnew when updating clean F-9 installation.

Option 2 could lead to another reports about confusing report about created and missing /etc/passwd.rpmnew file from RPM - I could not prevent it - it will happen as the changed /etc/passwd is already in setup rpm - so changing it back will produce that message for update from previous version - and keeping it as it is will produce that message when updating from clean F-9 installation.

Option 3 causes that /etc/passwd.rpmnew file will be created for updated F-9 installations (where /etc/passwd.rpmnew file was already created by previous package). No /etc/passwd.rpmnew file created after update of clean F-9 installation.

> If you're asking me, I personally do not find it desirable to leave it unfixed.
>  Just my opinion.  That may not be the question you're asking, though ....

You are right, but I'm afraid that the only option which generally "solves" the issue (Option 2) will cause reports about missing /etc/passwd.rpmnew file which was reported by RPM... but OK, I do plan F9 setup update soon (once rawhide will be opened again - now in F10 freeze status), so I'm ok to do that there. It's probably the best choice.

> Let me share my viewpoint, as one user (keeping in mind that this is a sample
> size of one, and I may not be representative, so you will need to filter this
> through your own experience as well).  In my mind, there are two problems:
> 
> Problem 1. Some RPM left my filesystem in a dangerous state (i.e., one where
> there is a /etc/passwd.rpmnew file that might break my system if it were
> installed).
> 
> Problem 2. Something about the way that RPMs work, or the quality assurance
> process, allowed Problem 1 to occur.  (I'm fuzzy about the details here.)

Quality assurance in Fedora is community. Maintainer of course checks the update on his machine, but it is easy to miss some issue. So Fedora users are Fedora QA and they should report problems. Maintainer should try to fix them. 

> Why is the RPM creating /etc/passwd.rpmnew files in the first place?  

File /etc/passwd is marked as config(noreplace). That prevents accident automatic replacing of modified file. Admin should merge the changes between old and new updated file if they apply to his system. In the case of /etc/passwd it is a bit misleading and such changes should not happen this way. 

> Why are modifications to the passwd file handled by creating a new file?  

Explained above ... but quickly again. New .rpmnew file is created if the file changed between installed version and rpm, but user/system already modified the file.

> If modifications are needed, should they perhaps be handled by writing a 
> script that modifies the passwd file in-place (after checking that all 
> preconditions are established and that the passwd file is in
> a state where the script can be guaranteed to work safely)?  

Yep, exactly ... files like /etc/passwd should not be changed directly, but by usermod/useradd commands in %post scriptlet - if necessary.

> Would it make sense to completely forbid, either through technical or nontechnical means, any RPM from creating/modifying the /etc/passwd file?  

I would say not - it's not only about /etc/passwd , there are more such files and hardcoding of some restricions is not the best way. Maintainer of the package should usually prevent such things.

> Would it make sense to have a restriction that no RPM shall create any 
> passwd.rpmnew file, if we can't be confident that the new file it creates
> will be safe to install? 

I guess every .rpmnew file should be handled carefully - it means that the basic config file changed between update and installed version. Simply using .rpmnew file will cause removing of user/system changes and potentially could make troubles. That's exactly the case of passwd file, as many packages (and administrator) add users to it. Will ask rpm maintainer if there is a way how to handle such situations.

> Would that be a possible approach to handling this? 

Possible approach would be to have some e.g. %config(noreplace, norpmnew) possibility for such files to update the file only if the file was not modified by user/system. That should keep rpm simple without hardcoded things and solve the issue for every such file.

> Should some check go into rpmlint?

I'm not sure if it does make sense to add rpmlint check only for few files which could make disaster when used as .rpmnew files. I would prefer having some option as written in previous answer.

OK, thanks for the detailed explanation.  I think I understand now, and it sounds like you're on top of this.  Sounds like this was an inadvertent slip-up; that you are attuned to this for the future; and the proposal is to be careful about this for the future.  Works for me.

Your proposal for the future (don't ship a replacement /etc/passwd file in RPM updates; instead, a %post scriptlet should modify the existing /etc/passwd file as needed) sounds like a reasonable solution to this problem.  Thanks.  OK with me to mark this bugzilla entry as CANTFIX, for what that's worth.

P.S. On a related topic, I'm not clear on what will happen when a user upgrades from, say, a F9 install to a F10 install (rather than reformatting their disk and installing F10 clean).  Will those users experience this problem in that scenario?  If so, it might be nice if the setup RPM had a way to detect this case: install /etc/passwd if there is no /etc/passwd file, otherwise modify the existing F9 /etc/passwd file in place as needed if it already exists.  I don't know if that's feasible or if I'm understanding the implications of this issue for F10 accurately.  Perhaps you've already got a plan for that as well.

I just upgraded F9 -> F10 and then did a 'yum -y upgrade', and it happened again, but this time with /etc/group.rpmnew instead of /etc/passwd.rpmnew.  It seems that rpm was trying to convert :x: to :: in /etc/group, but in the process it created a new /etc/group.rpmnew file that is missing a whole bunch of groups in my /etc/group.  Can I suggest fixing the handling of /etc/group, similar to the way that handling of /etc/passwd was fixed?

Not really - it's issue of rpm - and was reported against rpm ( http://www.rpm.org/ticket/6 ) - once the ticket will be closed and rpm config option added, I could fix it - otherwise it can't be done. Additionally - it is very hard to use post scriptlets in setup subpackage - as this one is installed very early in installation tree - so even the dependency on /bin/sh is not acceptable. I guess I should close that bugzilla deferred - could not be fixed easily by setup package at the moment and will be fixed after the rpm upstream ticket resolution.