Bug 426259

+++ This bug was initially created as a clone of Bug #426258 +++

+++ This bug was initially created as a clone of Bug #128622 +++

When upgrading packages which are installed for both the 32- and
64-bit architecture, rpm creates unnecessary .rpmsave and .rpmnew files:

rpm -q --queryformat '%{NAME}-%{VERSION}-%{RELEASE}.%{ARCH}\n' pam
pam-0.77-50.x86_64
pam-0.77-50.i386

warning: /etc/pam.d/other created as /etc/pam.d/other.rpmnew
warning: /etc/security/access.conf created as
/etc/security/access.conf.rpmnew
warning: /etc/security/chroot.conf created as
/etc/security/chroot.conf.rpmnew
warning: /etc/security/console.perms created as
/etc/security/console.perms.rpmnew
warning: /etc/security/group.conf created as
/etc/security/group.conf.rpmnew
warning: /etc/security/limits.conf created as
/etc/security/limits.conf.rpmnew
warning: /etc/security/pam_env.conf created as
/etc/security/pam_env.conf.rpmnew
warning: /etc/security/time.conf created as /etc/security/time.conf.rpmnew

All these files were identical.

-- Additional comment from jbj on 2004-08-03 10:02 EST --
rpm does exactly what it has always done, and must continue to do
so. That is not "unnecessary".

The only difference with elf32 and elf64 pkgs installed is
that path collisions are more likely, and so you are seeing
*.rpmnew files created more often.

-- Additional comment from thomasz on 2004-08-06 06:30 EST --
You are almost right, this is what rpm has always done and what was
appropiate for single architecture systems. But now that we have
x86_64 support this behaviour has become an unnecessary annoyance. The
alternative to fixing this in rpm would be to change every i386
package built for x86_64 to not include these files, but do we really
want this?

-- Additional comment from jbj on 2004-08-06 09:01 EST --
I cannot change the existing behavior in rpm without violating
some expectations.

Nor is there any clearly defined algotithm to improve the
existing behavior when given identical packages
for elf32/elf64 to be installed.

So, WONTFIX, but that does not mean that I disagree
with "unnecessary annoyance".

-- Additional comment from thomasz on 2004-08-06 14:59 EST --
Fine, so I guess this needs to be brought before the steering
committee, how do we proceed?

-- Additional comment from tmraz on 2005-08-05 08:51 EST --
Is it really necessary to create .rpmnew files when only timestamp differs?


-- Additional comment from tmraz on 2005-08-05 08:54 EST --
*** Bug 144747 has been marked as a duplicate of this bug. ***

-- Additional comment from goemon on 2005-10-14 07:37 EST --
this is silly.

/etc/cron.daily/yum.cron:

warning: /etc/pki/tls/certs/ca-bundle.crt created as
/etc/pki/tls/certs/ca-bundle.crt.rpmnew
warning: /etc/pki/tls/openssl.cnf created as /etc/pki/tls/openssl.cnf.rpmnew

# ls -la
total 20
drwxr-xr-x  5 root root  248 Oct 14 04:09 .
drwxr-xr-x  6 root root  144 Sep 14 23:13 ..
lrwxrwxrwx  1 root root   19 Oct 14 04:09 cert.pem -> certs/ca-bundle.crt
drwxr-xr-x  2 root root  272 Oct 14 04:09 certs
drwxr-xr-x  2 root root  192 Oct 14 04:10 misc
-rw-r--r--  1 root root  660 Sep 20 20:26 new.cert.csr
-rw-r--r--  1 root root 7965 Oct 12 03:22 openssl.cnf
-rw-r--r--  1 root root 7965 Oct 12 03:22 openssl.cnf.rpmnew

# md5sum openssl.cnf*
ef0fc8ad519bb9ed9eb7456cb25fa486  openssl.cnf
ef0fc8ad519bb9ed9eb7456cb25fa486  openssl.cnf.rpmnew

# ls -al
total 860
drwxr-xr-x  2 root root    272 Oct 14 04:09 .
drwxr-xr-x  5 root root    208 Oct 14 04:31 ..
-rw-r--r--  1 root root 427833 Oct 12 03:22 ca-bundle.crt
-rw-r--r--  1 root root 427833 Oct 12 03:22 ca-bundle.crt.rpmnew

# md5sum ca-bundle.crt*
7c14b8d716016ef958cebe56c1264b3f  ca-bundle.crt
7c14b8d716016ef958cebe56c1264b3f  ca-bundle.crt.rpmnew


-- Additional comment from thomasz on 2005-10-14 07:43 EST --
Created an attachment (id=119969)
perl script to remove duplicates

I have created a small perl script to erase unnecessary (identical)
*.rpm(new|save) files I have yet encountered.

-- Additional comment from n3npq on 2006-04-05 19:17 EST --
work around script WORKSFORME

-- Additional comment from orion.com on 2007-02-13 00:15 EST --
Reopening this hoping that with new RPM maintainer and direction that this will
be addressed.

-- Additional comment from tmraz on 2007-02-13 09:50 EST --
Created an attachment (id=147987)
Proposed patch

This patch will prevent RPM to create .rpmnew and .rpmsave files when
file/symlink on disk differs just by timestamp.


-- Additional comment from axel.thimm on 2007-02-13 10:32 EST --
This would still leave rpm -V output non-quiet.

How about instead of modifying runtime behaviour to implicitely assume a
%verify(not mtime) at rpmbuild time when encountering %config tags?

No conflicts, rpm -V would be silent, too and even an unpatched rpm would play
along. The latter is interesting for more conservative distributions like RHEL,
which could live with patching rpmbuild, but less with touching rpm/rpmlibs that
are being (or already have been) shipped.


-- Additional comment from tmraz on 2007-02-13 10:40 EST --
%verify(not mtime) is AFAIK not consulted in case of rpm package install.
So the .rpmnew .rpmsave files would be created if the patch above is not applied.


-- Additional comment from aleksey on 2007-02-13 12:07 EST --
According to Tomas, his patch (attachment 147987 [details]) should also fix bug 29470,
marking it as a dependency.

-- Additional comment from n3npq on 2007-02-13 12:22 EST --
%verify(not mtime) controls what parameters are to be verified and should not be
abused
for %config(...) handling. A different mechanism should be used, and it should
be end-user,
not package builder, configurable.

-- Additional comment from tmraz on 2007-02-13 12:27 EST --
I agree with comment #15.


-- Additional comment from axel.thimm on 2007-02-13 13:18 EST --
(In reply to comment #13)
> %verify(not mtime) is AFAIK not consulted in case of rpm package install.
> So the .rpmnew .rpmsave files would be created if the patch above is not applied.

It is, I just tried it and it works:



Name:           test
Version:        1
Release:        1
Summary:        test
Group:          somegroup
License:        none
#BuildRoot:     %{_tmppath}/%{name}-%{version}-%{release}-root
BuildRoot:      %(mktemp -d %{_tmppath}/%{name}-%{version}-XXXX)
%description
test

%prep
echo %{buildroot}

%build
echo %{buildroot}

%install
rm -rf %{buildroot}
mkdir -p %{buildroot}/etc/test-config
echo Test > %{buildroot}/etc/test-config/configfile

%clean
rm -rf %{buildroot}


%files
%defattr(-,root,root,-)
%dir /etc/test-config
%config %verify(not mtime) /etc/test-config/configfile
~


-- Additional comment from orion.com on 2007-02-13 13:33 EST --
(In reply to comment #17)
> %files
> %defattr(-,root,root,-)
> %dir /etc/test-config
> %config %verify(not mtime) /etc/test-config/configfile

Really?  More cruft we need to add to every spec file?  Let's get rid of this
kind of stuff, not add more.

-- Additional comment from n3npq on 2007-02-13 13:40 EST --
No, all files are going to need a %dist attribute so that file conflicts
between, say, fc6 and fc7
can be automagically resolved to prefer "fc7".

So more gunk, not less, is what is needed!

-- Additional comment from axel.thimm on 2007-02-13 13:48 EST --
(In reply to comment #15)
> %verify(not mtime) controls what parameters are to be verified and should not
be abused
> for %config(...) handling. A different mechanism should be used, and it should
be end-user,
> not package builder, configurable.

I think %config files should default to "not mtime" regardless of multilib. If
the packager knows that the timestamp matters he can turn the verify bit on
again for this config file.

Also the decision whether a timestamp of a config file should be part of
verification/conflict checking should be primarily at the packager's hands and
thus part of the resulting build - especially if there are any config files
where the timestamp does matter and the packager chose to turn timestamp
verification on.

That is not to say that having an option for the user to override this behaviour
is bad.

-- Additional comment from axel.thimm on 2007-02-13 13:53 EST --
(In reply to comment #18)
> (In reply to comment #17)
> > %files
> > %defattr(-,root,root,-)
> > %dir /etc/test-config
> > %config %verify(not mtime) /etc/test-config/configfile
> 
> Really?  More cruft we need to add to every spec file?  Let's get rid of this
> kind of stuff, not add more.

I wasn't suggesting to add this to the specfiles, at all, you misread the above.
Check comment #12 (especially the word "implicitely" ;).

Comment #17 was there only to demonstrate that conflict resolution in rpm does
take into account the verify flags set at rpmbuild time, nothing more. Next step
would be to have %config imply a default of %verify(not mtime) so no specfile
needs to be touched.

-- Additional comment from n3npq on 2007-02-13 14:06 EST --
Make sure that %config implies %{dist} as well. File conflicts need to be
resolved, not just detected.

-- Additional comment from hugh on 2007-02-13 16:23 EST --
We can (and should) debate what the right fix is.  Maybe it would be useful to
mitigate the problem while we wait for the utlimate solution.  I've had to live
with this problem for two and a half years already.

The current diagnostic could be refined to report what differences were
encountered, perhaps in the way verify does.  If I could read in the message
that only the timestamp differed, I could decide to ignore the message.  As it
is now, I feel I should diff each file that is reported.


The fundamental problem arises when two binary RPMs are really the same package,
just for different architectures.  RPM has no convenient way of modelling the
sharing involved.  There is an inconvenient way: add a noarch package for the
shared portion.  Perhaps there is a way to automate the inconvenient way so as
to make it convenient.

If we don't address the fundamental problem, there is a good chance that the fix
is just a cover-up.  It may turn out to suppress diagnostics for a real case
that deserves diagnostics.  For example, if I manually changed a config file,
and the result had the same text as the config file from the next version, I'd
like a diagnostic.

-- Additional comment from hugh on 2007-02-13 16:27 EST --
Another bug that reflects bad modelling of sharing between packages for
different architectures: https://bugzilla.redhat.com/bugzilla/show_bug.cgi?id=209306

-- Additional comment from n3npq on 2007-02-13 17:07 EST --
The definintion of "sharing" within rpm is quite clear, the items that are compared
have not changed for years.

What is different is that some of the items, like mtime, can and will differ
from builds
on different machines. Even content can differ on different builds.

But distributing configuration, the issue w %config, from two packages that are
supposedly,
but not actually, the same leads to *.rpmnew litter unsurprisingly.

This is a design flaw, not otherwise. You are quite correct.

-- Additional comment from philipp on 2007-02-13 20:13 EST --
(In reply to comment #23)
> We can (and should) debate what the right fix is.  Maybe it would be useful to
> mitigate the problem while we wait for the utlimate solution.  I've had to live
> with this problem for two and a half years already.
> 
> [...]
> The fundamental problem arises when two binary RPMs are really the same package,
> just for different architectures.  RPM has no convenient way of modelling the
> sharing involved.

That's a valid concern.  I think that the issue of "fat binaries" is perhaps the
most glaring instance of this problem:  it might be possible in the future to
have a single binary that is .i586, .i686, .x86_64, and .emt64 (or whatever the
64-bit Intel architecture is called).  In such instances, it will often be the
degenerative case of all 4 architectures (or compiler settings) sharing a common
config file.


-- Additional comment from n3npq on 2007-02-17 17:13 EST --
FAT binaries already exist on Mac OS X, so does rpm.

Meanwhile, arch is unlikely the correct attribute to model config files.

ELF32/ELF64 executables/libraries within rpm is already ok, its the "other""
files that are causing 
problems.

-- Additional comment from philipp on 2007-02-17 21:48 EST --
(In reply to comment #27)
> FAT binaries already exist on Mac OS X, so does rpm.
> 
> Meanwhile, arch is unlikely the correct attribute to model config files.
> 
> ELF32/ELF64 executables/libraries within rpm is already ok, its the "other""
files that are causing 
> problems.

That wasn't really my point.  I'm suggesting that if you had a single RPM with
all x86 binaries, then at install time you could specify that you wanted i386
and x86_64 libraries... and still have a single config file be installed because
that's all the RPM would need to contain... thus indirectly eliminating the case
of having two identical config files with different timestamps.



-- Additional comment from n3npq on 2007-02-18 00:24 EST --
There are practical difficulties compiling elf32 and elf64 for inclusion
in the same package with shared config.

The likelier solution is splitting config out into another common package, leaving
elf32 and elf64 as is.

-- Additional comment from n3npq on 2007-02-19 19:54 EST --
FWIW, the patch in comment #11 is in rpm-4.4.8-1.

-- Additional comment from n3npq on 2007-02-19 20:10 EST --
Eeek, too many %config bugs.

Ignoring mtime as end-user configurableI haven't done yet.

What has been added to rpm-4.4.8 is to scope %config more narrowly, so that
%config files are compered only against files using a digest, while %config
links are
compared only against links comparing symlink end-points.

That removes a source of (perhaps unnecessary) .rpmnew creation when a
symlink is followed to compare a file digest.

Paramaterizing mtime (and digest and symlink endpoint and ...) comparisons
very soon. All of the %config .rpmnew creation needs to be carefully simplified and
eliminated in rpm imho.

Hint: Saving virgin %config files in a shadow tree and adding --diff to rpmquery is
likely much more useful to most than adding a .rpmnew prefix.

-- Additional comment from aleksey on 2007-02-19 22:25 EST --
(In reply to comment #31)

> Hint: Saving virgin %config files in a shadow tree and adding --diff to
rpmquery is
> likely much more useful to most than adding a .rpmnew prefix.

I would like to disagree. Even if there was an "rpmquery --diff" (which, I
agree, would be very nice), I would still want some sort of indicator "the
default values for this config have changed, you need to review this config" -
this is what "locate .rpmnew" (e.g. following an upgrade) currently provides
nicely, while "rpmquery --diff" would be of no help for that.


-- Additional comment from n3npq on 2007-02-20 07:20 EST --
Everyone wants to disagree.

Let's see:

 1) You disagree.
 2) You agree that --diff would be nice.
 3) You want an indicator like .rpmnew.
 4) You don't want an indicator if mtime (or other values changed).
 5) You seem to want a dialog or report (you need to review ...)

all from a misposted comment in this bug report about vaporware.

-- Additional comment from n3npq on 2007-04-29 08:59 EST --
Comment added so that I can this bug in spite of bugzilla normal -> medium churn.

-- Additional comment from hugh on 2007-05-28 17:32 EST --
Here is a shell script to help with this problem.  It assumes that the locate
database has been updated recently enough.

For each .rpmnew file (found by locate), the file is compared with the
non-rpmnew version.  If the files have identical content (ignoring
permissions and timestamps) the .rpmnew is deleted; otherwise, a diff
is done.

# remove .rpmnew files that have content that is the same as the base file

set -u -e

locate '*.rpmnew' |
    while read FS
    do
        F=`expr match "$FS" '\(.*\)\.rpmnew$'`
        if cmp "$F" "$FS"
        then
            echo remove "$FS"
            rm "$FS"
        else
            [ "$?" == 1 ] && diff -u "$F" "$FS" || :
        fi
    done


-- Additional comment from pmatilai on 2007-07-04 09:49 EST --
*** Bug 246236 has been marked as a duplicate of this bug. ***

-- Additional comment from pmatilai on 2007-08-09 09:34 EST --
The original issue fixed upstream at rpm.org by patch from c#11, thanks Tomas.
Also will be in next rawhide build (rpm-4.4.2.1-5.fc8).

Open new bugs for the other issues discussed here if necessary, this one's
closing now.

-- Additional comment from daw-redhatbugzilla.berkeley.edu on
2007-08-26 14:30 EST --
*** Bug 254721 has been marked as a duplicate of this bug. ***

-- Additional comment from astokes on 2007-12-19 11:59 EST --
1. Provide time and date of problem

No specific logs available, problems happens almost on all RHEL systems when
updating packages which contain configuration files.

2. Provide clear and concise problem description as it is understood at the time
of escalation

When updating a package, RPM has the tendency to create *.rpm(new|save) files
even when it is not necessary. This causes a lot of unneeded files to be present
on the file-system which only confuse the user.

Currently RPM will treat a file as conflicting if any of the properties that
would be flagged up by "rpm -V" are different. However it often happens that an
.rpmnew file is created and that an "rpm -V" against this new file returns nothing.

This has been a long-standing issue for RPM which has been indirectly resolved
in Fedora by including a patch from Tomas Mraz <tmraz> from BZ#128622
which uses MD5 to decide whether the files are different or not.

3. State specific action requested of SEG

Escalate to Engineering and request backport of the fix from Fedora to RHEL5.

4. State whether or not a defect in the product is suspected

Yes, well known RPM problem.

5. If there is a proposed patch, make sure it is in unified diff format (diff -pruN)

Patch from Tomas Mraz <tmraz> in BZ#128622:
https://bugzilla.redhat.com/attachment.cgi?id=147987