Bug 1835424 - rpm test transaction fails with upgrade of package that uses %pretrans to replace a symlink to another package's files with a directory
Summary: rpm test transaction fails with upgrade of package that uses %pretrans to rep...
Keywords:
Status: NEW
Alias: None
Product: Fedora
Classification: Fedora
Component: rpm
Version: 35
Hardware: All
OS: Linux
unspecified
high
Target Milestone: ---
Assignee: Packaging Maintenance Team
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard:
: 2031013 (view as bug list)
Depends On:
Blocks: 2038302 1828899
TreeView+ depends on / blocked
 
Reported: 2020-05-13 18:23 UTC by Stephen Gallagher
Modified: 2022-08-30 12:14 UTC (History)
14 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2021-05-25 17:20:56 UTC
Type: Bug


Attachments (Terms of Use)
Specfile to reproduce the bug (1.44 KB, text/plain)
2020-05-13 18:23 UTC, Stephen Gallagher
no flags Details
Specfile to show example, that says when pretrans is run (1.48 KB, text/x-matlab)
2020-07-10 19:43 UTC, Troy Dawson
no flags Details
Specfile that shows possible work around (1.26 KB, text/x-matlab)
2020-07-10 19:43 UTC, Troy Dawson
no flags Details

Description Stephen Gallagher 2020-05-13 18:23:28 UTC
Created attachment 1688167 [details]
Specfile to reproduce the bug

Description of problem:
DNF falsely claims a conflict when attempting to upgrade a package that is replacing a symlink to a directory belonging to a different package with its own directory (for example, a package that has forked and modified a bundled dependency and can no longer refer to the system copy).

Version-Release number of selected component (if applicable):
dnf-4.2.21-1.fc32.noarch
libdnf-0.47.0-1.fc32.x86_64

How reproducible:
Every time

Steps to Reproduce:
1. Build the specfile provided as an attachment to this bug twice, once normally and once with `--with pretrans`
2. Install sgallagh-symlink-update-0-1.fc32 on the system. Verify that /opt/bug/package/etc is a symlink to /etc 
3. Use DNF to attempt to update to sgallagh-symlink-update-0-2.fc32.

Actual results:
Error: Transaction test error:
  file /opt/bug/package/etc/shells from install of sgallagh-symlink-update-0-2.fc32.noarch conflicts with file from package setup-2.13.6-2.fc32.noarch


Expected results:
The upgrade should complete successfully, because the %pretrans scriptlet
properly removes the symlink before the RPM transaction attempts to replace it.


Additional info:
It works fine when upgrading directly with `rpm -Uvh`:

```
[root@4bf814bd44f0 /]# rpm -Uvh /distgit/RPMS/noarch/sgallagh-symlink-update-0-2.fc32.noarch.rpm 
Verifying...                          ################################# [100%]
Preparing...                          ################################# [100%]
Updating / installing...
   1:sgallagh-symlink-update-0-2.fc32 ################################# [ 50%]
Cleaning up / removing...
   2:sgallagh-symlink-update-0-1.fc32 ################################# [100%]
```


This will affect a lot of Node.js packages in the near future, as the Node.js
SIG is working on reworking packaging to be more compatible with upstream
style (which emphasizes bundling).

Comment 1 Stephen Gallagher 2020-05-13 18:51:02 UTC
Reassigning to RPM; it turns out that the same error occurs if I run `rpm --test`. So there's a bug somewhere between the test transaction and the real transaction.

```
[root@75dffca6fbca noarch]# rpm --test -Uvh sgallagh-symlink-update-0-2.fc32.noarch.rpm
Verifying...                          ################################# [100%]
Preparing...                          ################################# [100%]
	file /opt/bug/package/etc/shells from install of sgallagh-symlink-update-0-2.fc32.noarch conflicts with file from package setup-2.13.6-2.fc32.noarch

```

Comment 2 Panu Matilainen 2020-05-20 10:26:05 UTC
Thanks for the nice reproducer!

Rpm currently only allows %pretrans presence to escape file conflicts on test upgrade in the very specific case that the conflicting file would get removed on the transaction anyway. Typically this would be an older version of the package itself, which naturally gets removed on an upgrade. Here, the conflict is on a file owned by setup, which isn't going anywhere. Of course the conflict is actually caused by an intermediate symlink that would go away in this transaction, but rpm doesn't see that.

Can't say offhand what, if anything, we can actually do about this (short of relaxing the %pretrans rule to cover more or less any conflict, which doesn't seem healthy) but at least the simple reproducer makes the case comprehensible.

Comment 3 Stephen Gallagher 2020-05-20 13:53:26 UTC
(In reply to Panu Matilainen from comment #2)
> Thanks for the nice reproducer!
> 
> Rpm currently only allows %pretrans presence to escape file conflicts on
> test upgrade in the very specific case that the conflicting file would get
> removed on the transaction anyway. Typically this would be an older version
> of the package itself, which naturally gets removed on an upgrade. Here, the
> conflict is on a file owned by setup, which isn't going anywhere. Of course
> the conflict is actually caused by an intermediate symlink that would go
> away in this transaction, but rpm doesn't see that.
> 
> Can't say offhand what, if anything, we can actually do about this (short of
> relaxing the %pretrans rule to cover more or less any conflict, which
> doesn't seem healthy) but at least the simple reproducer makes the case
> comprehensible.

Well, the main problem here seems to be that RPM thinks it's removing /etc/shells when it's actually just removing the symlink to /etc/shells. That suggests to me that somewhere in the check, it's following the symlink instead of treating it as a file.

Comment 4 Panu Matilainen 2020-05-25 07:39:45 UTC
Yes, like I said it's the symlink to /etc.

Rpm simply treats a directory and a symlink to one as equal and doesn't generally consider whether it should follow something or not. This is a fundamental part of making various things work, but also causes these kind of problems.

Comment 5 Troy Dawson 2020-07-09 21:53:57 UTC
I think if we add some comments in the %pretrans we see what's really going on.

## An actual update

# rpm -Uvh sgallagh-symlink-update2-0-4.el8.noarch.rpm
Verifying...                          ################################# [100%]
PRETRANS : sgallagh-symlink-update2-0-4.el8 
Preparing...                          ################################# [100%]
Updating / installing...
   1:sgallagh-symlink-update2-0-4.el8 ################################# [ 50%]
Cleaning up / removing...
   2:sgallagh-symlink-update2-0-3.el8 ################################# [100%]

## A test update

# rpm --test -Uvh sgallagh-symlink-update2-0-4.el8.noarch.rpm
Verifying...                          ################################# [100%]
Preparing...                          ################################# [100%]
        file /opt/bug/package/etc/shells from install of sgallagh-symlink-update2-0-4.el8.noarch conflicts with file from package setup-2.12.2-5.el8.noarch


As we see, during the --test, the %pretrans isn't happening.
And you don't want it to happen.
Running scripts during a --test is all sorts of bad.  It is no longer a test, it's the real thing, without the files being moved.

So, the whole %pretrans argument, if rpm is doing it correctly, seems to go out the window.
rpm is running %pretrans correctly.
But, during --test (which dnf calls) it isn't running %pretrans, which it shouldn't.


So, how to fix the problem.
A) have rpm treat symlinks to directories different.  But according to comment #4 that breaks other things.
B) don't have dnf call --test before it does updates.  That also leads to problems.  I remember the days of half done updates.
C) Something else

I'll be honest, I don't know what C is yet.  But it sure looks like that's what we need.

Comment 6 Troy Dawson 2020-07-10 19:42:01 UTC
A promising work around is changing the link to point to a different directory.  That directory can be part of the rpm.
Using the example above, we start with 

#sgallagh-symlink-update3-0-1
/opt/bug/package/file1
/opt/bug/package/etc -> /etc

And we can safely update to the following

#sgallagh-symlink-update3-0-1
/opt/bug/package/file1
/opt/bug/package/etc -> /opt/bug/package/bundle-etc
/opt/bug/package/bundle-etc/
/opt/bug/package/bundle-etc/shells

That works for rpm -Uvh, rpm --test -Uvh, and dnf install or update

Comment 7 Troy Dawson 2020-07-10 19:43:09 UTC
Created attachment 1700647 [details]
Specfile to show example, that says when pretrans is run

Comment 8 Troy Dawson 2020-07-10 19:43:50 UTC
Created attachment 1700648 [details]
Specfile that shows possible work around

Comment 9 Fedora Program Management 2021-04-29 16:53:42 UTC
This message is a reminder that Fedora 32 is nearing its end of life.
Fedora will stop maintaining and issuing updates for Fedora 32 on 2021-05-25.
It is Fedora's policy to close all bug reports from releases that are no longer
maintained. At that time this bug will be closed as EOL if it remains open with a
Fedora 'version' of '32'.

Package Maintainer: If you wish for this bug to remain open because you
plan to fix it in a currently maintained version, simply change the 'version' 
to a later Fedora version.

Thank you for reporting this issue and we are sorry that we were not 
able to fix it before Fedora 32 is end of life. If you would still like 
to see this bug fixed and are able to reproduce it against a later version 
of Fedora, you are encouraged  change the 'version' to a later Fedora 
version prior this bug is closed as described in the policy above.

Although we aim to fix as many bugs as possible during every release's 
lifetime, sometimes those efforts are overtaken by events. Often a 
more recent Fedora release includes newer upstream software that fixes 
bugs or makes them obsolete.

Comment 10 Ben Cotton 2021-05-25 17:20:56 UTC
Fedora 32 changed to end-of-life (EOL) status on 2021-05-25. Fedora 32 is
no longer maintained, which means that it will not receive any further
security or bug fix updates. As a result we are closing this bug.

If you can reproduce this bug against a currently maintained version of
Fedora please feel free to reopen this bug against that version. If you
are unable to reopen this bug, please file a new report against the
current release. If you experience problems, please add a comment to this
bug.

Thank you for reporting this bug and we are sorry it could not be fixed.

Comment 11 Ben Cotton 2021-08-10 13:39:53 UTC
This bug appears to have been reported against 'rawhide' during the Fedora 35 development cycle.
Changing version to 35.

Comment 12 Remi Collet 2021-12-10 10:26:12 UTC
*** Bug 2031013 has been marked as a duplicate of this bug. ***


Note You need to log in before you can comment on or make changes to this bug.