Note: This bug is displayed in read-only format because the product is no longer active in Red Hat Bugzilla.
RHEL Engineering is moving the tracking of its product development work on RHEL 6 through RHEL 9 to Red Hat Jira (issues.redhat.com). If you're a Red Hat customer, please continue to file support cases via the Red Hat customer portal. If you're not, please head to the "RHEL project" in Red Hat Jira and file new tickets here. Individual Bugzilla bugs in the statuses "NEW", "ASSIGNED", and "POST" are being migrated throughout September 2023. Bugs of Red Hat partners with an assigned Engineering Partner Manager (EPM) are migrated in late September as per pre-agreed dates. Bugs against components "kernel", "kernel-rt", and "kpatch" are only migrated if still in "NEW" or "ASSIGNED". If you cannot log in to RH Jira, please consult article #7032570. That failing, please send an e-mail to the RH Jira admins at rh-issues@redhat.com to troubleshoot your issue as a user management inquiry. The email creates a ServiceNow ticket with Red Hat. Individual Bugzilla bugs that are migrated will be moved to status "CLOSED", resolution "MIGRATED", and set with "MigratedToJIRA" in "Keywords". The link to the successor Jira issue will be found under "Links", have a little "two-footprint" icon next to it, and direct you to the "RHEL project" in Red Hat Jira (issue links are of type "https://issues.redhat.com/browse/RHEL-XXXX", where "X" is a digit). This same link will be available in a blue banner at the top of the page informing you that that bug has been migrated.

Bug 1822951

Summary: Failure installing a base system in an installroot
Product: Red Hat Enterprise Linux 8 Reporter: Renaud Métrich <rmetrich>
Component: dnfAssignee: Packaging Maintenance Team <packaging-team-maint>
Status: CLOSED NOTABUG QA Contact: swm-qe
Severity: low Docs Contact:
Priority: low    
Version: 8.1CC: ahajkova, fweimer, jakub, james.antill, jrohel, kwalker, mblaha, ohudlick, pmatilai
Target Milestone: rcKeywords: Triaged
Target Release: 8.0Flags: pm-rhel: mirror+
Hardware: All   
OS: Linux   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2020-04-15 07:38:44 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Renaud Métrich 2020-04-10 15:49:12 UTC
Description of problem:

Trying to install a base system in an installroot using "--installroot <path>", the installation fails installing "kmod" and "filesystem" packages:

-------- 8< ---------------- 8< ---------------- 8< ---------------- 8< --------
Error unpacking rpm package filesystem-3.8-2.el8.x86_64
  Installing       : basesystem-11-5.el8.noarch                                                   12/158 
error: unpacking of archive failed on file /lib64: cpio: File from package already exists as a directory in system
error: filesystem-3.8-2.el8.x86_64: install failed

...

Error unpacking rpm package kmod-25-13.el8.x86_64
  Installing       : libfdisk-2.32.1-17.el8.x86_64                                               122/158 
error: unpacking of archive failed on file /usr/share/bash-completion/completions/kmod;5e90915f: cpio: open
error: kmod-25-13.el8.x86_64: install failed

...

Failed:
  filesystem-3.8-2.el8.x86_64                            kmod-25-13.el8.x86_64                           

Error: Transaction failed
-------- 8< ---------------- 8< ---------------- 8< ---------------- 8< --------


Version-Release number of selected component (if applicable):

libdnf-0.35.1-9.el8_1.x86_64


How reproducible:

Always


Steps to Reproduce:
1. As root user, download the packages

  # dnf --installroot=$PWD/FOO --releasever=/ install python3-libs --downloadonly
  # echo $$
  1442

2. As root user, from another terminal, strace the terminal that will execute dnf command

  # strace -fttTvyy -o dnf_unshare.strace -s 1024 -p 1442

3. As root user, install the packages

  # dnf --installroot=$PWD/FOO --releasever=/ install python3-libs --noplugins --setopt=tsflags=noscripts


Actual results:

See description.


Expected results:

No error.


Additional info:

The filesystem package fails because /lib64 already exists at the time filesystem is installed:
-------- 8< ---------------- 8< ---------------- 8< ---------------- 8< --------
2967  17:34:43.928484 rename("/lib64;5e90920d", "/lib64") = -1 EISDIR (Is a directory) <0.000007>
-------- 8< ---------------- 8< ---------------- 8< ---------------- 8< --------

The kmod package fails because /usr/share/bash-completion/completions/kmod;5e90920d doesn't exist for some reason:
-------- 8< ---------------- 8< ---------------- 8< ---------------- 8< --------
2967  17:35:11.507937 openat(AT_FDCWD, "/usr/share/bash-completion/completions/kmod;5e90920d", O_WRONLY|O_CREAT|O_EXCL|O_TRUNC, 0666) = -1 ENOENT (No such file or directory) <0.000015>
2967  17:35:11.507980 umask(022)        = 0577 <0.000005>
2967  17:35:11.508001 lstat("/usr/share/bash-completion/completions/kmod;5e90920d", 0x7fff8b53d100) = -1 ENOENT (No such file or directory) <0.000006>
2967  17:35:11.508028 unlink("/usr/share/bash-completion/completions/kmod;5e90920d") = -1 ENOENT (No such file or directory) <0.000006>
-------- 8< ---------------- 8< ---------------- 8< ---------------- 8< --------

Comment 1 Marek Blaha 2020-04-14 13:43:28 UTC
There is probably nothing dnf can do with this. This is my understanding of what is going on:

1. Both `libgcc` and `filesystem` packages are going to be installed. As they do not depend each on another rpm can choose in which order they will got installed. Here the `libgcc` is installed before the `filesystem`.

2. `libgcc` installs files into `/lib64/` directory:
# rpm -ql libgcc
/lib64/libgcc_s-8-20190507.so.1
/lib64/libgcc_s.so.1
/usr/lib/.build-id
/usr/lib/.build-id/00
/usr/lib/.build-id/00/3d5b796089f1092af9d12350a2e04a6507d730
/usr/share/licenses/libgcc
/usr/share/licenses/libgcc/COPYING
/usr/share/licenses/libgcc/COPYING.LIB
/usr/share/licenses/libgcc/COPYING.RUNTIME
/usr/share/licenses/libgcc/COPYING3
/usr/share/licenses/libgcc/COPYING3.LIB

So after `libgcc` installation the directory <installroot>/lib64 exists.

3. `filesystem` package tries to install symlink /lib64 -> /usr/lib64 and that is where it fails. See `ln -snf usr/%{_lib} %{_lib}` line in filesystem.spec

This error is reproducible even without dnf, using plain rpm (the parameters of the second rpm run are quite huge, but only filesystem.rpm is essential for our example, the rest are filesystem's dependencies):
# mkdir IR
# rpm --root=${PWD}/IR -i libgcc-8.3.1-4.5.el8.x86_64.rpm
# rpm --root=${PWD}/IR -i filesystem-3.8-2.el8.x86_64.rpm bash-4.4.19-10.el8.x86_64.rpm  glibc-2.28-72.el8_1.1.x86_64.rpm glibc-minimal-langpack-2.28-72.el8_1.1.x86_64.rpm basesystem-11-5.el8.noarch.rpm glibc-common-2.28-72.el8_1.1.x86_64.rpm tzdata-2019c-1.el8.noarch.rpm ncurses-libs-6.1-7.20180224.el8.x86_64.rpm ncurses-base-6.1-7.20180224.el8.noarch.rpm libselinux-2.9-2.1.el8.x86_64.rpm pcre2-10.32-1.el8.x86_64.rpm libsepol-2.9-1.el8.x86_64.rpm setup-2.12.2-2.el8_1.1.noarch.rpm basesystem-11-5.el8.noarch.rpm redhat-release-8.1-3.3.el8.x86_64.rpm 
error: unpacking of archive failed on file /lib64: cpio: File from package already exists as a directory in system
error: filesystem-3.8-2.el8.x86_64: install failed

The issue could be work-arounded by creating lib64 -> usr/lib64 symlink inside the installroot directory before actual installation:

# mkdir -p $PWD/FOO/usr/lib64
# ln -s usr/lib64 $PWD/FOO/lib64
# dnf --installroot=$PWD/FOO --releasever=/ install python3-libs --noplugins --setopt=tsflags=noscripts
-------- 8< ----------
Complete!

I'm reassigning this issue to gcc component for further investigation / solution.

Comment 2 Renaud Métrich 2020-04-14 13:57:01 UTC
Hi Marek,

I understand the filesystem issue is a kind of chicken and egg issue.

But not the kmod package issue.

Comment 3 Marek Blaha 2020-04-14 14:05:10 UTC
Sorry, my bad. I missed that `filesystem` plugin has pre-transaction scriptlet, which is supposed to create this lib64->usr/lib64 symlink.
By adding `--setopt=tsflags=noscripts` switch to dnf install command you disable running the scriptlets and the symlink is not created.
Not sure about kmod issue, but it disappeared after fixing the symlink issue.
I'm moving the bug back to dnf component.

Comment 4 Panu Matilainen 2020-04-15 06:53:28 UTC
> 3. As root user, install the packages
>
>  # dnf --installroot=$PWD/FOO --releasever=/ install python3-libs --noplugins --setopt=tsflags=noscripts

--setopt=tsflags=noscripts is the problem, you can't expect things to work correctly when that's used.

Optimally filesystem package should always get installed first but that gets... complicated, which is why it has a %pretrans script to ensure the necessary directories and their symlinks get created before anything gets installed. By disabling scripts you prevent that from working.

NOTABUG as far as I can tell.

Comment 5 Renaud Métrich 2020-04-15 07:38:44 UTC
OK I understand now. Sorry for losing your time.
Thanks.