Bug 2063910
Summary: | LEAPP upgrade fails with [Errno 2] No such file or directory: '/var/lib/leapp/scratch/mounts/root_/system_overlay/etc/pki/pulp/content/pulp-global-repo.ca' | ||
---|---|---|---|
Product: | Red Hat Satellite | Reporter: | Lukas Pramuk <lpramuk> |
Component: | Satellite Maintain | Assignee: | satellite6-bugs <satellite6-bugs> |
Status: | CLOSED ERRATA | QA Contact: | Lukas Pramuk <lpramuk> |
Severity: | high | Docs Contact: | |
Priority: | unspecified | ||
Version: | 6.11.0 | CC: | apatel, aupadhye, egolov, ehelms, gtalreja, kgaikwad |
Target Milestone: | 6.11.0 | Keywords: | Triaged, Upgrades |
Target Release: | Unused | ||
Hardware: | Unspecified | ||
OS: | Unspecified | ||
Whiteboard: | |||
Fixed In Version: | rubygem-foreman_maintain-1.0.5 | Doc Type: | If docs needed, set a value |
Doc Text: | Story Points: | --- | |
Clone Of: | Environment: | ||
Last Closed: | 2022-07-05 14:34:27 UTC | Type: | Bug |
Regression: | --- | Mount Type: | --- |
Documentation: | --- | CRM: | |
Verified Versions: | Category: | --- | |
oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |
Cloudforms Team: | --- | Target Upstream Version: | |
Embargoed: | |||
Bug Depends On: | 2054182 | ||
Bug Blocks: |
Description
Lukas Pramuk
2022-03-14 15:50:00 UTC
>>> source tree # ll /var/lib/leapp/scratch/ ls: cannot access /var/lib/leapp/scratch/: No such file or directory >>> target tree # ll /var/lib/leapp/el8userspace/etc/pki/pulp total 28 drwxr-xr-x. 2 root root 34 Oct 1 11:09 content drwxr-x---. 2 root root 38 Nov 3 2019 qpid -rw-r--r--. 1 root root 5632 Dec 1 2016 server_ca.crt -rw-r--r--. 1 root root 5775 Dec 1 2016 ssl_apache.crt -r--------. 1 root root 1679 Dec 1 2016 ssl_apache.key -rw-r--r--. 1 root root 5632 Dec 1 2016 ssl_ca.crt # ll /var/lib/leapp/el8userspace/etc/pki/pulp/content/ total 0 -rw-r--r--. 1 root root 0 Dec 12 22:00 pulp-protected-repos While this is certainly the wrong leapp version (it's not the one containing the Satellite patches), can you please show the output of `ls -lahR /etc/pki/pulp/` and/or give me SSH access to the machine? Okay, so the problem is that the global repo ca file is a dangling symlink: [root@dhcp-2-184 ~]# ls -alh /etc/pki/pulp/content/pulp-global-repo.ca lrwxrwxrwx. 1 root root 20 Oct 29 2020 /etc/pki/pulp/content/pulp-global-repo.ca -> /etc/pki/pulp/ca.crt [root@dhcp-2-184 ~]# ls -alh /etc/pki/pulp/ca.crt ls: cannot access /etc/pki/pulp/ca.crt: No such file or directory And LEAPP tries to copy all of /etc/pki (and /etc/rhsm) from the EL7 system to the EL8 target: https://github.com/oamg/leapp-repository/blob/a098738a2d248d78a3c07b6cdf5317a4d1ffbb54/repos/system_upgrade/common/actors/targetuserspacecreator/libraries/userspacegen.py#L231-L232 It does so by using shutil.copytree: https://github.com/oamg/leapp-repository/blob/a098738a2d248d78a3c07b6cdf5317a4d1ffbb54/repos/system_upgrade/common/libraries/mounting.py#L213-L221 And shutil.copytree doesn't like to copy dangling symlinks (TIL!) by default: shutil.copytree(src, dst, symlinks=False, ignore=None) If symlinks is true, symbolic links in the source tree are represented as symbolic links in the new tree, but the metadata of the original links is NOT copied; if false or omitted, the contents and metadata of the linked files are copied to the new tree. https://docs.python.org/2.7/library/shutil.html#shutil.copytree (same-ish also in Python 3: https://docs.python.org/3/library/shutil.html#shutil.copytree). We can probably add a cleanup step to the installer/maintain before the upgrade, but I'd also argue LEAPP should catch this error better (and will talk to the LEAPP team). Ad cleanup, we can probably add /etc/pki/pulp to the pulp_data_dirs of the "foreman-maintain content remove-pulp2" command: https://github.com/theforeman/foreman_maintain/blob/dfd7f911363f51b112163dfeab0e74d1cc5abfaf/definitions/procedures/pulp/remove.rb#L18-L27 This LEAPP failure happens only on setups upgraded since 6.9 (and older) for both Satellite and Capsule Moved the leapp part of this to https://github.com/oamg/leapp-repository/issues/859 Making this now a FM BZ to clean up that path during the "remove pulp2" procedure Created redmine issue https://projects.theforeman.org/issues/34618 from this bug Adding a "blocked by" on BZ#2054182 as without that the cleanup won't happen during an upgrade of a capsule. VERIFIED. @Satellite 6.11.0 Snap16 rubygem-foreman_maintain-1.0.7-1.el7sat.noarch by the following manual reproducer: 1) Have a Sat 6.9.9 and Capsule 6.9.9 associated 2) Upgrade both Satellite and Capsule to 6.10.4 (symlink /etc/pki/pulp/content/pulp-global-repo.ca -> /etc/pki/pulp/ca.crt is present) 3) Upgrade both Satellite and Capsule to 6.11.0 and check for broken symlink # ls -l /etc/pki/pulp/content/pulp-global-repo.ca ls: cannot access /etc/pki/pulp/content/pulp-global-repo.ca: No such file or directory >>> automatic pulp2 removal removes also the broken symlink >>> LEAPP upgrade is going to be successful as there is no pulp2 broken symlink Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory (Moderate: Satellite 6.11 Release), and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHSA-2022:5498 |