Bug 1676685
| Summary: | yum makes a wrong check of space available on /opt, when /opt is automounted; should not depend on /opt/rh | ||
|---|---|---|---|
| Product: | Red Hat Enterprise Linux 7 | Reporter: | Norman Gray <gray> |
| Component: | rpm | Assignee: | Pavlina Moravcova Varekova <pmoravco> |
| Status: | CLOSED NEXTRELEASE | QA Contact: | BaseOS QE Security Team <qe-baseos-security> |
| Severity: | medium | Docs Contact: | |
| Priority: | medium | ||
| Version: | 7.6 | CC: | dmach, gray, james.antill, pmatilai, pmoravco |
| Target Milestone: | rc | Keywords: | Triaged |
| Target Release: | --- | ||
| Hardware: | All | ||
| OS: | Linux | ||
| Whiteboard: | |||
| Fixed In Version: | Doc Type: | If docs needed, set a value | |
| Doc Text: | Story Points: | --- | |
| Clone Of: | Environment: | ||
| Last Closed: | 2019-04-04 18:15:04 UTC | Type: | Bug |
| Regression: | --- | Mount Type: | --- |
| Documentation: | --- | CRM: | |
| Verified Versions: | Category: | --- | |
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |
| Cloudforms Team: | --- | Target Upstream Version: | |
| Embargoed: | |||
Disk space calculation is done by rpm, not yum -> reassigning. Disk space calculations on NFS are notoriously off in any case (see bug 847960) but that's not the issue here yet. If df /opt reports 0 blocks available then that's what rpm will see and refuse to install. I guess /opt isn't actually mounted at that point, ie the automount doesn't trigger by df and equivalent? To be precise, `/opt` is mounted at the time the check is done, since it's the automounter process that's mounted there, but no, `/opt/rh` isn't mounted until specifically that directory is examined.
Invoking `df` on that directory does trigger the mount, though:
% df /opt/rh
Filesystem 1K-blocks Used Available Use% Mounted on
/dev/mapper/centos-home 5799155712 58369960 5740785752 2% /opt/rh
It might therefore be that a test of specifically `/opt/rh` would fix the immediate problem.
I'll highlight the enhancement request, though. The check of `/opt/rh` is done even when `yum` is about to update Core, when nothing (always?, typically?) is going to be written to `/opt/rh`. Thus in the default configuration, the absence of `/opt/rh` will stop `yum update`.
(thanks for the reassignment to rpm from yum; I've continued to refer to yum above since that's the component I'm handling here)
Please can you sent the relevant lines from fstab file? The filesystem /opt is automounted, so there is no relevant line in /etc/fstab I tested such scenario and rpm instalation was without any problem. Please could you write the relevant line from the mapping file? Hello Pavlina.
The relevant entry in the `auto.opt` automount map is:
rh -defaults,ro optmachine.example.org:/local/opt/by-os/$OSNAME/rh
and the entry in `auto.master` is
/opt auto.opt ro,intr,hard,nosuid,nodev,root_squash
Both of these are obtained from LDAP rather than NIS (though I'd be surprised if that were relevant). The same thing happens when `optmachine.example.org` is the machine mounting this, and when it is not.
Note that if `diskspacecheck=0` is in `/etc/yum.conf`, then the installation will proceed without a problem.
This is using (`automount --version`) "Linux automount version 5.0.7-99.el7", from package autofs-5.0.7-99.el7.x86_64
Hmm.... What else could be relevant?
I have:
client:etc# df /opt
Filesystem 1K-blocks Used Available Use% Mounted on
auto.opt 0 0 0 - /opt
client:etc# df /opt/rh
Filesystem 1K-blocks Used Available Use% Mounted on
optmachine.example.org:/local/opt/by-os/centos7/rh 5799155712 70018048 5729137664 2% /opt/rh
The `/etc/nsswitch.conf` has:
automount: files sss
`/etc/auto.master` includes:
+auto.master
(I notice that on one machine I have the default other entries in this file -- eg `/net` -- commented out, and on another they're uncommented as by default; I don't think this makes a difference).
I don't think I do anything beyond these configurations, but I'd be happy to report any other configuration details you can think of, if you think they might be relevant.
I've just reviewed the original problem report, and I should note that the setup I've described in the last comment is slightly different from that in the original report. The initial report describes an NFS mount of localhost:/local/rh whereas the last comment describes a mount of optmachine.example.org (both when this is the same as localhost and when it is different). The behaviour appears to be identical in each case, so I don't think this is relevant, but I wanted to acknowledge the marginal drift in the config being described. Also, as reported in the comment above, the _current_ export of `/opt` to this machine is read-only (which of course won't work when actually trying to install an SCL package like rh-python36), but this is a later adjustment -- sorry. When I reported this I did have the filesystem mounted read-write. Also, I see that I mentioned in the original CentOS report (https://bugs.centos.org/view.php?id=10301) but apparently changed the text when I copied it here, that this behaviour also happens when doing `yum update`, ie, when one would _not_ expect anything to be installed in `/opt/rh`. The process of `yum update`, without `diskspacecheck=0`, appears to include an attempted install, or at leats a check of space, on `/opt/rh`. Or at least an unwarranted assumption that `/opt/rh` exists and is writable. Sorry for these slight drifts. If you want me to produce an absolutely from-scratch reproduction of the problem, I can surely do so, but I hope the notes here will make the problem reproducible, or otherwise lead someone to the underlying issue. Thanks for your continued attention. Thank you. With the settings from Comment 9 ("auto.opt" + "auto.master") I reproduced the reported problem. I tested result of command: rpm -U scl-utils-20130529-19.el7.x86_64 on three operating systems: CentOS-7.6 (rpm-4.11.3-35.el7.x86_64), RHEL 7.6 (rpm-4.11.3-35.el7.x86_64.rpm) and Fedora 27 (rpm-4.14.2.1-1.fc27.x86_64). In all cases I used the automount configuration from #c9. When using rpm-4.11.3 the upgrade was not successful and it emits: error: unpacking of archive failed on file /opt/rh: cpio: lsetfilecon failed - Read-only file system In case of rpm-4.14.2 the upgrade was done without any problem. When using rpm-4.11.3 with "rw" permissions in `auto.opt`: the upgrade was done without any problem. The similar error like the one in #c1 occurs only if the updated rpm includes an additional file in directory /opt/rh and the directory (auto.opt) was without "ro" permissions. In that case it writes: installing package dtest-2.0-1.noarch needs 16KB on the /opt/rh file system (I modified a package for this test) So it looks that in CentOS-8.0 the problem will be solved and for CentOS-7.6 I was not able to reproduce the exact problem. Because of the stage of CentOS-7.6 I will close the bug as NEXTRELEASE. Thanks for investigating. I'll keep an eye out for this when I first try CentOS 8. If it's still present then, I presume that reopening this would be the correct thing to do? Yes, in that case it would be correct. Thank you for your very fast and precise replies on needinfo. |
Description of problem: Starting point: fresh install of CentOS 7.6-1810, plus configuration to make the machine an NFS client. Machine configured so that `/opt` is automounted, and `/opt/rh` maps to `localhost:/local/rh`, which does exist on the *local* machine. This is mounted writable (ie `# echo foo >/opt/rh/stamp.txt`) A `yum install rh-python36` fails with the error Transaction check error: installing package scl-utils-20130529-19.el7.x86_64 needs 4KB on the /opt filesystem I note that the system reports `/opt` as having zero size available: # df /opt Filesystem 1K-blocks Used Available Use% Mounted on auto.opt 0 0 0 - /opt (the /local filesystem has plenty of space) I speculate therefore that at some point when checking the presence of, and space available under, `/opt/rh`, yum checks the space available on `/opt` (or something equivalent to this), and therefore erroneously gets a wrong size, causing it to fail. I then set `diskspacecheck=0` in `yum.conf`, and the installation proceeds without error. Note the *same* 'transaction check error' failure happens with `yum update` unless `diskspacecheck=0`, even though in that case nothing is actually written to `/opt/rh`. So: * bug: when `/opt` is automouted, yum fails to detect that `/opt/rh` exists and is writable * enhancement: yum should not, I think, demand that `/opt/rh` exists, in the case where it is not going to install anything there. That is, given that I do not plan to install any Software Collections on a client machine, I should be able to mount `/opt` read-only, without `/opt/rh`. (I'm happy to make this a separate enhancement request if you feel that would be neater). Version-Release number of selected component (if applicable): $ yum --version 3.4.3 $ cat /etc/redhat-release CentOS Linux release 7.6.1810 (Core) Steps to Reproduce: 1. See above Actual results: Software cannot be installed when `/opt/rh` is automounted. Expected results: Software should be installable in this case. Additional info: 1. It would be desirable to be able to choose to relocate `/opt` at installation time (as opposed to when one is constructing the metapackage), but I appreciate this may be infeasible, and so I have not included this as an enhancement request. 2. The notes at <https://access.redhat.com/documentation/en-US/Red_Hat_Software_Collections/2/html/Packaging_Guide/chap-Advanced_Topics.html#sect-Using_Software_Collections_over_NFS> remark that 'This enables you to mount the /opt file system hierarchy over NFS as read-only', so that this is at least intended as a supported configuration (though I can see that might be approaching the problem from the slightly different point of view of doing the yum install to /opt on the NFS server). 3. The FHS description at <http://refspecs.linuxfoundation.org/FHS_3.0/fhs-3.0.html#optAddonApplicationSoftwarePackages> is not particularly clear, but the remark 'Package files that are variable (change in normal operation) must be installed in /var/opt.' seems to imply that /opt is mountable read-only. The FHS doesn't say anything about NFS in the context of /opt The remark 'Distributions may install and otherwise manage software in /opt under an appropriately registered subdirectory' does not, I think, imply that /opt should be writable other than when deliberately installing things in /opt. 4. I mistakenly posted this in the CentOS bugparade first.