From https://bugzilla.redhat.com/show_bug.cgi?id=1548403#c33 FWIW, /dev is also a mount point that will have the same problems as /proc and /sys in a toolbox container for example: $ toolbox create -r 34 [...] $ toolbox enter -r 34 $ mount | grep "/dev " devtmpfs on /dev type devtmpfs (rw,nosuid,noexec,seclabel,size=12226400k,nr_inodes=3056600,mode=755,inode64) devtmpfs on /home/hadess/.local/share/containers/storage/overlay/90c78f938c7caa3cd027e22f6652e05b413c39376fa0fc5b4485caf121ae1477/merged/dev type devtmpfs (rw,nosuid,noexec,seclabel,size=12226400k,nr_inodes=3056600,mode=755,inode64) $ $ rpm -V filesystem .M....... / .....UG.. /dev .....UG.. /media .....UG.. /mnt .....UG.. /proc .....UG.. /sys .....UG.. /tmp missing /usr/share/locale/en missing /usr/share/locale/en/LC_MESSAGES missing /usr/share/locale/en@arabic missing /usr/share/locale/en@arabic/LC_MESSAGES missing /usr/share/locale/en@boldquot missing /usr/share/locale/en@boldquot/LC_MESSAGES missing /usr/share/locale/en@cyrillic missing /usr/share/locale/en@cyrillic/LC_MESSAGES missing /usr/share/locale/en@greek missing /usr/share/locale/en@greek/LC_MESSAGES missing /usr/share/locale/en@hebrew missing /usr/share/locale/en@hebrew/LC_MESSAGES missing /usr/share/locale/en@piglatin missing /usr/share/locale/en@piglatin/LC_MESSAGES missing /usr/share/locale/en@quot missing /usr/share/locale/en@quot/LC_MESSAGES missing /usr/share/locale/en@shaw missing /usr/share/locale/en@shaw/LC_MESSAGES missing /var/cache/bpf $ sudo dnf update filesystem Fedora 34 openh264 (From Cisco) - x86_64 4.5 kB/s | 2.5 kB 00:00 Fedora - Modular Rawhide - Developmental packages for the next Fedora release 3.3 MB/s | 5.3 MB 00:01 Fedora - Rawhide - Developmental packages for the next Fedora release 16 MB/s | 73 MB 00:04 Dependencies resolved. =================================================================================================================================================================================================================== Package Architecture Version Repository Size =================================================================================================================================================================================================================== Upgrading: filesystem x86_64 3.14-4.fc34 rawhide 1.1 M Transaction Summary =================================================================================================================================================================================================================== Upgrade 1 Package Total download size: 1.1 M Is this ok [y/N]: y Downloading Packages: filesystem-3.14-4.fc34.x86_64.rpm 4.6 MB/s | 1.1 MB 00:00 ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Total 3.1 MB/s | 1.1 MB 00:00 Running transaction check Transaction check succeeded. Running transaction test Transaction test succeeded. Running transaction Running scriptlet: filesystem-3.14-4.fc34.x86_64 1/1 Preparing : 1/1 Upgrading : filesystem-3.14-4.fc34.x86_64 1/2 Error unpacking rpm package filesystem-3.14-4.fc34.x86_64 Verifying : filesystem-3.14-4.fc34.x86_64 1/2 Verifying : filesystem-3.14-3.fc33.x86_64 2/2 Failed: filesystem-3.14-3.fc33.x86_64 filesystem-3.14-4.fc34.x86_64 Error: Transaction failed
Actually, all those are mountpoints in toolbox: mount | grep -e ' /media ' -e ' /mnt ' -e ' /dev ' -e ' /tmp ' -e ' /sys ' -e ' /proc ' And all of them have non-standard ownership: $ rpm -V filesystem | grep UG .....UG.. /dev .....UG.. /media .....UG.. /mnt .....UG.. /proc .....UG.. /sys .....UG.. /tmp Are you sure only /dev causes problems? FTR, normal podman run doesn't suffer from this issue (only /proc and /sys needs to be handled, and /dev is mountpoint too, but it has standard ownership so rpm doesn't complain).
(In reply to Pavel Raiskup from comment #1) <snip> > Are you sure only /dev causes problems? I don't, but that's the only error I saw, so that's the only one I reported. I didn't investigate further as I already spent a day and a half fighting other rawhide regressions in containers :)
This is a duplicate of 1830830 It doesn't just affect Toolbox containers, but Podman containers in general.
*** This bug has been marked as a duplicate of bug 1830830 ***
> It doesn't just affect Toolbox containers, but Podman containers in general. Not on my box. We fixed filesystem package on F33+ so rpm doesn't complain in normal rootless podman container.
Ok, from 1830830 it looks like toolbox people are working on the work-around: https://github.com/containers/toolbox/pull/640 I commented there the bug in the PR code, but I don't think we can generally expect similar problems elsewhere (normal podman runtime) as bind-mounting down the filesystem-owned files isn't common. Switching against toolbox.
Umm... why did you un-duplicate the bug? There are too many of these bugs about the same issue flying around. It would be nice to consolidate them a bit. *** This bug has been marked as a duplicate of bug 1830830 ***
Because I disagree this is a duplicate. But it should be obvious from my comments? If you use container F33+, this only happens in toolbox, not buildah.
> this only happens in toolbox, not buildah. I mean, that duplicate bug 1830830 is assigned to buildah.
It's not about finding someone or some package (toolbox or buildah or whatever) to blame. We need to figure out a solution, whatever that might be and ship it to our users. For that it's really useful if the discussion stays consolidated in one place instead of getting fragmented across multiple bugs.
Really, I'm not trying to blame anything/one. I'm sad you see me like that. I really don't see Buildah affected by _this_ bug. We invested some time to solve one buildah+filesytem in bug 1548403. Perhaps that solved the bug 1830830 as well? ATM there's a problem with /dev, /media, ... (not /proc or /sys). And this is not a problem for Buildah because it _doesn't mount those new paths_ to actually trigger the problem. No problem to track _this_ in bug 1830830, but it looks like something different when there's Component=buildah field. That's why I actually filled this bug. Looks like my comments in upstream PR#640 are not applied. Please consider them.
Is there some reason this would be a problem again upgrading a F35 toolbox container to F36 (on a silverblue F36 host)? The workaround doesn't seem to make a difference now
(In reply to kxra from comment #12) > Is there some reason this would be a problem again upgrading a F35 toolbox > container to F36 (on a silverblue F36 host)? The workaround doesn't seem to > make a difference now Dunno about the age of your f35 container, but I can't reproduce that in a fresh f35 updated toolbox if I dnf upgrade it to f36, at least.
(In reply to kxra from comment #12) > Is there some reason this would be a problem again upgrading a F35 toolbox > container to F36 (on a silverblue F36 host)? The workaround doesn't seem to > make a difference now What was the exact failure that you saw? Do you have the /usr/lib/rpm/macros.d/macros.toolbox file inside the container?
Yes, I think there were a combination of factors, as I was able to resolve it by: - dnf autoremove - dnf list --installed | rg fc35 - dnf remove fc35 packages - remove the /etc/rpm/macros.dist file https://ask.fedoraproject.org/t/how-to-upgrade-fedora-version-inside-a-toolbox-container/10463/8
I see. I am glad you managed to find a way forward. :)