Bug 1797163

Summary: Rebooting and shutting down Rawhide Plasma spin errors like "/shutdown: line 115: reboot: command not found" with cpio-2.13-3.fc32
Product: [Fedora] Fedora Reporter: Matt Fagnani <matt.fagnani>
Component: cpioAssignee: Petr Kubat <pkubat>
Status: CLOSED CURRENTRELEASE QA Contact: Fedora Extras Quality Assurance <extras-qa>
Severity: unspecified Docs Contact:
Priority: unspecified    
Version: rawhideCC: awilliam, kdudka, nixuser, ovasik, panovotn, pkubat, praiskup, robatino, sergey.avseyev, tomm.momi
Target Milestone: ---   
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard: openqa
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2020-02-05 19:52:57 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 1705303    
Attachments:
Description Flags
Screen shot of shutdown failure none

Description Matt Fagnani 2020-02-01 04:51:05 UTC 
Description of problem:

I ran sudo dnf upgrade --refresh in a Rawhide KDE Plasma spin installation on 2020-1-31 which updated about 130 rpms. When I rebooted the system after the update, an error occurred while the dracut shutdown initramfs was running its shutdown script.
/shutdown: line 115: reboot: command not found
dracut: Warning: reboot failed!
Dropping to debug shell

In the dracut shutdown shell the reboot and poweroff commands gave the error 
"sh: command not found". reboot and poweroff were links to bin/systemctl.
Running systemctl showed the error "Failed to connect to bus: connection refused". systemd wasn't running in the dracut shutdown shell.

Shutting down the system from Plasma led to the error 
/shutdown: line 115: poweroff: command not found
The other errors were the same when shutting down as rebooting.

I downgraded many packages from the update individually from koji and then rebooted. When I downgraded from cpio-2.13-3.fc32.x86_64 to cpio-2.12-12.fc31 and then rebooted, the system rebooted correctly without the errors above.
I upgraded to cpio-2.13-3.fc32.x86_64 and rebooted, and the dracut shutdown errors happened again. dracut-049-27.git20181204.fc31.1.x86_64 requires cpio.


Version-Release number of selected component (if applicable):
cpio-2.13-3.fc32.x86_64
dracut-049-27.git20181204.fc31.1.x86_64
systemd-244.1-2.fc32.x86_64

How reproducible:
Rebooting and shutting down Rawhide Plasma spin had errors like "/shutdown: line 115: reboot: command not found" each of more than 10 times with cpio-2.13-3.fc32. Those errors didn't happen with cpio-2.12-12.fc31.

Steps to Reproduce:
1. Boot Rawhide KDE Plasma spin fully updated to 2020-1-30
2. Log in to Plasma on Wayland from sddm
3. sudo dnf upgrade --refresh (in konsole) The update should include cpio-2.13-3.fc32.x86_64
4. Reboot or shut down the system

Actual results:
Rebooting and shutting down Rawhide Plasma spin had errors like "/shutdown: line 115: reboot: command not found" with cpio-2.13-3.fc32

Expected results:
The system would reboot or shut down normally.

Additional info:
The following openQA tests from 2020-1-31 show these rebooting errors https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org/thread/TT3SFJTE2YORCD64PBCWQ4ZZV3IDEVA7/

Test: x86_64 KDE-live-iso install_default_upload 
https://openqa.fedoraproject.org/tests/515184#step/_graphical_wait_login/4

Test: x86_64 KDE-live-iso install_default@uefi
https://openqa.fedoraproject.org/tests/515189#step/_graphical_wait_login/4

Test: x86_64 KDE-live-iso install_no_user 
https://openqa.fedoraproject.org/tests/515195#step/_graphical_wait_login/4
Comment 1 Matt Fagnani 2020-02-02 00:29:08 UTC 
This problem was reported to cpio upstream by Mike Gilbert at https://lists.gnu.org/archive/html/bug-cpio/2019-11/msg00013.html
cpio --no-absolute-filenames in 2.13 changed "./sbin/reboot -> ../bin/systemctl (symlink)" to "./sbin/reboot -> bin/systemctl (broken symlink)". Sergey Poznyakoff wrote "It is definitely a bug.  I'm working on a solution and will let you know when it is available.  Until then, the best course of action would be to revert 45b0ee2b407913c533f7ded8d6f8cbeec16ff6ca." at https://lists.gnu.org/archive/html/bug-cpio/2019-11/msg00016.html
Comment 2 Ian Laurie 2020-02-03 01:03:34 UTC 
Created attachment 1657229 [details]
Screen shot of shutdown failure
Comment 3 Ian Laurie 2020-02-03 01:06:49 UTC 
I have this issue on 3 virtual machines after updating very early February.  I then built a fresh VM with Fedora-Everything-netinst-x86_64-Rawhide-20200201.n.0.iso and it has the same problem.  I have attached a screen shot.  I think this is the same problem.
Comment 4 Ian Laurie 2020-02-03 03:10:15 UTC 
Matt's proposed solution of downgrading to cpio-2.12-12.fc31 from the F31 repo fixes the problem for me.
Comment 5 Adam Williamson 2020-02-04 17:48:34 UTC 
openQA is also hitting this; it's breaking all tests where the machine is explicitly shut down (so, ones where we upload the hard disk image). I'm also hitting it on my local Rawhide box when trying to shut down.

Proposing as a Beta blocker per Basic criterion "It must be possible to trigger a clean system shutdown using standard console commands." - https://fedoraproject.org/wiki/Basic_Release_Criteria#Shutdown
Comment 6 Petr Kubat 2020-02-05 06:56:27 UTC 
Hi all,

thanks for bringing this to our attention. The issue here is that 45b0ee2b407913c533f7ded8d6f8cbeec16ff6ca is the fix for CVE-2015-1197, so reverting it does not seem like the correct way to move forward... But as upstream has been silent on the issue since it was reported in November 2019, the CVE is low prio and the bug having impact on beta I am leaning toward doing the revert anyway. WDYT Pavel?
Comment 7 Pavel Raiskup 2020-02-05 07:40:33 UTC 
> I am leaning toward doing the revert anyway. WDYT Pavel?

+1, go for it
Comment 8 Petr Kubat 2020-02-05 10:21:31 UTC 
Prepared in: https://src.fedoraproject.org/rpms/cpio/pull-request/6
Comment 9 Pavel Raiskup 2020-02-05 19:52:57 UTC 
Cpio with reverted patch is built in Rawhide.