Bug 1644097

Summary: The host entered to emergency mode when rollback/reboot twice after upgrade twice
Product: [oVirt] ovirt-node Reporter: Huijuan Zhao <huzhao>
Component: Installation & UpdateAssignee: Yuval Turgeman <yturgema>
Status: CLOSED DUPLICATE QA Contact: Huijuan Zhao <huzhao>
Severity: high Docs Contact:
Priority: unspecified    
Version: 4.2CC: bugs, cshao, dougsland, huzhao, qiyuan, rbarry, sbonazzo, weiwang, yaniwang, ycui, yturgema
Target Milestone: ---Flags: cshao: testing_ack+
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2018-10-30 03:16:03 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: Node RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Attachments:
Description Flags
All logs from host(sosreport, all files in /var/log)
none
nodectl info none

Description Huijuan Zhao 2018-10-30 02:47:08 UTC
Created attachment 1498779 [details]
All logs from host(sosreport, all files in /var/log)

Description of problem:
RHVH entered to emergency mode when rollback/reboot twice after upgrade twice.

For example:
Upgrade from build1 to build2 successful, then upgrade from build2 to build3 successful. Rollback/reboot can enter to build2 successful, rollback again in build2, reboot host, then entered to emergency mode.

Version-Release number of selected component (if applicable):
Build1: redhat-virtualization-host-4.1-20180426.0
Build2: redhat-virtualization-host-4.2-20180828.2
Build3: redhat-virtualization-host-4.2-20181026.0


How reproducible:
100%

Steps to Reproduce:
1. Install redhat-virtualization-host-4.1-20180426.0
2. Upgrade rhvh from 4.1 to 4.2.6 redhat-virtualization-host-4.2-20180828.2
3. Reboot rhvh to 4.2.6, then upgrade again to 4.2.7 redhat-virtualization-host-4.2-20181026.0
4. Reboot rhvh, enter to 4.2.7 successful, run "#imgbase rollback"
5. Reboot rhvh, can enter to 4.2.6 successful, run "#imgbase rollback" in rhvh-4.2.6
6. Reboot rhvh

Actual results:
1. After step6, rhvh entered to emergency mode.

Expected results:
After step6, rhvh should enter to rhvh-4.2.7 successful.


Additional info:
No such issue if only upgrade once.

Comment 1 Huijuan Zhao 2018-10-30 03:16:03 UTC
Same with bug 1636028, so duplicate this bug.

*** This bug has been marked as a duplicate of bug 1636028 ***

Comment 2 Ryan Barry 2018-10-30 04:17:38 UTC
Is this a duplicate?

nodectl info would be helpful.

The oldest layer should be garbage collected, and nodectl should block rollbacks to a layer which doesn't exist

Comment 3 Huijuan Zhao 2018-10-30 05:26:11 UTC
(In reply to Ryan Barry from comment #2)
> Is this a duplicate?
> 
> nodectl info would be helpful.
> 
> The oldest layer should be garbage collected, and nodectl should block
> rollbacks to a layer which doesn't exist

Thanks Ryan for your quick response.

1. In the build 2, the nodectl info is as below:
# nodectl info
layers: 
  rhvh-4.2.6.0-0.20180828.0: 
    rhvh-4.2.6.0-0.20180828.0+1
  rhvh-4.2.7.3-0.20181026.0: 
    rhvh-4.2.7.3-0.20181026.0+1
bootloader: 
  default: rhvh-4.2.7.3-0.20181026.0+1
  entries: 
    rhvh-4.2.7.3-0.20181026.0+1: 
      index: 0
      title: rhvh-4.2.7.3-0.20181026.0
      kernel: /boot/rhvh-4.2.7.3-0.20181026.0+1/vmlinuz-3.10.0-957.el7.x86_64
      args: "ro crashkernel=auto rd.lvm.lv=rhvh_dhcp-11-33/rhvh-4.2.7.3-0.20181026.0+1 rd.lvm.lv=rhvh_dhcp-11-33/swap rhgb quiet LANG=en_US.UTF-8 img.bootid=rhvh-4.2.7.3-0.20181026.0+1"
      initrd: /boot/rhvh-4.2.7.3-0.20181026.0+1/initramfs-3.10.0-957.el7.x86_64.img
      root: /dev/rhvh_dhcp-11-33/rhvh-4.2.7.3-0.20181026.0+1
    rhvh-4.2.6.0-0.20180828.0+1: 
      index: 1
      title: rhvh-4.2.6.0-0.20180828.0
      kernel: /boot/rhvh-4.2.6.0-0.20180828.0+1/vmlinuz-3.10.0-862.11.6.el7.x86_64
      args: "ro crashkernel=auto rd.lvm.lv=rhvh_dhcp-11-33/rhvh-4.2.6.0-0.20180828.0+1 rd.lvm.lv=rhvh_dhcp-11-33/swap rhgb quiet LANG=en_US.UTF-8 img.bootid=rhvh-4.2.6.0-0.20180828.0+1"
      initrd: /boot/rhvh-4.2.6.0-0.20180828.0+1/initramfs-3.10.0-862.11.6.el7.x86_64.img
      root: /dev/rhvh_dhcp-11-33/rhvh-4.2.6.0-0.20180828.0+1
current_layer: rhvh-4.2.6.0-0.20180828.0+1

2. In the build 3, enter to emergency mode, nodectl info please refer to attachment "nodectl info"


3. In the build 3, enter to emergency mode, "#ls /dev/sda*", also can not see sda1(boot dev), which is same as bug 1636028.
#ls /dev/sda*
/dev/sda /dev/sda2

Comment 4 Huijuan Zhao 2018-10-30 05:30:51 UTC
Created attachment 1498813 [details]
nodectl info