Bug 1422743 - [4.0 Only] After RHVH upgrade kdump service fails to start with "Error: /boot/vmlinuz-3.10.0-514.2.2.el7.x86_64 not found."
Summary: [4.0 Only] After RHVH upgrade kdump service fails to start with "Error: /boot...
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat Enterprise Virtualization Manager
Classification: Red Hat
Component: rhev-hypervisor-ng
Version: 4.0.6
Hardware: Unspecified
OS: Unspecified
unspecified
high
Target Milestone: ovirt-4.1.1
: ---
Assignee: Ryan Barry
QA Contact: Huijuan Zhao
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2017-02-16 04:12 UTC by Ameya Charekar
Modified: 2020-03-11 15:48 UTC (History)
15 users (show)

Fixed In Version: 4.1 GA
Doc Type: Bug Fix
Doc Text:
Previously, Red Hat Virtualization Host (RHVH) placed several requisite files (initrd, kernel) in a subdirectory under /boot. As a result, some system utilities, including kdump, were unable to locate the files in the subdirectory, regardless of the BOOT_IMAGE settings. In this release, the boot files are copied to /boot on startup and kdump is functioning properly.
Clone Of:
Environment:
Last Closed: 2017-04-20 19:03:40 UTC
oVirt Team: Node
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Red Hat Knowledge Base (Solution) 2951141 0 None None None 2017-03-02 17:39:38 UTC
Red Hat Product Errata RHEA-2017:1114 0 normal SHIPPED_LIVE redhat-virtualization-host bug fix and enhancement update 2017-04-20 22:57:46 UTC

Description Ameya Charekar 2017-02-16 04:12:53 UTC
Description of problem:
After RHVH upgrade from older image having kernel version prior to 3.10.0-514.2.2.el7.x86_64 kdump service fails as /boot/vmlinuz-3.10.0-514.2.2.el7.x86_64 is missing.

Version-Release number of selected component (if applicable):


How reproducible:
Always.

Steps to Reproduce:
1. Install older RHVH e.g. RHVH-4.0-20160817.1.
2. Upgrade RHVH to latest one.
3. Check kdump service status.

Actual results:
Service kdump fails to start.

Expected results:
Service kdump should start without any issue.

Additional info:
Service starts after creating symlink as a workaround:
# ln -s /boot/rhvh-4.0-0.20170201.0+1/vmlinuz-3.10.0-514.2.2.el7.x86_64 /boot/

Comment 1 Ameya Charekar 2017-02-16 04:20:46 UTC
Can anyone please confirm is creating symlink is a right workaround or do we have any other recommended workaround?

Comment 3 Ryan Barry 2017-02-16 04:51:18 UTC
This appears to be yet another utility (along with grub2-mkconfig, virt-v2v, and dracut-fips) which doesn't honor BOOT_IMAGE

We now have a service which copies the kernel and initrd to /boot. This resolves the other services, but I suspect kdump may be running first.

I'll try to reproduce tomorrow, but can you please attach syslog? I'd like to see whether imgbase-copy-bootfiles ran after kdump.

I'd also expect the right vmlinuz to be in /boot before you symlinked, but can't confirm without logs (or until tomorrow when I try to reproduce)

Comment 5 Ryan Barry 2017-02-16 17:40:08 UTC
After testing, this is fixed in 4.1.

Until then, yes, symlinking the kernel (or copying it) is a reasonable workaround.

Sandro, since there's a ticket attached, I'd like to backport https://gerrit.ovirt.org/#/c/68749/ to ovirt-4.0 / 4.0.7. I'm not sure if it's worth it given the timeframe and easy workaround, though.

Comment 7 Sandro Bonazzola 2017-02-17 09:26:33 UTC
(In reply to Ryan Barry from comment #5)
> After testing, this is fixed in 4.1.
> 
> Until then, yes, symlinking the kernel (or copying it) is a reasonable
> workaround.
> 
> Sandro, since there's a ticket attached, I'd like to backport
> https://gerrit.ovirt.org/#/c/68749/ to ovirt-4.0 / 4.0.7. I'm not sure if
> it's worth it given the timeframe and easy workaround, though.

Ack from devel side for the backport since patch is already available and need just a backport. Given the timeframe, probably not worth to be backported.
Adding Marina for getting her opinion.

Comment 8 Marina Kalinin 2017-02-17 22:03:30 UTC
Based on the 2 facts:
1. Any new RHVH is backward compatible with both engine 4.0 and 3.6
+
2. We have an easy confirmed workaround - comment#5

I believe we can have it on 4.1 GA only.

Since this is also not something that has a global effect on all the users and would not bring the environment down, it seems ok to me to wait till 4.1 GA.

Comment 10 Huijuan Zhao 2017-02-20 07:10:29 UTC
Can reproduce according to Comment 0.

Comment 11 Sandro Bonazzola 2017-02-21 09:34:58 UTC
According to comment #8, I'm closing next release.
It's fixed in 4.1

Comment 12 Marina Kalinin 2017-03-07 05:20:42 UTC
(In reply to Sandro Bonazzola from comment #11)
> According to comment #8, I'm closing next release.
> It's fixed in 4.1

I am setting the "Fixed In Version" field to 4.1 GA then.

Comment 13 cshao 2017-03-17 03:11:35 UTC
(In reply to Sandro Bonazzola from comment #11)
> According to comment #8, I'm closing next release.
> It's fixed in 4.1

Sandro,

Kindly reminder, according bugzilla process, it is not recommending that close a bug as close next release without no QE verification it.

If the bug is fixed in 4.1, whether set target milestone to 4.1 would be more appropriate then 4.0.7?

Thanks.

Comment 14 Huijuan Zhao 2017-03-17 11:32:44 UTC
Test version:
From: RHVH-4.0-20160822.8-RHVH-x86_64-dvd1.iso
To: rhvh-4.1-0.20170314.0

Steps to Reproduce:
1. Install older RHVH e.g. RHVH-4.0-20160822.8-RHVH-x86_64-dvd1.iso
2. Reboot and login RHVH, setup local repos, upgrade RHVH to latest one, rhvh-4.1-0.20170314.0
# yum update
3. Reboot and login new build rhvh-4.1-0.20170314.0, check kdump service status.

Test results:
After step3, kdump service is active.
[root@dell-per730-35 ~]# imgbase layout
rhvh-4.0-0.20160817.0
 +- rhvh-4.0-0.20160817.0+1
rhvh-4.1-0.20170315.0
 +- rhvh-4.1-0.20170315.0+1
[root@dell-per730-35 ~]# imgbase w
[INFO] You are on rhvh-4.1-0.20170315.0+1
[root@dell-per730-35 ~]# systemctl status kdump
● kdump.service - Crash recovery kernel arming
   Loaded: loaded (/usr/lib/systemd/system/kdump.service; enabled; vendor preset: disabled)
   Active: active (exited) since Fri 2017-03-17 11:40:05 GMT; 7min ago
  Process: 3457 ExecStart=/usr/bin/kdumpctl start (code=exited, status=0/SUCCESS)
 Main PID: 3457 (code=exited, status=0/SUCCESS)
   CGroup: /system.slice/kdump.service

Mar 17 11:40:04 dell-per730-35.lab.eng.pek2.redhat.com dracut[7586]: drwxr-xr-x   2 root     root            0 Mar 17 11:39 usr/share/zone...frica
Mar 17 11:40:04 dell-per730-35.lab.eng.pek2.redhat.com dracut[7586]: -rw-r--r--   1 root     root          156 Mar  2 15:01 usr/share/zone...idjan
Mar 17 11:40:04 dell-per730-35.lab.eng.pek2.redhat.com dracut[7586]: drwxr-xr-x   2 root     root            0 Mar 17 11:39 var
Mar 17 11:40:04 dell-per730-35.lab.eng.pek2.redhat.com dracut[7586]: lrwxrwxrwx   1 root     root           11 Mar 17 11:39 var/lock -> ...../lock
Mar 17 11:40:04 dell-per730-35.lab.eng.pek2.redhat.com dracut[7586]: lrwxrwxrwx   1 root     root            6 Mar 17 11:39 var/run -> ../run
Mar 17 11:40:04 dell-per730-35.lab.eng.pek2.redhat.com dracut[7586]: ========================================================================
Mar 17 11:40:04 dell-per730-35.lab.eng.pek2.redhat.com dracut[7586]: *** Creating initramfs image file '/boot/initramfs-3.10.0-514.6.1.el7...e ***
Mar 17 11:40:05 dell-per730-35.lab.eng.pek2.redhat.com kdumpctl[3457]: kexec: loaded kdump kernel
Mar 17 11:40:05 dell-per730-35.lab.eng.pek2.redhat.com kdumpctl[3457]: Starting kdump: [OK]
Mar 17 11:40:05 dell-per730-35.lab.eng.pek2.redhat.com systemd[1]: Started Crash recovery kernel arming.
Hint: Some lines were ellipsized, use -l to show in full.

So this bug is fixed in rhvh-4.1-0.20170314.0, change the status to VERIFIED.

Comment 15 errata-xmlrpc 2017-04-20 19:03:40 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHEA-2017:1114


Note You need to log in before you can comment on or make changes to this bug.