Bug 1120650 - RHEV-Hypervisor 7.0 auto install failed
Summary: RHEV-Hypervisor 7.0 auto install failed
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat Enterprise Virtualization Manager
Classification: Red Hat
Component: ovirt-node
Version: 3.5.0
Hardware: Unspecified
OS: Unspecified
urgent
urgent
Target Milestone: ---
: 3.5.0
Assignee: Ryan Barry
QA Contact: Virtualization Bugs
URL:
Whiteboard: node
Depends On: rebase-ovirt-node-3.1
Blocks: rhevh-7.0 1125452 rhev35betablocker rhev35rcblocker rhev35gablocker
TreeView+ depends on / blocked
 
Reported: 2014-07-17 10:50 UTC by cshao
Modified: 2016-02-10 20:10 UTC (History)
15 users (show)

Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Clone Of:
: 1125452 (view as bug list)
Environment:
Last Closed: 2015-02-11 21:00:22 UTC
oVirt Team: Node


Attachments (Terms of Use)
auto-failed1.png (11.48 KB, image/png)
2014-07-17 10:50 UTC, cshao
no flags Details
auto-failed2.png (21.97 KB, image/png)
2014-07-17 10:51 UTC, cshao
no flags Details
fail (48.83 KB, image/png)
2014-07-31 06:36 UTC, cshao
no flags Details
auto-failed-new (16.72 KB, image/png)
2014-09-05 05:19 UTC, cshao
no flags Details
ovirt.log (12.17 KB, text/plain)
2014-09-05 09:25 UTC, cshao
no flags Details
ovirt-node.log (1.17 KB, text/plain)
2014-09-05 09:30 UTC, cshao
no flags Details
ovirt.log-0918 (46.95 KB, text/plain)
2014-09-19 06:53 UTC, cshao
no flags Details
ovirt-node.log-0918 (5.58 KB, text/plain)
2014-09-19 06:54 UTC, cshao
no flags Details


Links
System ID Priority Status Summary Last Updated
Red Hat Product Errata RHEA-2015:0160 normal SHIPPED_LIVE ovirt-node bug fix and enhancement update 2015-02-12 01:34:52 UTC
oVirt gerrit 30907 master MERGED Use the new codebase for autoinstalls, check for None Never
oVirt gerrit 31152 None None None Never
oVirt gerrit 31168 None None None Never
oVirt gerrit 32534 master MERGED Revert logic changes in ovirt-functions.in Never
oVirt gerrit 33077 master MERGED Fix persistence logic Never
oVirt gerrit 33078 master MERGED selinux: Some more rules Never
oVirt gerrit 33319 node-3.0 MERGED Fix persistence logic Never
oVirt gerrit 33365 master MERGED functions: Yet another fix for retnum persistence Never
oVirt gerrit 34338 node-3.0 MERGED functions: Yet another fix for retnum persistence Never
Red Hat Bugzilla 1144359 None None None Never

Internal Links: 1144359

Description cshao 2014-07-17 10:50:33 UTC
Created attachment 918673 [details]
auto-failed1.png

Description of problem:
Hypervisor auto install failed

Entering energency mode. Exit the shell to continue.
Type "journalctl" to view system logs.
You might want to save "/run/initramfs/rdsosreport.txt" to USB stick or /boot after mounting them and attach is to a bug report.

Version-Release number of selected component (if applicable):
rhev-hypervisor7-7.0-20140714.0
ovirt-node-3.1.0-0.5.20140711git7197118.el7.noarch

How reproducible:
100%

Steps to Reproduce:
1. Auto install RHEVH with below parameters:
BOOTIF=etho storage_init=/dev/sda adminpw=XXXXXX firstboot
2.
3.

Actual results:
Hypervisor auto install failed

Expected results:
Hypervisor auto install can successful.

Additional info:

Comment 1 cshao 2014-07-17 10:51:11 UTC
Created attachment 918674 [details]
auto-failed2.png

Comment 2 Ying Cui 2014-07-17 11:22:49 UTC
This bug blocked auto parameters test cases, so add testblocker keywords.

and update the description to BOOTIF=eth0, not BOOTIF=etho

Comment 4 Ryan Barry 2014-07-17 19:25:10 UTC
I'm working on this, but I'd appreciate rdsosreport from a system with rd.debug on the kernel cmdline. This appears to be reproducable with BOOTIF only.

Comment 6 Fabian Deutsch 2014-07-30 13:19:03 UTC
RHEL 7 is using predictive device names. Could you please try to use ens3 or p*p* based names?

Easiest to determine the name is to boot into the TUI installer drop to shell and run ip l.

Comment 7 cshao 2014-07-31 06:35:54 UTC
(In reply to Fabian Deutsch from comment #6)
> RHEL 7 is using predictive device names. Could you please try to use ens3 or
> p*p* based names?
> 
> Easiest to determine the name is to boot into the TUI installer drop to
> shell and run ip l.

Hi fabiand,

The auto installation still got failed with "BOOTIF=ens3 storage_init=/dev/sda firstboot" parameters, but this time with the different error info.

Test version:
rhev-hypervisor7-7.0-20140714.0

Please see attachment "fail.png" for more details.

Comment 8 cshao 2014-07-31 06:36:38 UTC
Created attachment 922827 [details]
fail

Comment 9 Fabian Deutsch 2014-07-31 13:58:41 UTC
Hey Chen,

thanks.

Julie, do we need to document that the "new" device names need to be used? Or is it assumed that the users knows that RHEL7 follows a different NIC naming scheme?

Comment 10 Julie 2014-07-31 22:03:03 UTC
(In reply to Fabian Deutsch from comment #9)
> Hey Chen,
> 
> thanks.
> 
> Julie, do we need to document that the "new" device names need to be used?
> Or is it assumed that the users knows that RHEL7 follows a different NIC
> naming scheme?

Thanks for bringing this bug to my attention. Instructions for RHEV-H 7 will need to be updated. Once updated, we will be requesting a tech review from the engineering team. 

Cheers,
Julie

Comment 11 haiyang,dong 2014-08-07 06:33:30 UTC
Test version:
ovirt-node-3.1.0-0.7.20140806gitef5c5cb.el7.noarch
rhev-hypervisor7-7.0-20140806.1.iso 

autoinstall still failed with the follow error:
[root@dhcp-66-72-90 admin]# python /etc/ovirt-config-boot.d/snmp_autoinstall.py
Traceback (most recent call last):
  File "/etc/ovirt-config-boot.d/snmp_autoinstall.py", line 24, in <module>
    args = system.kernel_cmdline_args()
AttributeError: 'module' object has no attribute 'kernel_cmdline_args'


[root@dhcp-66-72-90 admin]# python /etc/ovirt-config-boot.d/cim_autoinstall.py
Traceback (most recent call last):
  File "/etc/ovirt-config-boot.d/cim_autoinstall.py", line 26, in <module>
    args = system.kernel_cmdline_args()
AttributeError: 'module' object has no attribute 'kernel_cmdline_args'

so need change it into "assigned" again.

Comment 12 cshao 2014-09-05 05:18:07 UTC
Test version:
rhev-hypervisor7-7.0-20140904.0.el7ev
ovirt-node-3.1.0-0.10.20140904gitb828c37.el7.noarch


Test steps:
Clean auto install with "BOOTIF=xxx storage_init=/dev/sda adminpw=xxx enforcing=0 firstboot" still failed, please see attachment for more details.

so need to re-assigned it again.

Comment 13 cshao 2014-09-05 05:19:03 UTC
Created attachment 934683 [details]
auto-failed-new

Comment 14 Fabian Deutsch 2014-09-05 09:19:28 UTC
Chen, could you please attach /var/log.ovirt.log and /var/log/ovirt-node.log

Comment 15 cshao 2014-09-05 09:25:05 UTC
Created attachment 934720 [details]
ovirt.log

Comment 16 cshao 2014-09-05 09:30:18 UTC
Created attachment 934722 [details]
ovirt-node.log

Comment 17 cshao 2014-09-19 06:52:51 UTC
I have to assigned this bug due to auto install still failed, the error same as #c11.

Manual execute snmp*.py no issue, but not sure why tui such issue.
#python /etc/ovirt-config-boot.d/snmp_autoinstall.pyc
#echo $?
0

Teset version
rhev-hypervisor7-7.0-20140918.0.iso
ovirt-node-3.1.0-0.13.20140918gitdda78cb.el7.noarch
vdsm-4.14.13-2.el7ev.x86_64
vdsm-reg-4.14.13-2.el7ev.noarch
ovirt-node-plugin-vdsm-0.1.2-3.el7ev.noarch
libvirt-1.1.1-29.el7_0.1.x86_64

Comment 18 cshao 2014-09-19 06:53:44 UTC
Created attachment 939104 [details]
ovirt.log-0918

Comment 19 cshao 2014-09-19 06:54:50 UTC
Created attachment 939105 [details]
ovirt-node.log-0918

Comment 20 Fabian Deutsch 2014-09-19 14:33:18 UTC
(In reply to shaochen from comment #17)
> I have to assigned this bug due to auto install still failed, the error same
> as #c11.
> 
> Manual execute snmp*.py no issue, but not sure why tui such issue.
> #python /etc/ovirt-config-boot.d/snmp_autoinstall.pyc
> #echo $?
> 0


Right.
I could reproduce a failed auto-installation. But it was a different sympton than before.

My findings: A new persistence code returns different falues for different errors, this made the installer think that some persistence lead to a severe failure. This ha sbeen fixed.
In addition to that there were some selinux denials preventing the normal operation, this has also been addressed.

Tested as follows:

1. Add BOOTIF=ens3 storage_init=/dev/sda
2. Watch the auto-installation taking place ending with a reboot

Comment 22 cshao 2014-09-29 06:18:11 UTC
Test version:
rhev-hypervisor7-7.0-20140926.0.iso
ovirt-node-3.1.0-0.17.20140925git29c3403.el7.noarch

Test result:
Auto install RHEV-H with below parameters can succeed.
BOOTIF=xxx storage_init=/dev/sda adminpw=xxx firstboot

So the bug is fixed, change bug status to VERIFIED.

NOTE:
Also auto install can succeed, but this still have another issue about login password.
User can't login the hypervisor with the new setting password.
Pop-up: Authentication token manipulation error.
I am debugging on it, and will report a new bug to trace this issue.

Thanks!

Comment 26 errata-xmlrpc 2015-02-11 21:00:22 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://rhn.redhat.com/errata/RHEA-2015-0160.html


Note You need to log in before you can comment on or make changes to this bug.