Bug 1297404

Summary: The vm consoles are not set correctly after the upgrade.
Product: Red Hat Enterprise Virtualization Manager Reporter: Roman Hodain <rhodain>
Component: ovirt-engineAssignee: jniederm
Status: CLOSED CURRENTRELEASE QA Contact: Israel Pinto <ipinto>
Severity: urgent Docs Contact:
Priority: urgent    
Version: 3.6.1CC: ahadas, cshao, eedri, gklein, ipinto, jniederm, lsurette, mavital, mgoldboi, michal.skrivanek, rbalakri, Rhev-m-bugs, rhodain, sbonazzo, smelamud, yeylon, ykaul, ylavi
Target Milestone: ovirt-3.6.3Keywords: Regression
Target Release: 3.6.3   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
The internal information about VM consoles was updated incorrectly during upgrade from 3.5 to 3.6 when VMs were running or started via REST API or Run Once. This was preventing console access and caused confusion. The now they should appear correctly, with the same console settings as they used to be in 3.5 (no restart needed)
Story Points: ---
Clone Of: Environment:
Last Closed: 2016-03-21 13:02:53 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: Virt RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 1285700    
Attachments:
Description Flags
vdsm_log
none
engine_log
none
console_error
none
vms_after_upgrade_spice_and_vnc none

Description Roman Hodain 2016-01-11 12:28:23 UTC
Description of problem:
When the system is upgraded from RHEV-M 3.5.6 to RHEV-M 3.6.1 all running VMs have console set to SPICE+VNC.

Version-Release number of selected component (if applicable):
rhevm-3.6.1.3-0.1.el6.noarch

How reproducible:
100%

Steps to Reproduce:
1.upgrade 3.5 to 3.6

Actual results:
The console is set to spice+vnc. In case the console was set to vnc before the upgrade. The attempt to open the console fails as engine opens spice connection to vnc port.

Expected results:
The console configuration is inherited from the previous version.

Comment 1 Michal Skrivanek 2016-01-12 09:37:57 UTC
is it set in the VM properties or just the frontend?

Comment 2 Roman Hodain 2016-01-12 11:30:47 UTC
(In reply to Michal Skrivanek from comment #1)
> is it set in the VM properties or just the frontend?

Only frontend.

Comment 4 Michal Skrivanek 2016-01-12 11:38:46 UTC
so, for the timebeing you can workaround it by selecting "vnc" in Console Options, right?
https://gerrit.ovirt.org/#/c/48855 fixed the Edit dialog...but we need to check it fixes the connection as well

Comment 5 Israel Pinto 2016-01-13 08:35:24 UTC
Tested upgrade form 3.5.7 to 3.6.2 
Did not reproduce

Comment 6 meital avital 2016-01-13 16:23:45 UTC
(In reply to Israel Pinto from comment #5)
> Tested upgrade form 3.5.7 to 3.6.2 
> Did not reproduce

This is the flow we did:

1) Create VM1 with VNC console before the upgrade on 3.5.7

2) Upgrade to 3.6 - 3.6.2-0.1.el6

3) After the upgrade the console of the VM1 still VNC

Roman, Is this the correct scenario? do I need to try anything else?

Comment 7 Roman Hodain 2016-01-14 08:08:02 UTC
(In reply to meital avital from comment #6)
> (In reply to Israel Pinto from comment #5)
> > Tested upgrade form 3.5.7 to 3.6.2 
> > Did not reproduce
> 
> This is the flow we did:
> 
> 1) Create VM1 with VNC console before the upgrade on 3.5.7
> 
> 2) Upgrade to 3.6 - 3.6.2-0.1.el6
> 
> 3) After the upgrade the console of the VM1 still VNC
> 
> Roman, Is this the correct scenario? do I need to try anything else?

It is our production environment so we did not create any VMs. They were already running there.

We just ran upgrade from 3.5.5 to 3.5.6 (latest available version at that time) and then upgrade to 3.6 beta2

The issue is obviously only on running vms.

Comment 8 Michal Skrivanek 2016-01-15 08:33:32 UTC
missed 3.6.2. Somehow bot reset pm_ack, please re-ack

Comment 9 Yaniv Lavi 2016-01-17 08:39:41 UTC
(In reply to Michal Skrivanek from comment #8)
> missed 3.6.2. Somehow bot reset pm_ack, please re-ack

I put in the request to stop the bot from doing pm ack reset on component change, should happen soon.

Comment 10 Arik 2016-01-17 14:41:06 UTC
Roman, I understand that it happened only for running VMs but did it happen for all running VMs? is there any chance that the VMs were added with SPICE and actually ran with VNC (it is possible if they run in run-once mode or via rest-api where you specify the configuration when running the vm) or vice-versa?

Comment 11 Israel Pinto 2016-01-25 13:57:56 UTC
(In reply to Arik from comment #10)
> Roman, I understand that it happened only for running VMs but did it happen
> for all running VMs? is there any chance that the VMs were added with SPICE
> and actually ran with VNC (it is possible if they run in run-once mode or
> via rest-api where you specify the configuration when running the vm) or
> vice-versa?

We reproduce the issue: 
Upgrade form 3.5.7  to 3.6.2.6-0.1 
1. Create VM with display SPICE - vm_spice
2. Create VM with display VNC - vm_vnc
3. Run vm_spice with Run Once and set display to VNC
4. Run vm_vnc with Run Once 
5. Upgrade engine to 3.6.2.6-0.1 
After upgrade the Graphics protocol is change to SPICE+VNC to both VMs
Can't open console got error for both VMs:
Unable to connect to the graphic server (null)
Could not connect to 10.35.161.159: No route to host

adding engine and vdsm logs

Comment 12 Israel Pinto 2016-01-25 14:05:48 UTC
Created attachment 1118038 [details]
vdsm_log

Comment 13 Israel Pinto 2016-01-25 14:06:30 UTC
Created attachment 1118039 [details]
engine_log

Comment 14 Israel Pinto 2016-01-25 16:22:41 UTC
Created attachment 1118101 [details]
console_error

Comment 15 Israel Pinto 2016-01-25 16:23:34 UTC
Created attachment 1118102 [details]
vms_after_upgrade_spice_and_vnc

Comment 16 Israel Pinto 2016-02-07 10:28:32 UTC
Verify on:
3.6: RHEVM Version: 3.6.3-0.1.el6
3.5: RHEVM:3.5.7-0.1.el6ev 

Scenario:
In 3.5:
1. Create VM with SPICE console: vm_spice
2. Create with VNC console: vm_vnc
3. Run vm_spice with run once and update console to VNC
4. Run vm_vnc with run once
5. Verify the both consoles are opened
6. Upgrade engine from 3.5 to 3.6
7. Check that after upgrade both consoles are opened and 'Graphics protocol' stay VNC
In 3.6:
8. Migration both VM to 3.6 host 
9. Check that after migration both consoles are opened and Graphics protocol stay VNC

Results:
After upgrade: both consoles opened  -- PASS
After migration: both consoles opened  -- PASS

Comment 17 Roman Hodain 2016-02-17 12:52:31 UTC
(In reply to Arik from comment #10)
> Roman, I understand that it happened only for running VMs but did it happen
> for all running VMs? is there any chance that the VMs were added with SPICE
> and actually ran with VNC (it is possible if they run in run-once mode or
> via rest-api where you specify the configuration when running the vm) or
> vice-versa?

I happened to all VMs. I do not believe that all the VMs were started as run ones.

Comment 18 Eyal Edri 2016-03-21 13:02:53 UTC
3.6.3 was GA, closing