Bug 1279167 - supervdsm does not start automatically on upgrade from 3.4 to 3.5 or 3.6
Summary: supervdsm does not start automatically on upgrade from 3.4 to 3.5 or 3.6
Keywords:
Status: CLOSED CURRENTRELEASE
Alias: None
Product: vdsm
Classification: oVirt
Component: SuperVDSM
Version: 4.17.11
Hardware: Unspecified
OS: Unspecified
unspecified
high
Target Milestone: ovirt-3.6.2
: 4.17.17
Assignee: Yaniv Bronhaim
QA Contact: Lukas Svaty
URL:
Whiteboard:
Depends On:
Blocks: RHEV3.6Upgrade
TreeView+ depends on / blocked
 
Reported: 2015-11-08 10:33 UTC by Yeela Kaplan
Modified: 2019-04-28 09:47 UTC (History)
7 users (show)

Fixed In Version:
Clone Of:
Environment:
Last Closed: 2016-02-18 11:18:26 UTC
oVirt Team: Infra
Embargoed:
rule-engine: ovirt-3.6.z+
mgoldboi: planning_ack+
oourfali: devel_ack+
pstehlik: testing_ack+


Attachments (Terms of Use)
syslog (61.58 KB, text/plain)
2015-11-08 10:33 UTC, Yeela Kaplan
no flags Details


Links
System ID Private Priority Status Summary Last Updated
oVirt gerrit 49159 0 master ABANDONED [WIP] upgrade: fix import paths to match noarch Never
oVirt gerrit 50115 0 master MERGED Remove __init__.py file from lib64 vdsm python directory 2016-01-10 20:51:25 UTC
oVirt gerrit 51272 0 ovirt-3.6 MERGED Remove __init__.py file from lib64 vdsm python directory 2016-01-11 09:12:00 UTC
oVirt gerrit 96848 0 master MERGED spec, remove unnecessary hack for an upgrade from 4.14 to 4.17 2019-04-02 05:38:43 UTC

Description Yeela Kaplan 2015-11-08 10:33:15 UTC
Created attachment 1091231 [details]
syslog

Description of problem:
Install vdsm-4.14,
configure vdsm,
start vdsm.
Upgrade to vdsm-4.17.
vdsm does not start beacuse its dependency supervdsm fails to start.

syslog is attached, looks like the problem is related to:
daemonAdapter: Traceback (most recent call last):
daemonAdapter: File "/usr/share/vdsm/supervdsmServer", line 34, in <module>
daemonAdapter: from vdsm.infra import sigutils
daemonAdapter: ImportError: No module named infra


Version-Release number of selected component (if applicable):
vdsm-4.17

Expected results:
vdsm and supervdsm should start after upgrade.

Comment 1 Yaniv Bronhaim 2015-11-08 11:57:00 UTC
please attach the yum.log and give more information about the env - was it over rhel or fedora? vdsm-infra is required by vdsm package.. it should be installed during the upgrade

Comment 2 Yeela Kaplan 2015-11-08 13:17:23 UTC
I have upgraded using rpm -Uvh.
host is EL7.2.
vdsm-infra package is installed.
On manual start of vdsm it will start correctly. Fails on upgrade without manual start of services...

Investigating.

Comment 3 Yeela Kaplan 2015-11-24 09:13:35 UTC
When ugrading from 3.4 to 3.5 or from 3.4 to 3.6,
supervdsm server fails to import any vdsm python lib files,
causing supervdsm to fail.

Trying to start vdsm manually works.

My assessment of what happens is as follows:
In vdsm-4.14 vdsm-python was an architecture dependent rpm.
Therefore all vdsm python files were in lib64 (in case of arch x86_64).

In later versions, including vdsm-4.16 vdsm-python was changed to a noarch rpm,
meaning now all vdsm python files are under lib.

postun script defined in spec is running on upgrade of the vdsm package,
trying to start supervdsm.
The tricky part is that its using the spec file of the old rpm,
in which vdsm python files are supposed to be under lib64.

supervdsm is trying to import the vdsm python files from lib64, but of course failing because they were already deleted by upgrade... 

This is why starting vdsm manually succeeds (it is trying the new lib location). 

Dan, 
Since this is an ongoing issue of upgrade for a long time what is its priority?

Comment 4 Yeela Kaplan 2015-11-24 09:50:50 UTC
Yaniv introduced me to a similar hack already existing for this same issue used in vdsm-tool: https://gerrit.ovirt.org/#/c/39410/4

Will follow accordingly.

Comment 5 Oved Ourfali 2015-11-30 08:07:08 UTC
It doesn't make sense that it has been that way when upgrading from 3.4 to 3.5, as people do this upgrade and it works properly.

What am I missing?

Comment 6 Yeela Kaplan 2015-11-30 09:04:10 UTC
The upgrade does work.

Only problem is that it does not start automatically, but you have to start vdsm manually.

Maybe people upgrading did not mind that bit and did not complain about it.

Comment 7 Yeela Kaplan 2015-12-13 14:59:05 UTC
Moving to MODIFIED as it was merged into ovirt-3.6 branch.

Comment 8 Yaniv Bronhaim 2016-01-03 09:36:58 UTC
Upgrading from latest 3.5 to 3.6 when vdsm is up - works [1]
upgrading from latest 3.4 to 3.6 (!which should not be supported!) fails with [2] as mentioned in the bug report.
upgrading from 3.4 to 3.5 fails as well with the following [3]

those fails introduced due to https://gerrit.ovirt.org/#/c/33711/ which caused sooo many troubles :/ which we still face

Anyhow, for 3.6 I wouldn't take https://gerrit.ovirt.org/#/c/50115/2/vdsm.spec.in if upgrade works from 3.5

for 3.5 - we will need a respin if we want to hack it..
Dan, Oved - please advice how to proceed.



[1]
[root@satellite vdsm]# service vdsmd start
Redirecting to /bin/systemctl start  vdsmd.service
[root@satellite vdsm]# service vdsmd status
Redirecting to /bin/systemctl status  vdsmd.service
● vdsmd.service - Virtual Desktop Server Manager
   Loaded: loaded (/usr/lib/systemd/system/vdsmd.service; enabled; vendor preset: disabled)
   Active: active (running) since Sun 2016-01-03 04:22:56 EST; 3s ago
  Process: 19008 ExecStartPre=/usr/libexec/vdsm/vdsmd_init_common.sh --pre-start (code=exited, status=0/SUCCESS)
 Main PID: 19074 (vdsm)
   CGroup: /system.slice/vdsmd.service
           └─19074 /usr/bin/python /usr/share/vdsm/vdsm

Jan 03 04:22:56 satellite.internal.com vdsmd_init_common.sh[19008]: vdsm: Running test_space
Jan 03 04:22:56 satellite.internal.com vdsmd_init_common.sh[19008]: vdsm: Running test_lo
Jan 03 04:22:56 satellite.internal.com systemd[1]: Started Virtual Desktop Server Manager.
Jan 03 04:22:57 satellite.internal.com python[19074]: DIGEST-MD5 client step 2
Jan 03 04:22:57 satellite.internal.com python[19074]: DIGEST-MD5 parse_server_challenge()
Jan 03 04:22:57 satellite.internal.com python[19074]: DIGEST-MD5 ask_user_info()
Jan 03 04:22:57 satellite.internal.com python[19074]: DIGEST-MD5 client step 2
Jan 03 04:22:57 satellite.internal.com python[19074]: DIGEST-MD5 ask_user_info()
Jan 03 04:22:57 satellite.internal.com python[19074]: DIGEST-MD5 make_client_response()
Jan 03 04:22:57 satellite.internal.com python[19074]: DIGEST-MD5 client step 3
[root@satellite vdsm]# vdsm_install 
Loaded plugins: fastestmirror, product-id, subscription-manager
Examining /root/rpmbuild/RPMS/noarch/vdsm-jsonrpc-4.17.15-2.gitac0a428.el7.centos.noarch.rpm: vdsm-jsonrpc-4.17.15-2.gitac0a428.el7.centos.noarch
Marking /root/rpmbuild/RPMS/noarch/vdsm-jsonrpc-4.17.15-2.gitac0a428.el7.centos.noarch.rpm as an update to vdsm-jsonrpc-4.16.30-0.el7.centos.noarch
Examining /root/rpmbuild/RPMS/noarch/vdsm-4.17.15-2.gitac0a428.el7.centos.noarch.rpm: vdsm-4.17.15-2.gitac0a428.el7.centos.noarch
Marking /root/rpmbuild/RPMS/noarch/vdsm-4.17.15-2.gitac0a428.el7.centos.noarch.rpm as an update to vdsm-4.16.30-0.el7.centos.x86_64
Examining /root/rpmbuild/RPMS/noarch/vdsm-python-4.17.15-2.gitac0a428.el7.centos.noarch.rpm: vdsm-python-4.17.15-2.gitac0a428.el7.centos.noarch
Marking /root/rpmbuild/RPMS/noarch/vdsm-python-4.17.15-2.gitac0a428.el7.centos.noarch.rpm as an update to vdsm-python-4.16.30-0.el7.centos.noarch
Examining /root/rpmbuild/RPMS/noarch/vdsm-xmlrpc-4.17.15-2.gitac0a428.el7.centos.noarch.rpm: vdsm-xmlrpc-4.17.15-2.gitac0a428.el7.centos.noarch
Marking /root/rpmbuild/RPMS/noarch/vdsm-xmlrpc-4.17.15-2.gitac0a428.el7.centos.noarch.rpm as an update to vdsm-xmlrpc-4.16.30-0.el7.centos.noarch
Examining /root/rpmbuild/RPMS/noarch/vdsm-cli-4.17.15-2.gitac0a428.el7.centos.noarch.rpm: vdsm-cli-4.17.15-2.gitac0a428.el7.centos.noarch
Marking /root/rpmbuild/RPMS/noarch/vdsm-cli-4.17.15-2.gitac0a428.el7.centos.noarch.rpm to be installed
Examining /root/rpmbuild/RPMS/noarch/vdsm-hook-vmfex-dev-4.17.15-2.gitac0a428.el7.centos.noarch.rpm: vdsm-hook-vmfex-dev-4.17.15-2.gitac0a428.el7.centos.noarch
Marking /root/rpmbuild/RPMS/noarch/vdsm-hook-vmfex-dev-4.17.15-2.gitac0a428.el7.centos.noarch.rpm to be installed
Examining /root/rpmbuild/RPMS/noarch/vdsm-yajsonrpc-4.17.15-2.gitac0a428.el7.centos.noarch.rpm: vdsm-yajsonrpc-4.17.15-2.gitac0a428.el7.centos.noarch
Marking /root/rpmbuild/RPMS/noarch/vdsm-yajsonrpc-4.17.15-2.gitac0a428.el7.centos.noarch.rpm as an update to vdsm-yajsonrpc-4.16.30-0.el7.centos.noarch
Examining /root/rpmbuild/RPMS/noarch/vdsm-tests-4.17.15-2.gitac0a428.el7.centos.noarch.rpm: vdsm-tests-4.17.15-2.gitac0a428.el7.centos.noarch
Marking /root/rpmbuild/RPMS/noarch/vdsm-tests-4.17.15-2.gitac0a428.el7.centos.noarch.rpm to be installed
Examining /root/rpmbuild/RPMS/noarch/vdsm-infra-4.17.15-2.gitac0a428.el7.centos.noarch.rpm: vdsm-infra-4.17.15-2.gitac0a428.el7.centos.noarch
Marking /root/rpmbuild/RPMS/noarch/vdsm-infra-4.17.15-2.gitac0a428.el7.centos.noarch.rpm to be installed
Resolving Dependencies
--> Running transaction check
---> Package vdsm.x86_64 0:4.16.30-0.el7.centos will be updated
---> Package vdsm.noarch 0:4.17.15-2.gitac0a428.el7.centos will be an update
---> Package vdsm-cli.noarch 0:4.17.15-2.gitac0a428.el7.centos will be installed
---> Package vdsm-hook-vmfex-dev.noarch 0:4.17.15-2.gitac0a428.el7.centos will be installed
---> Package vdsm-infra.noarch 0:4.17.15-2.gitac0a428.el7.centos will be obsoleting
---> Package vdsm-jsonrpc.noarch 0:4.16.30-0.el7.centos will be updated
---> Package vdsm-jsonrpc.noarch 0:4.17.15-2.gitac0a428.el7.centos will be an update
---> Package vdsm-python.noarch 0:4.16.30-0.el7.centos will be updated
---> Package vdsm-python.noarch 0:4.17.15-2.gitac0a428.el7.centos will be an update
---> Package vdsm-python-zombiereaper.noarch 0:4.16.30-0.el7.centos will be obsoleted
---> Package vdsm-tests.noarch 0:4.17.15-2.gitac0a428.el7.centos will be installed
---> Package vdsm-xmlrpc.noarch 0:4.16.30-0.el7.centos will be updated
---> Package vdsm-xmlrpc.noarch 0:4.17.15-2.gitac0a428.el7.centos will be an update
---> Package vdsm-yajsonrpc.noarch 0:4.16.30-0.el7.centos will be updated
---> Package vdsm-yajsonrpc.noarch 0:4.17.15-2.gitac0a428.el7.centos will be an update
--> Finished Dependency Resolution

Dependencies Resolved

=========================================================================================================================================================================================
 Package                           Arch                 Version                                          Repository                                                                 Size
=========================================================================================================================================================================================
Installing:
 vdsm-cli                          noarch               4.17.15-2.gitac0a428.el7.centos                  /vdsm-cli-4.17.15-2.gitac0a428.el7.centos.noarch                          340 k
 vdsm-hook-vmfex-dev               noarch               4.17.15-2.gitac0a428.el7.centos                  /vdsm-hook-vmfex-dev-4.17.15-2.gitac0a428.el7.centos.noarch                20 k
 vdsm-infra                        noarch               4.17.15-2.gitac0a428.el7.centos                  /vdsm-infra-4.17.15-2.gitac0a428.el7.centos.noarch                         22 k
     replacing  vdsm-python-zombiereaper.noarch 4.16.30-0.el7.centos
 vdsm-tests                        noarch               4.17.15-2.gitac0a428.el7.centos                  /vdsm-tests-4.17.15-2.gitac0a428.el7.centos.noarch                        2.4 M
Updating:
 vdsm                              noarch               4.17.15-2.gitac0a428.el7.centos                  /vdsm-4.17.15-2.gitac0a428.el7.centos.noarch                              3.6 M
 vdsm-jsonrpc                      noarch               4.17.15-2.gitac0a428.el7.centos                  /vdsm-jsonrpc-4.17.15-2.gitac0a428.el7.centos.noarch                      701 k
 vdsm-python                       noarch               4.17.15-2.gitac0a428.el7.centos                  /vdsm-python-4.17.15-2.gitac0a428.el7.centos.noarch                       831 k
 vdsm-xmlrpc                       noarch               4.17.15-2.gitac0a428.el7.centos                  /vdsm-xmlrpc-4.17.15-2.gitac0a428.el7.centos.noarch                       105 k
 vdsm-yajsonrpc                    noarch               4.17.15-2.gitac0a428.el7.centos                  /vdsm-yajsonrpc-4.17.15-2.gitac0a428.el7.centos.noarch                     93 k

Transaction Summary
=========================================================================================================================================================================================
Install  4 Packages
Upgrade  5 Packages

Total size: 8.1 M
Downloading packages:
Running transaction check
Running transaction test
Transaction test succeeded
Running transaction
  Installing : vdsm-infra-4.17.15-2.gitac0a428.el7.centos.noarch                                                                                                                    1/15 
  Updating   : vdsm-python-4.17.15-2.gitac0a428.el7.centos.noarch                                                                                                                   2/15 
  Updating   : vdsm-xmlrpc-4.17.15-2.gitac0a428.el7.centos.noarch                                                                                                                   3/15 
  Updating   : vdsm-yajsonrpc-4.17.15-2.gitac0a428.el7.centos.noarch                                                                                                                4/15 
  Updating   : vdsm-jsonrpc-4.17.15-2.gitac0a428.el7.centos.noarch                                                                                                                  5/15 
  Installing : vdsm-hook-vmfex-dev-4.17.15-2.gitac0a428.el7.centos.noarch                                                                                                           6/15 
  Updating   : vdsm-4.17.15-2.gitac0a428.el7.centos.noarch                                                                                                                          7/15 
  Installing : vdsm-tests-4.17.15-2.gitac0a428.el7.centos.noarch                                                                                                                    8/15 
  Installing : vdsm-cli-4.17.15-2.gitac0a428.el7.centos.noarch                                                                                                                      9/15 
  Cleanup    : vdsm-4.16.30-0.el7.centos.x86_64                                                                                                                                    10/15 
  Cleanup    : vdsm-jsonrpc-4.16.30-0.el7.centos.noarch                                                                                                                            11/15 
  Cleanup    : vdsm-xmlrpc-4.16.30-0.el7.centos.noarch                                                                                                                             12/15 
  Cleanup    : vdsm-python-4.16.30-0.el7.centos.noarch                                                                                                                             13/15 
  Erasing    : vdsm-python-zombiereaper-4.16.30-0.el7.centos.noarch                                                                                                                14/15 
  Cleanup    : vdsm-yajsonrpc-4.16.30-0.el7.centos.noarch                                                                                                                          15/15 
Loading mirror speeds from cached hostfile
 * base: mirror.nonstop.co.il
 * extras: mirror.nonstop.co.il
 * ovirt-3.4-epel: mirror.nonstop.co.il
 * ovirt-3.4-stable: ftp.plusline.net
 * ovirt-3.5: ftp.plusline.net
 * ovirt-3.5-epel: mirror.nonstop.co.il
 * updates: mirror.nonstop.co.il
  Verifying  : vdsm-infra-4.17.15-2.gitac0a428.el7.centos.noarch                                                                                                                    1/15 
  Verifying  : vdsm-cli-4.17.15-2.gitac0a428.el7.centos.noarch                                                                                                                      2/15 
  Verifying  : vdsm-4.17.15-2.gitac0a428.el7.centos.noarch                                                                                                                          3/15 
  Verifying  : vdsm-hook-vmfex-dev-4.17.15-2.gitac0a428.el7.centos.noarch                                                                                                           4/15 
  Verifying  : vdsm-xmlrpc-4.17.15-2.gitac0a428.el7.centos.noarch                                                                                                                   5/15 
  Verifying  : vdsm-tests-4.17.15-2.gitac0a428.el7.centos.noarch                                                                                                                    6/15 
  Verifying  : vdsm-python-4.17.15-2.gitac0a428.el7.centos.noarch                                                                                                                   7/15 
  Verifying  : vdsm-jsonrpc-4.17.15-2.gitac0a428.el7.centos.noarch                                                                                                                  8/15 
  Verifying  : vdsm-yajsonrpc-4.17.15-2.gitac0a428.el7.centos.noarch                                                                                                                9/15 
  Verifying  : vdsm-xmlrpc-4.16.30-0.el7.centos.noarch                                                                                                                             10/15 
  Verifying  : vdsm-python-zombiereaper-4.16.30-0.el7.centos.noarch                                                                                                                11/15 
  Verifying  : vdsm-4.16.30-0.el7.centos.x86_64                                                                                                                                    12/15 
  Verifying  : vdsm-jsonrpc-4.16.30-0.el7.centos.noarch                                                                                                                            13/15 
  Verifying  : vdsm-yajsonrpc-4.16.30-0.el7.centos.noarch                                                                                                                          14/15 
  Verifying  : vdsm-python-4.16.30-0.el7.centos.noarch                                                                                                                             15/15 

Installed:
  vdsm-cli.noarch 0:4.17.15-2.gitac0a428.el7.centos         vdsm-hook-vmfex-dev.noarch 0:4.17.15-2.gitac0a428.el7.centos       vdsm-infra.noarch 0:4.17.15-2.gitac0a428.el7.centos      
  vdsm-tests.noarch 0:4.17.15-2.gitac0a428.el7.centos      

Updated:
  vdsm.noarch 0:4.17.15-2.gitac0a428.el7.centos               vdsm-jsonrpc.noarch 0:4.17.15-2.gitac0a428.el7.centos          vdsm-python.noarch 0:4.17.15-2.gitac0a428.el7.centos       
  vdsm-xmlrpc.noarch 0:4.17.15-2.gitac0a428.el7.centos        vdsm-yajsonrpc.noarch 0:4.17.15-2.gitac0a428.el7.centos       

Replaced:
  vdsm-python-zombiereaper.noarch 0:4.16.30-0.el7.centos                                                                                                                                 

Complete!

Checking configuration status...

Current revision of multipath.conf detected, preserving
libvirt is already configured for vdsm
SUCCESS: ssl configured to true. No conflicts

Running configure...
Reconfiguration of libvirt is done.

Done configuring modules to VDSM.


[2]
Jan 03 04:26:50 satellite.internal.com systemd[1]: Starting Auxiliary vdsm service for running helper functions as root...
Jan 03 04:26:50 satellite.internal.com daemonAdapter[20654]: Traceback (most recent call last):
Jan 03 04:26:50 satellite.internal.com daemonAdapter[20654]: File "/usr/share/vdsm/supervdsmServer", line 34, in <module>
Jan 03 04:26:50 satellite.internal.com daemonAdapter[20654]: from vdsm.infra import sigutils
Jan 03 04:26:50 satellite.internal.com daemonAdapter[20654]: ImportError: No module named infra
Jan 03 04:26:50 satellite.internal.com systemd[1]: supervdsmd.service: main process exited, code=exited, status=1/FAILURE
Jan 03 04:26:50 satellite.internal.com systemd[1]: Unit supervdsmd.service entered failed state.
Jan 03 04:26:50 satellite.internal.com systemd[1]: supervdsmd.service failed.
Jan 03 04:26:50 satellite.internal.com systemd[1]: supervdsmd.service holdoff time over, scheduling restart.
Jan 03 04:26:50 satellite.internal.com systemd[1]: Started Auxiliary vdsm service for running helper functions as root.
Jan 03 04:26:50 satellite.internal.com systemd[1]: Starting Auxiliary vdsm service for running helper functions as root...
Jan 03 04:26:50 satellite.internal.com daemonAdapter[20662]: Traceback (most recent call last):
Jan 03 04:26:50 satellite.internal.com daemonAdapter[20662]: File "/usr/share/vdsm/supervdsmServer", line 34, in <module>
Jan 03 04:26:50 satellite.internal.com daemonAdapter[20662]: from vdsm.infra import sigutils
Jan 03 04:26:50 satellite.internal.com daemonAdapter[20662]: ImportError: No module named infra
Jan 03 04:26:50 satellite.internal.com systemd[1]: supervdsmd.service: main process exited, code=exited, status=1/FAILURE
Jan 03 04:26:50 satellite.internal.com systemd[1]: Unit supervdsmd.service entered failed state.
Jan 03 04:26:50 satellite.internal.com systemd[1]: supervdsmd.service failed.
Jan 03 04:26:50 satellite.internal.com vdsm-tool[20661]: Traceback (most recent call last):
Jan 03 04:26:50 satellite.internal.com vdsm-tool[20661]: File "/usr/share/vdsm/vdsm-restore-net-config", line 35, in <module>
Jan 03 04:26:50 satellite.internal.com vdsm-tool[20661]: from vdsm.netconfpersistence import RunningConfig, PersistentConfig, \
Jan 03 04:26:50 satellite.internal.com vdsm-tool[20661]: ImportError: cannot import name KernelConfig
Jan 03 04:26:50 satellite.internal.com vdsm-tool[20661]: Traceback (most recent call last):
Jan 03 04:26:50 satellite.internal.com vdsm-tool[20661]: File "/usr/bin/vdsm-tool", line 219, in main
Jan 03 04:26:50 satellite.internal.com vdsm-tool[20661]: return tool_command[cmd]["command"](*args)
Jan 03 04:26:50 satellite.internal.com vdsm-tool[20661]: File "/usr/lib/python2.7/site-packages/vdsm/tool/restore_nets.py", line 41, in restore_command
Jan 03 04:26:50 satellite.internal.com vdsm-tool[20661]: exec_restore(cmd)
Jan 03 04:26:50 satellite.internal.com vdsm-tool[20661]: File "/usr/lib/python2.7/site-packages/vdsm/tool/restore_nets.py", line 54, in exec_restore
Jan 03 04:26:50 satellite.internal.com vdsm-tool[20661]: raise EnvironmentError('Failed to restore the persisted networks')
Jan 03 04:26:50 satellite.internal.com vdsm-tool[20661]: EnvironmentError: Failed to restore the persisted networks


[3]
Jan 03 04:29:58 satellite.internal.com systemd[1]: supervdsmd.service holdoff time over, scheduling restart.
Jan 03 04:29:58 satellite.internal.com systemd[1]: Started "Auxiliary vdsm service for running helper functions as root".
Jan 03 04:29:58 satellite.internal.com systemd[1]: Starting "Auxiliary vdsm service for running helper functions as root"...
Jan 03 04:29:59 satellite.internal.com daemonAdapter[21610]: Traceback (most recent call last):
Jan 03 04:29:59 satellite.internal.com daemonAdapter[21610]: File "/usr/share/vdsm/supervdsmServer", line 56, in <module>
Jan 03 04:29:59 satellite.internal.com daemonAdapter[21610]: from vdsm import sysctl
Jan 03 04:29:59 satellite.internal.com daemonAdapter[21610]: ImportError: cannot import name sysctl
Jan 03 04:29:59 satellite.internal.com systemd[1]: supervdsmd.service: main process exited, code=exited, status=1/FAILURE
Jan 03 04:29:59 satellite.internal.com systemd[1]: Unit supervdsmd.service entered failed state.
Jan 03 04:29:59 satellite.internal.com systemd[1]: supervdsmd.service failed.
Jan 03 04:29:59 satellite.internal.com vdsm-tool[21609]: Traceback (most recent call last):
Jan 03 04:29:59 satellite.internal.com vdsm-tool[21609]: File "/usr/share/vdsm/vdsm-restore-net-config", line 35, in <module>
Jan 03 04:29:59 satellite.internal.com vdsm-tool[21609]: from vdsm.netconfpersistence import KernelConfig, BaseConfig
Jan 03 04:29:59 satellite.internal.com vdsm-tool[21609]: ImportError: cannot import name KernelConfig
Jan 03 04:29:59 satellite.internal.com vdsm-tool[21609]: Traceback (most recent call last):
Jan 03 04:29:59 satellite.internal.com vdsm-tool[21609]: File "/usr/bin/vdsm-tool", line 219, in main
Jan 03 04:29:59 satellite.internal.com vdsm-tool[21609]: return tool_command[cmd]["command"](*args)
Jan 03 04:29:59 satellite.internal.com vdsm-tool[21609]: File "/usr/lib/python2.7/site-packages/vdsm/tool/restore_nets.py", line 40, in restore_command
Jan 03 04:29:59 satellite.internal.com vdsm-tool[21609]: exec_restore(cmd)
Jan 03 04:29:59 satellite.internal.com vdsm-tool[21609]: File "/usr/lib/python2.7/site-packages/vdsm/tool/restore_nets.py", line 53, in exec_restore
Jan 03 04:29:59 satellite.internal.com vdsm-tool[21609]: raise EnvironmentError('Failed to restore the persisted networks')
Jan 03 04:29:59 satellite.internal.com vdsm-tool[21609]: EnvironmentError: Failed to restore the persisted networks
Jan 03 04:29:59 satellite.internal.com systemd[1]: vdsm-network.service: main process exited, code=exited, status=1/FAILURE
Jan 03 04:29:59 satellite.internal.com systemd[1]: Failed to start Virtual Desktop Server Manager network restoration.
Jan 03 04:29:59 satellite.internal.com systemd[1]: Dependency failed for Virtual Desktop Server Manager.
Jan 03 04:29:59 satellite.internal.com systemd[1]: Job vdsmd.service/start failed with result 'dependency'.
Jan 03 04:29:59 satellite.internal.com systemd[1]: Unit vdsm-network.service entered failed state.
Jan 03 04:29:59 satellite.internal.com systemd[1]: vdsm-network.service failed.
Jan 03 04:29:59 satellite.internal.com systemd[1]: supervdsmd.service holdoff time over, scheduling restart.
Jan 03 04:29:59 satellite.internal.com systemd[1]: Starting Virtual Desktop Server Manager network restoration...
Jan 03 04:29:59 satellite.internal.com systemd[1]: Started "Auxiliary vdsm service for running helper functions as root".
Jan 03 04:29:59 satellite.internal.com systemd[1]: Starting "Auxiliary vdsm service for running helper functions as root"...
Jan 03 04:29:59 satellite.internal.com daemonAdapter[21620]: Traceback (most recent call last):
Jan 03 04:29:59 satellite.internal.com daemonAdapter[21620]: File "/usr/share/vdsm/supervdsmServer", line 56, in <module>
Jan 03 04:29:59 satellite.internal.com daemonAdapter[21620]: from vdsm import sysctl
Jan 03 04:29:59 satellite.internal.com daemonAdapter[21620]: ImportError: cannot import name sysctl

Comment 9 Nir Soffer 2016-01-03 10:29:45 UTC
Yaniv, how are we preventing upgrades from 3.4 to 3.6?

If this upgrade is possible by running yum update, it must work.

Comment 10 Oved Ourfali 2016-01-03 10:38:19 UTC
(In reply to Nir Soffer from comment #9)
> Yaniv, how are we preventing upgrades from 3.4 to 3.6?
> 
> If this upgrade is possible by running yum update, it must work.

I tend to disagree here.
The documentation of the upgrade process is to go through every release (not z-stream, of course).
However, I do think we should fix that anyway, for 3.6, unless there is a technical issue to do so.
Yaniv - What's the effort estimation to fix that?

Comment 11 Nir Soffer 2016-01-03 10:47:16 UTC
(In reply to Oved Ourfali from comment #10)
> (In reply to Nir Soffer from comment #9)
> > Yaniv, how are we preventing upgrades from 3.4 to 3.6?
> > 
> > If this upgrade is possible by running yum update, it must work.
> 
> I tend to disagree here.
> The documentation of the upgrade process is to go through every release (not
> z-stream, of course).

You cannot expect people to read documentation. You should enforce this via
packaging, or make it work.

For example, vdsm 4.17 can fail an upgrade from vdsm 4.15, asking to install
vdsm 4.16 first.

Comment 12 Yaniv Bronhaim 2016-01-03 19:32:53 UTC
Ok, I post update to https://gerrit.ovirt.org/50115 and upgrade works from 3.4 to 3.6 with it. ack if you agree with that approach..

Comment 13 Lukas Svaty 2016-01-21 15:11:55 UTC
working on vdsm-4.17.18-0.el7ev.noarch

Comment 14 Yaniv Bronhaim 2016-01-22 11:00:16 UTC
can you specify exactly the steps for the verification - 
from which version you upgrade? 
what do you do after vdsm installation? do you run the service before the upgrade?
did you use engine to deploy the host or you installed the rpms directly on host? 
how did you perform the upgrade? yum? engine call?

Comment 15 Lukas Svaty 2016-01-22 11:27:15 UTC
Steps used:
1. Create 3.4 dc + cluster in 3.6 engine
2. deploy host with 3.4 vdsm (vdsm-4.14.18-7.el7ev.x86_64.rpm)
3. wait for vdsm up
4. move host to maintenance
5. add repo to last 3.6 to host (vdsm-4.17.18-0.el7ev.noarch.rpm)
6. in webadmin redeploy host (will download new packages, install, configure, start)
7. check vdsm and supervdsm are running and correctly configured

If you believe further testing is necessary here, pls propose some scenarios which needs to be covered and move back to ON_QA.

Comment 16 Yaniv Bronhaim 2016-01-23 16:57:51 UTC
iirc reinstall(redeploy) first delete old vdsm and then upgrade the rpms (you can check what it did in /var/log/yum.log , so its actually not an upgrade.. in 3.6 you should have the option to perform upgrade when newer available vdsm rpms on the host. can you try to use it? or just run "yum upgrade vdsm" on host?

Comment 17 Lukas Svaty 2016-01-25 11:16:39 UTC
tried with manual update of vdsm and with upgrade via webadmin, both tuns successful


Note You need to log in before you can comment on or make changes to this bug.