Bug 855872

Summary: vdsm: "Failed to collect auth credentials" error from vdsm after yum downgrade on vdsm from 4.9.6-32 back to vdsm-4.9-113.3
Product: Red Hat Enterprise Linux 6 Reporter: Dafna Ron <dron>
Component: vdsmAssignee: Barak <bazulay>
Status: CLOSED INSUFFICIENT_DATA QA Contact: Pavel Stehlik <pstehlik>
Severity: high Docs Contact:
Priority: high    
Version: 6.3CC: abaron, alonbl, bazulay, iheim, lpeer, sgrinber, ykaul
Target Milestone: rc   
Target Release: ---   
Hardware: x86_64   
OS: Linux   
Whiteboard: infra
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2012-10-14 07:50:31 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Attachments:
Description Flags
logs none

Description Dafna Ron 2012-09-10 13:22:41 UTC
Created attachment 611428 [details]
logs

Description of problem:

I am testing upgrade from 3.0 cluster to 3.1
after upgrading the vdsm I decided to downgrade and rerun the vdsm upgrade so I ran yum downgrade vdsm 
when vdsm came back up it cannot communicate with libvirt with the below error: 

MainThread::ERROR::2012-09-10 16:06:06,512::vdsm::74::vds::(run) Traceback (most recent call last):
  File "/usr/share/vdsm//vdsm", line 72, in run
    serve_clients(log)
  File "/usr/share/vdsm//vdsm", line 40, in serve_clients
    cif = clientIF.clientIF(log)
  File "/usr/share/vdsm/clientIF.py", line 92, in __init__
    self._libvirt = libvirtconnection.get()
  File "/usr/share/vdsm/libvirtconnection.py", line 94, in get
    conn = libvirt.openAuth('qemu:///system', auth, 0)
  File "/usr/lib64/python2.6/site-packages/libvirt.py", line 102, in openAuth
    if ret is None:raise libvirtError('virConnectOpenAuth() failed')
libvirtError: authentication failed: Failed to collect auth credentials


Version-Release number of selected component (if applicable):


How reproducible:

100%

Steps to Reproduce:
1. install vdsm vdsm-4.9-113.3.el6_3.x86_64
2. upgrade to 4.9.6-32.0.el6_3
3. restart vdsm
4. run yum downgrade vdsm
  
Actual results:

when vdsm starts we cannot comunicate to libvirt with:
authentication failed: Failed to collect auth credentials

Expected results:

we should be able to connect to libvirt

Additional info: attaching vdsm logs and libvirt_password files. 

I tried restarting libvirt/vdsm and the host. 
looking at cd /etc/pki/vdsm/keys/ I can see two files: 
libvirt_password.rpmnew   libvirt_password.rpmsave
both have the correct password.

Comment 1 Alon Bar-Lev 2012-09-12 22:17:40 UTC
Is downgrade supported?

Comment 2 Barak 2012-09-14 14:56:12 UTC
There are 2 aspects for that:
- QE scenario as shown above
- Customers that upgrade a zstream VDSM that brings out a problem in their 
  setup, they may want to downgrade it.

I have instructed Dafna to open this BZ just to bring up the discussion and get PM's response.

I think that a GSS KB is enough.

Should it be tested by QE ?

Andrew ? Miki ?

Comment 3 Simon Grinberg 2012-09-23 16:41:01 UTC
(In reply to comment #2)
> There are 2 aspects for that:
> - QE scenario as shown above
> - Customers that upgrade a zstream VDSM that brings out a problem in their 
>   setup, they may want to downgrade it.

The problem is more complicated then that, since if the customer used re-install the host to upgrade the host then there may be other packaged that may have been upgraded including libvirt. Roll back should be done for all in this case or old VDSM may not work properly - right?

Comment 4 Alon Bar-Lev 2012-09-27 17:59:21 UTC
I need answer here.

I don't think this should be supported.

Downgrade should be done using full bootstrap procedure.

Comment 5 Alon Bar-Lev 2012-10-04 16:47:41 UTC
Barak, please assign to me when we have some answer.

I think this should not be supported.