Bug 1282796

Summary: [Cinder] ThawVDSCommand fails when taking live snapshot without a guest agent
Product: [oVirt] ovirt-engine Reporter: Ori Gofen <ogofen>
Component: BLL.StorageAssignee: Daniel Erez <derez>
Status: CLOSED CURRENTRELEASE QA Contact: Natalie Gavrielov <ngavrilo>
Severity: medium Docs Contact:
Priority: unspecified    
Version: 3.6.0.2CC: acanan, amureini, bugs, derez, ogofen, tnisan, ylavi
Target Milestone: ovirt-3.6.2Flags: rule-engine: ovirt-3.6.z+
ylavi: planning_ack+
tnisan: devel_ack+
rule-engine: testing_ack+
Target Release: 3.6.2   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2016-02-18 11:20:41 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: Storage RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Attachments:
Description Flags
Logs
none
cinder - live snapshot without guest none

Description Ori Gofen 2015-11-17 12:59:00 UTC
Created attachment 1095481 [details]
Logs

Description of problem:
Taking live snapshot of a Vm with Cinder volumes (ceph backend) fails on logs and UI, it looks like that operation fails due to failure to find a guestagent on the Vm.
2015-11-17 14:45:48,990 INFO  [org.ovirt.engine.core.bll.storage.CreateCinderSnapshotCommand] (DefaultQuartzScheduler_Worker-1) [108f473e] Ending command 'org.ovirt.engine.co
re.bll.storage.CreateCinderSnapshotCommand' successfully.
2015-11-17 14:45:53,626 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.ThawVDSCommand] (org.ovirt.thread.pool-7-thread-44) [42818ef1] Failed in 'ThawVDS' method
2015-11-17 14:45:53,631 ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-7-thread-44) [42818ef1] Correlation ID: null, Call
 Stack: null, Custom Event ID: -1, Message: VDSM fury66.tlv.redhat.com command failed: Guest agent is not responding: Guest agent not available for now
2015-11-17 14:45:53,631 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ThawVDSCommand] (org.ovirt.thread.pool-7-thread-44) [42818ef1] Command 'org.ovirt.engine.core.vdsbrok
er.vdsbroker.ThawVDSCommand' return value 'StatusOnlyReturnForXmlRpc [status=StatusForXmlRpc [code=19, message=Guest agent is not responding: Guest agent not available for no
w]]'

Version-Release number of selected component (if applicable):
rhevm-3.6-0.0.2

How reproducible:
100%

Steps to Reproduce:
1.take live snapshot of a Vm with cinder storage

Actual results:
operation throws Exceptions on Logs and fails on UI

Expected results:
operation should throw log Warnings and display a warning on UI

Additional info:

Comment 1 Allon Mureinik 2015-11-17 15:06:14 UTC
You can't freeze or thaw without a guest agent. Perhaps it's worth having a warning audit log, but let's leave it to PM to decide.

Comment 2 Daniel Erez 2015-11-17 15:54:56 UTC
(In reply to Allon Mureinik from comment #1)
> You can't freeze or thaw without a guest agent. Perhaps it's worth having a
> warning audit log, but let's leave it to PM to decide.

Indeed, this is by design. There should be an event log similar to: "Failed to thaw guest filesystems on VM... The filesystems might be unresponsive until the VM is restarted.". @Ori - can you please attach a screenshot of the events tab?

Comment 3 Daniel Erez 2015-11-18 07:54:21 UTC
Created attachment 1095892 [details]
cinder - live snapshot without guest

Comment 4 Daniel Erez 2015-11-18 07:56:19 UTC
Attached a screenshot with the expected events errors regarding freeze/thaw when invoking live snapshot on a VM without a guest agent. @Ori - did you get anything different?

Comment 5 Ori Gofen 2015-11-18 12:28:38 UTC
@Derez - That's the error message alright, note that when using a non cinder storage images this error messages become ui warnings, so, in order to keep the behavior consistent, this  should be taken care of.

Comment 6 Daniel Erez 2015-11-18 13:18:32 UTC
(In reply to Ori Gofen from comment #5)
> @Derez - That's the error message alright, note that when using a non cinder
> storage images this error messages become ui warnings, so, in order to keep
> the behavior consistent, this  should be taken care of.

I don't follow.. What do you mean by consistent? In images we don't use freeze/thaw.

Comment 7 Ori Gofen 2015-11-18 13:40:55 UTC
@derez I meant that the errors being thrown should lower level to warning, because that is the general behavior

Comment 8 Daniel Erez 2015-11-18 15:50:37 UTC
The level is higher since failure in thaw is usually more severe than freeze (failure in thaw means that the VM is still frozen and unresponsive).

@Yaniv/Allon - what do you think, should we lower the level to warning? See the attached screenshot for reference: https://bugzilla.redhat.com/attachment.cgi?id=1095892

Comment 9 Allon Mureinik 2015-11-18 16:19:37 UTC
The error level is currently misleading - this isn't an important failure - the thawing failed since the freezing didn't succeed.
Let's reduce it to a warning.

Comment 10 Sandro Bonazzola 2015-12-23 13:44:15 UTC
oVirt 3.6.2 RC1 has been released for testing, moving to ON_QA

Comment 11 Natalie Gavrielov 2015-12-30 18:08:04 UTC
Daniel,

The only thing that's changed here is the event: "Failed to thaw guest filesystems on VM ..." (from error to warning)?

Anything else? messages in engine.log?

Comment 12 Daniel Erez 2015-12-31 09:41:23 UTC
(In reply to Natalie Gavrielov from comment #11)
> Daniel,
> 
> The only thing that's changed here is the event: "Failed to thaw guest
> filesystems on VM ..." (from error to warning)?

Indeed. That's was the only issue. See https://bugzilla.redhat.com/show_bug.cgi?id=1282796#c9

> 
> Anything else? messages in engine.log?

Comment 13 Natalie Gavrielov 2016-01-04 19:10:00 UTC
Verified using: rhevm-3.6.2-0.1.el6.noarch