Bug 1282796 - [Cinder] ThawVDSCommand fails when taking live snapshot without a guest agent
[Cinder] ThawVDSCommand fails when taking live snapshot without a guest agent
Status: CLOSED CURRENTRELEASE
Product: ovirt-engine
Classification: oVirt
Component: BLL.Storage (Show other bugs)
3.6.0.2
Unspecified Unspecified
unspecified Severity medium (vote)
: ovirt-3.6.2
: 3.6.2
Assigned To: Daniel Erez
Natalie Gavrielov
:
Depends On:
Blocks:
  Show dependency treegraph
 
Reported: 2015-11-17 07:59 EST by Ori Gofen
Modified: 2016-05-25 21:50 EDT (History)
7 users (show)

See Also:
Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed: 2016-02-18 06:20:41 EST
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: Storage
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---
rule-engine: ovirt‑3.6.z+
ylavi: planning_ack+
tnisan: devel_ack+
rule-engine: testing_ack+


Attachments (Terms of Use)
Logs (7.18 MB, text/plain)
2015-11-17 07:59 EST, Ori Gofen
no flags Details
cinder - live snapshot without guest (46.32 KB, image/png)
2015-11-18 02:54 EST, Daniel Erez
no flags Details


External Trackers
Tracker ID Priority Status Summary Last Updated
oVirt gerrit 49115 master MERGED core: lowering priority of thaw and freeze failure events Never
oVirt gerrit 49116 master MERGED core: correct AuditLogType on ThawVmCommand Never
oVirt gerrit 49164 ovirt-engine-3.6 MERGED core: correct AuditLogType on ThawVmCommand Never
oVirt gerrit 49165 ovirt-engine-3.6 MERGED core: lowering priority of thaw and freeze failure events Never

  None (edit)
Description Ori Gofen 2015-11-17 07:59:00 EST
Created attachment 1095481 [details]
Logs

Description of problem:
Taking live snapshot of a Vm with Cinder volumes (ceph backend) fails on logs and UI, it looks like that operation fails due to failure to find a guestagent on the Vm.
2015-11-17 14:45:48,990 INFO  [org.ovirt.engine.core.bll.storage.CreateCinderSnapshotCommand] (DefaultQuartzScheduler_Worker-1) [108f473e] Ending command 'org.ovirt.engine.co
re.bll.storage.CreateCinderSnapshotCommand' successfully.
2015-11-17 14:45:53,626 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.ThawVDSCommand] (org.ovirt.thread.pool-7-thread-44) [42818ef1] Failed in 'ThawVDS' method
2015-11-17 14:45:53,631 ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-7-thread-44) [42818ef1] Correlation ID: null, Call
 Stack: null, Custom Event ID: -1, Message: VDSM fury66.tlv.redhat.com command failed: Guest agent is not responding: Guest agent not available for now
2015-11-17 14:45:53,631 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ThawVDSCommand] (org.ovirt.thread.pool-7-thread-44) [42818ef1] Command 'org.ovirt.engine.core.vdsbrok
er.vdsbroker.ThawVDSCommand' return value 'StatusOnlyReturnForXmlRpc [status=StatusForXmlRpc [code=19, message=Guest agent is not responding: Guest agent not available for no
w]]'

Version-Release number of selected component (if applicable):
rhevm-3.6-0.0.2

How reproducible:
100%

Steps to Reproduce:
1.take live snapshot of a Vm with cinder storage

Actual results:
operation throws Exceptions on Logs and fails on UI

Expected results:
operation should throw log Warnings and display a warning on UI

Additional info:
Comment 1 Allon Mureinik 2015-11-17 10:06:14 EST
You can't freeze or thaw without a guest agent. Perhaps it's worth having a warning audit log, but let's leave it to PM to decide.
Comment 2 Daniel Erez 2015-11-17 10:54:56 EST
(In reply to Allon Mureinik from comment #1)
> You can't freeze or thaw without a guest agent. Perhaps it's worth having a
> warning audit log, but let's leave it to PM to decide.

Indeed, this is by design. There should be an event log similar to: "Failed to thaw guest filesystems on VM... The filesystems might be unresponsive until the VM is restarted.". @Ori - can you please attach a screenshot of the events tab?
Comment 3 Daniel Erez 2015-11-18 02:54 EST
Created attachment 1095892 [details]
cinder - live snapshot without guest
Comment 4 Daniel Erez 2015-11-18 02:56:19 EST
Attached a screenshot with the expected events errors regarding freeze/thaw when invoking live snapshot on a VM without a guest agent. @Ori - did you get anything different?
Comment 5 Ori Gofen 2015-11-18 07:28:38 EST
@Derez - That's the error message alright, note that when using a non cinder storage images this error messages become ui warnings, so, in order to keep the behavior consistent, this  should be taken care of.
Comment 6 Daniel Erez 2015-11-18 08:18:32 EST
(In reply to Ori Gofen from comment #5)
> @Derez - That's the error message alright, note that when using a non cinder
> storage images this error messages become ui warnings, so, in order to keep
> the behavior consistent, this  should be taken care of.

I don't follow.. What do you mean by consistent? In images we don't use freeze/thaw.
Comment 7 Ori Gofen 2015-11-18 08:40:55 EST
@derez I meant that the errors being thrown should lower level to warning, because that is the general behavior
Comment 8 Daniel Erez 2015-11-18 10:50:37 EST
The level is higher since failure in thaw is usually more severe than freeze (failure in thaw means that the VM is still frozen and unresponsive).

@Yaniv/Allon - what do you think, should we lower the level to warning? See the attached screenshot for reference: https://bugzilla.redhat.com/attachment.cgi?id=1095892
Comment 9 Allon Mureinik 2015-11-18 11:19:37 EST
The error level is currently misleading - this isn't an important failure - the thawing failed since the freezing didn't succeed.
Let's reduce it to a warning.
Comment 10 Sandro Bonazzola 2015-12-23 08:44:15 EST
oVirt 3.6.2 RC1 has been released for testing, moving to ON_QA
Comment 11 Natalie Gavrielov 2015-12-30 13:08:04 EST
Daniel,

The only thing that's changed here is the event: "Failed to thaw guest filesystems on VM ..." (from error to warning)?

Anything else? messages in engine.log?
Comment 12 Daniel Erez 2015-12-31 04:41:23 EST
(In reply to Natalie Gavrielov from comment #11)
> Daniel,
> 
> The only thing that's changed here is the event: "Failed to thaw guest
> filesystems on VM ..." (from error to warning)?

Indeed. That's was the only issue. See https://bugzilla.redhat.com/show_bug.cgi?id=1282796#c9

> 
> Anything else? messages in engine.log?
Comment 13 Natalie Gavrielov 2016-01-04 14:10:00 EST
Verified using: rhevm-3.6.2-0.1.el6.noarch

Note You need to log in before you can comment on or make changes to this bug.