Bug 1036341 - [guest-agent] [vss] creating a shadow copy using qemu-guest-agent (with vss support) is failing on a windows server
Summary: [guest-agent] [vss] creating a shadow copy using qemu-guest-agent (with vss s...
Keywords:
Status: CLOSED CURRENTRELEASE
Alias: None
Product: Red Hat Enterprise Linux 7
Classification: Red Hat
Component: virtio-win
Version: 7.0
Hardware: x86_64
OS: Unspecified
high
high
Target Milestone: rc
: 7.0
Assignee: Gal Hammer
QA Contact: Virtualization Bugs
URL:
Whiteboard: infra
: 1036342 1036343 (view as bug list)
Depends On:
Blocks: 1073208 1074613
TreeView+ depends on / blocked
 
Reported: 2013-12-01 08:36 UTC by Elad
Modified: 2014-06-18 08:57 UTC (History)
26 users (show)

Fixed In Version: qemu-ga-win-7.0-7
Doc Type: Release Note
Doc Text:
The current implementation of the QEMU guest agent VSS provider should only be used as a mean to freeze the file system. It doesn't support the creation of shadows copies within the guest.
Clone Of:
Environment:
Last Closed: 2014-06-13 10:47:17 UTC
Target Upstream Version:


Attachments (Terms of Use)
Screenshots, event log and engine.log (304.03 KB, application/x-gzip)
2013-12-01 08:36 UTC, Elad
no flags Details


Links
System ID Priority Status Summary Last Updated
Red Hat Bugzilla 1046075 None None None 2019-02-18 02:56:19 UTC
Red Hat Bugzilla 1066886 None CLOSED [virtio-win][qemu-ga] Error from QEMU Guest Agent VSS Provider: RegisterProvider failed.(Error: 80042303) (null) 2019-02-18 02:56:20 UTC

Internal Links: 1046075 1066886

Description Elad 2013-12-01 08:36:34 UTC
Created attachment 831129 [details]
Screenshots, event log and engine.log

Description of problem:
I tried to create a shadow copy of a single disk on a windows VM, using qemu-guest-agent via vshadow tool.
This attempt failed with this error (on the windows guest):

Error during the last asynchronous operation.
- Returned HRESULT = 0x8004230f
- Error text: VSS_E_UNEXPECTED_PROVIDER_ERROR
- Please re-run VSHADOW.EXE with the /tracing option to get more details

Version-Release number of selected component (if applicable):
is26
Windows server 2008 64bit
rhevm-3.3.0-0.37.beta1.el6ev.noarch
qemu-ga-win-6.5-4

How reproducible:
100%

Steps to Reproduce:
1. install a windows VM with 3 disks and rhevm-guest-tools with the mentioned qemu-guest-agent version which support vss for windows.
2. install Volume Shadow Copy Service SDK 7.2 on the guest. Check the providers list:
C:\Program Files (x86)\Microsoft\VSSSDK72\TestApps\vshadow\bin\obj-fre\amd64>vss
admin list providers
qemu-guest-agent is suppose to be the main provider:

vssadmin 1.1 - Volume Shadow Copy Service administrative command-line tool
(C) Copyright 2001-2005 Microsoft Corp.

Provider name: 'QEMU Guest Agent VSS Provider'
   Provider type: Software
   Provider Id: {3629d4ed-ee09-4e0e-9a5c-6d8ba2872aef}
   Version: 0.12.1

Provider name: 'Microsoft Software Shadow Copy provider 1.0'
   Provider type: System
   Provider Id: {b5946137-7b9f-4925-af80-51abd60b20d5}
   Version: 1.0.0.7

3. create a shadow copy using vshadow command on the guest:

C:\Program Files (x86)\Microsoft\VSSSDK72\TestApps\vshadow\bin\obj-fre\amd64>vsh
adow.exe e:


Actual results:
Shadow copy creation fails on the guest with this error on the event logs:

Volume Shadow Copy Service error: Error calling a routine on a Shadow Copy Provider {3629d4ed-ee09-4e0e-9a5c-6d8ba2872aef}. Routine details GetSnapshotProperties({7f97ecd6-b278-4057-abbf-d67cd1db2334}) [hr = 0x80042308].

Expected results:
It should be possible to create a shadow copy of a disk on a windows machine using the qemu-guest-agent as a vss provider.

Additional info: Screenshots, event log and engine.log

Comment 1 Elad 2013-12-01 08:42:11 UTC
*** Bug 1036342 has been marked as a duplicate of this bug. ***

Comment 2 Elad 2013-12-01 08:42:14 UTC
*** Bug 1036343 has been marked as a duplicate of this bug. ***

Comment 3 Gal Hammer 2013-12-01 14:14:13 UTC
It seems that this error occurs because the function GetSnapshotProperties is currently not implemented in the QEMU's vss provider (it returns VSS_E_OBJECT_NOT_FOUND).

I guess it is because the vss provider is used only as a mean to freeze the file system and not to create and manage snapshots.

Comment 5 Elad 2013-12-02 07:27:11 UTC
Version-Release number of selected component (if applicable):
is25

Comment 6 Sibiao Luo 2013-12-02 09:57:25 UTC
Reproduce this issue with the same steps as comment #0.
host info:
2.6.32-425.el6.x86_64
qemu-kvm-rhev-0.12.1.2-2.415.el6.x86_64
guest info:
win2008-64bit
virtio-win-prewhql-0.1-74
qemu-ga-win-6.5-4 (qga/installer/x64/qemu-ga-x64.msi)
Volume Shadow Copy Service SDK 7.2 (download from Microsoft http://www.microsoft.com/en-us/download/confirmation.aspx?id=23490)

Steps:
1.install a win2008 64bit windows VM with 3 disks and rhevm-guest-tools with the mentioned qemu-guest-agent version which support vss for windows.
2. install Volume Shadow Copy Service SDK 7.2 on the guest.
Check the providers list:
C:\Program Files (x86)\Microsoft\VSSSDK72\TestApps\vshadow\bin\obj-fre\amd64>
3.create a shadow copy using vshadow command on the guest.

Result:
aftet step 3, there is VSS_E_UNEXPECTED_PROVIDER_ERROR.

C:\Program Files (x86)\Microsoft\VSSSDK72\TestApps\vshadow\bin\obj-fre\amd64>vsh
adow.exe D:

VSHADOW.EXE 2.2 - Volume Shadow Copy sample client
Copyright (C) 2005 Microsoft Corporation. All rights reserved.


(Option: Create shadow copy set)
(Gathering writer metadata...)
(Waiting for the asynchronous operation to finish...)
Initialize writer metadata ...
Discover directly excluded components ...
- Excluding writer 'Shadow Copy Optimization Writer' since it has no selected co
mponents for restore.
- Excluding writer 'BITS Writer' since it has no selected components for restore
.
Discover components that reside outside the shadow set ...
- Component '\System Files' from writer 'System Writer' is excluded from backup
(it requires C:\ in the shadow set)
- Component '\BCD\BCD' from writer 'ASR Writer' is excluded from backup (it requ
ires C:\ in the shadow set)
- Component '\Registry' from writer 'Registry Writer' is excluded from backup (i
t requires C:\ in the shadow set)
- Component '\COM+ REGDB' from writer 'COM+ REGDB Writer' is excluded from backu
p (it requires C:\ in the shadow set)
- Component '\WMI' from writer 'WMI Writer' is excluded from backup (it requires
 C:\ in the shadow set)
Discover all excluded components ...
Discover excluded writers ...
- The writer 'System Writer' is now entirely excluded from the backup:
  (it does not contain any components that can be potentially included in the ba
ckup)
- The writer 'ASR Writer' is now entirely excluded from the backup:
  (the top-level non-selectable component '\BCD\BCD' is an excluded component)
- The writer 'Registry Writer' is now entirely excluded from the backup:
  (it does not contain any components that can be potentially included in the ba
ckup)
- The writer 'COM+ REGDB Writer' is now entirely excluded from the backup:
  (it does not contain any components that can be potentially included in the ba
ckup)
- The writer 'WMI Writer' is now entirely excluded from the backup:
  (it does not contain any components that can be potentially included in the ba
ckup)
Discover explicitly included components ...
Verifying explicitly specified writers/components ...
Select explicitly included components ...
Creating shadow set {259769c6-af96-410c-ab88-5c43d1e935c1} ...
- Adding volume \\?\Volume{97b90ea9-5b4d-11e3-98d5-98e5232569b9}\ [D:\] to the s
hadow set...
Preparing for backup ...
(Waiting for the asynchronous operation to finish...)
(Waiting for the asynchronous operation to finish...)
Creating the shadow (DoSnapshotSet) ...
(Waiting for the asynchronous operation to finish...)
Error during the last asynchronous operation.
- Returned HRESULT = 0x8004230f
- Error text: VSS_E_UNEXPECTED_PROVIDER_ERROR
- Please re-run VSHADOW.EXE with the /tracing option to get more details

My qemu-kvm command line:
# /usr/libexec/qemu-kvm -M pc -S -cpu SandyBridge -enable-kvm -m 2048 -smp 2,sockets=2,cores=1,threads=1 -no-kvm-pit-reinjection -usb -device usb-tablet,id=input0 -name sluo -uuid 990ea161-6b67-47b2-b803-19fb01d30d30 -rtc base=localtime,clock=host,driftfix=slew -drive file=/home/win2008-64-virtio.qcow2,if=none,id=drive-virtio-disk,format=qcow2,cache=none,aio=native,werror=stop,rerror=stop -device virtio-blk-pci,vectors=0,bus=pci.0,addr=0x4,scsi=off,drive=drive-virtio-disk,id=virtio-disk,bootindex=1 -netdev tap,id=hostnet0,vhost=on,script=/etc/qemu-ifup -device virtio-net-pci,netdev=hostnet0,id=virtio-net-pci0,mac=00:01:02:B6:40:21,bus=pci.0,addr=0x5 -device virtio-balloon-pci,id=ballooning,bus=pci.0,addr=0x6 -k en-us -boot menu=on -qmp tcp:0:4444,server,nowait -vnc :2 -spice disable-ticketing,port=5932 -monitor stdio -chardev socket,path=/tmp/qga.sock,server,nowait,id=qga0 -device virtio-serial -device virtserialport,chardev=qga0,name=org.qemu.guest_agent.0 -drive file=/home/my-data-disk1.qcow2,if=none,id=drive-disk1,format=qcow2,cache=none,aio=native,werror=stop,rerror=stop -device virtio-blk-pci,vectors=0,bus=pci.0,addr=0x7,drive=drive-disk1,id=virtio-disk1 -drive file=/home/my-data-disk2.qcow2,if=none,id=drive-disk2,format=qcow2,cache=none,aio=native,werror=stop,rerror=stop -device virtio-blk-pci,vectors=0,bus=pci.0,addr=0x8,drive=drive-disk2,id=virtio-disk2 -drive file=/home/my-data-disk3.qcow2,if=none,id=drive-disk3,format=qcow2,cache=none,aio=native,werror=stop,rerror=stop -device virtio-blk-pci,vectors=0,bus=pci.0,addr=0x9,drive=drive-disk3,id=virtio-disk3

Base on above, mark qa_ack+ to this issue, please correct me if any mistake.

Best Regards,
sluo

Comment 8 Ronen Hod 2014-01-02 16:09:24 UTC
Gal,
It seems as if we can register our VSS provider type as "Software" http://msdn.microsoft.com/en-us/library/aa384594(v=vs.85).aspx ,
and limit ourselves to this specific provider http://msdn.microsoft.com/en-us/library/aa384607(v=vs.85).aspx .

Still, it will not solve the problem of a different requester that tries to use VSS, and runs into our VSS-provider. Maybe our VSS provider should register itself only during the request, so that it will not interfere with other requesters at other times.

Comment 9 Tomoki Sekiyama 2014-01-10 21:18:53 UTC
(In reply to Ronen Hod from comment #8)
I'm investigating this bug. I think CQGAVssProvider::IsVolumeSupported should
return with FALSE (unsupported) when it is kicked from a different requester,
in order to make the qga provider skipped by the algorithm documented at
http://msdn.microsoft.com/en-us/library/aa384607(v=vs.85).aspx .
Now I'm going to experiment whether it works.

Thanks,
Tomoki

Comment 10 Tomoki Sekiyama 2014-01-13 17:37:08 UTC
(In reply to Tomoki Sekiyama from comment #9)
I posted a patchset to fix the interference with other requesters to qemu-devel ML.
http://lists.gnu.org/archive/html/qemu-devel/2014-01/msg01479.html

Thanks,
Tomoki

Comment 11 Gal Hammer 2014-01-15 15:28:51 UTC
Moving to POST based on comment#10.

Comment 12 Qunfang Zhang 2014-01-17 04:55:49 UTC
Hi, Ronen and Vadim 

As this bug is POST already, do we need to wait for the new qemu-ga-win installer to be shipped in our current rhel6.5-z virtio-win errata?  

Thanks
Qunfang

Comment 20 Tomoki Sekiyama 2014-02-25 13:25:32 UTC
FYI, the bugfix patchset is merged to qemu "upstream".
commit ids:
  4c1b8f1e8357d85c613d779596e4079cc581d74f
  ff8adbcfdbbd9c0f2b01ff8a32bc75082fdd9844
  d9e1f574cb6eac0a3a2f97b67d2e7a3ad9c1dc95

Comment 21 Ronen Hod 2014-02-26 13:27:06 UTC
Gal,
please backport, as it is urgently needed for Bug 1046075 & Bug 1066886
Ronen.

Comment 22 Qunfang Zhang 2014-03-04 05:25:26 UTC
(In reply to Ronen Hod from comment #21)
> Gal,
> please backport, as it is urgently needed for Bug 1046075 & Bug 1066886
> Ronen.

Hi, Ronen

Do we plan to include this bug fix together with Bug 1046075 in next virtio-win errata? If so, do you know what is the current status for this bug? Have we started the backport work? 

Thanks!
Qunfang

Comment 23 Gal Hammer 2014-03-09 14:59:35 UTC
(In reply to Ronen Hod from comment #21)
> Gal,
> please backport, as it is urgently needed for Bug 1046075 & Bug 1066886
> Ronen.

The patches were back ported.

Comment 26 Ronen Hod 2014-03-16 14:11:31 UTC
Moving this bug from RHEL6.6 to RHEL7.0 in order to comply with our current process (everything is RHEL7).
It is a virtio-win bug fix, that will be included in the RHEL7 package, so it needs the blocker+ flag.

Comment 27 Xu Han 2014-03-19 10:02:32 UTC
Reproduce this bug with component:
qemu-ga-win-6.5-4
virtio-win-prewhql-0.1-75

Guest: win2008-64bit

Steps:
1.install a win2008 64bit windows VM with 3 disks and rhevm-guest-tools with the mentioned qemu-guest-agent version which support vss for windows.
2. install Volume Shadow Copy Service SDK 7.2 on the guest.
Check the providers list:
C:\Program Files (x86)\Microsoft\VSSSDK72\TestApps\vshadow\bin\obj-fre\amd64>
3.create a shadow copy using vshadow command on the guest.
C:\Program Files (x86)\Microsoft\VSSSDK72\TestApps\vshadow\bin\obj-fre\amd64>vsh
adow.exe D:

Results:
After step 3, there is VSS_E_UNEXPECTED_PROVIDER_ERROR.
--
VSHADOW.EXE 2.2 - Volume Shadow Copy sample client
Copyright (C) 2005 Microsoft Corporation. All rights reserved.
...
(Waiting for the asynchronous operation to finish...)
Error during the last asynchronous operation.
- Returned HRESULT = 0x8004230f
- Error text: VSS_E_UNEXPECTED_PROVIDER_ERROR           <-- *ERROR HERE*
- Please re-run VSHADOW.EXE with the /tracing option to get more details

------
Verify this bug with component:
qemu-ga-win-7.0-7
virtio-win-prewhql-0.1-75

Same steps as above.

Results:
After step 3, volume shadow copy created successfully.
--
VSHADOW.EXE 2.2 - Volume Shadow Copy sample client
Copyright (C) 2005 Microsoft Corporation. All rights reserved.


(Option: Create shadow copy set)
(Gathering writer metadata...)
(Waiting for the asynchronous operation to finish...)
Initialize writer metadata ...
Discover directly excluded components ...
- Excluding writer 'BITS Writer' since it has no selected components for restore.
- Excluding writer 'Shadow Copy Optimization Writer' since it has no selected components for restore.
Discover components that reside outside the shadow set ...
- Component '\System Files' from writer 'System Writer' is excluded from backup (it requires C:\ in the shadow set)
- Component '\BCD\BCD' from writer 'ASR Writer' is excluded from backup (it requires C:\ in the shadow set)
- Component '\Registry' from writer 'Registry Writer' is excluded from backup (it requires C:\ in the shadow set)
- Component '\WMI' from writer 'WMI Writer' is excluded from backup (it requires C:\ in the shadow set)
- Component '\COM+ REGDB' from writer 'COM+ REGDB Writer' is excluded from backup (it requires C:\ in the shadow set)
Discover all excluded components ...
Discover excluded writers ...
- The writer 'System Writer' is now entirely excluded from the backup:
  (it does not contain any components that can be potentially included in the backup)
- The writer 'ASR Writer' is now entirely excluded from the backup:
  (the top-level non-selectable component '\BCD\BCD' is an excluded component)
- The writer 'Registry Writer' is now entirely excluded from the backup:
  (it does not contain any components that can be potentially included in the backup)
- The writer 'WMI Writer' is now entirely excluded from the backup:
  (it does not contain any components that can be potentially included in the backup)
- The writer 'COM+ REGDB Writer' is now entirely excluded from the backup:
  (it does not contain any components that can be potentially included in the backup)
Discover explicitly included components ...
Verifying explicitly specified writers/components ...
Select explicitly included components ...
Creating shadow set {a8116f2f-036b-4dfe-ad7a-87a2754428ba} ...
- Adding volume \\?\Volume{a8acd50d-afc6-11e3-9b7c-082e5f0a1db1}\ [D:\] to the shadow set...
Preparing for backup ... 
(Waiting for the asynchronous operation to finish...)
(Waiting for the asynchronous operation to finish...)
Creating the shadow (DoSnapshotSet) ... 
(Waiting for the asynchronous operation to finish...)
(Waiting for the asynchronous operation to finish...)
Shadow copy set succesfully created.

List of created shadow copies: 


Querying all shadow copies with the SnapshotSetID {a8116f2f-036b-4dfe-ad7a-87a2754428ba} ...

* SNAPSHOT ID = {a40ba4b0-30f2-464a-a483-7f424347ecfa} ...
   - Shadow copy Set: {a8116f2f-036b-4dfe-ad7a-87a2754428ba}
   - Original count of shadow copies = 1
   - Original Volume name: \\?\Volume{a8acd50d-afc6-11e3-9b7c-082e5f0a1db1}\ [D:\]
   - Creation Time: 3/19/2014 5:48:58 PM
   - Shadow copy device name: \\?\GLOBALROOT\Device\HarddiskVolumeShadowCopy1
   - Originating machine: WIN-QS1F0WVNCXL
   - Service machine: WIN-QS1F0WVNCXL
   - Not Exposed
   - Provider id: {b5946137-7b9f-4925-af80-51abd60b20d5}
   - Attributes:  Auto_Release Differential

- Mark all writers as succesfully backed up... 
Completing the backup (BackupComplete) ... 
(Waiting for the asynchronous operation to finish...)
(Waiting for the asynchronous operation to finish...)

Snapshot creation done.

Base on these test results above, this bug has been fixed.

Comment 32 juzhang 2014-03-20 01:57:04 UTC
According to comment27, set this issue as verified.

Comment 33 Ludek Smid 2014-06-13 10:47:17 UTC
This request was resolved in Red Hat Enterprise Linux 7.0.

Contact your manager or support representative in case you have further questions about the request.


Note You need to log in before you can comment on or make changes to this bug.