Description of problem: python-sdk Example script "list_disk_snapshots.py" fails to work with a disk which belongs to a VM based on template (not cloned from template). Trying to list disk snapshots for a disk which belongs to a VM based on template, results with following ERROR: [root@storage-ge5-vdsm3 examples]# python3 list_disk_snapshots.py -c engine c2e70b36-cb46-4899-bc8d-29da86bf5306 Traceback (most recent call last): File "list_disk_snapshots.py", line 84, in <module> child = snapshots.pop(parent) KeyError: None The reason for the failure seems to be that engine does not return the template disk in the disk snapshots list:] GET https://storage-ge-05.scl.lab.tlv.redhat.com/ovirt-engine/api/storagedomains/3d12969c-d21a-4a1d-8587-5425f5d031bd/disksnapshots?include_active <disk_snapshots> <disk_snapshot href="/ovirt-engine/api/storagedomains/3d12969c-d21a-4a1d-8587-5425f5d031bd/disksnapshots/d6d321f5-516b-4ecc-97a9-a74c6625bbdb" id="d6d321f5-516b-4ecc-97a9-a74c6625bbdb"> <name>latest-rhel-guest-image-8.2-infra</name> <description>latest-rhel-guest-image-8.2-infra (91fc53b)</description> <actual_size>1073741824</actual_size> <alias>latest-rhel-guest-image-8.2-infra</alias> <backup>none</backup> <content_type>data</content_type> <format>cow</format> <propagate_errors>false</propagate_errors> <provisioned_size>10737418240</provisioned_size> <qcow_version>qcow2_v3</qcow_version> <shareable>false</shareable> <sparse>true</sparse> <status>ok</status> <storage_type>image</storage_type> <total_size>0</total_size> <wipe_after_delete>false</wipe_after_delete> <disk_profile href="/ovirt-engine/api/diskprofiles/372dd64a-4046-4f69-a8a1-a15bca4aaaef" id="372dd64a-4046-4f69-a8a1-a15bca4aaaef"/> <quota href="/ovirt-engine/api/datacenters/acecc970-b3ce-4907-ac74-0999e04b4549/quotas/ea96fcac-a1b5-4d1b-b20a-f5e625aa10a2" id="ea96fcac-a1b5-4d1b-b20a-f5e625aa10a2"/> <storage_domain href="/ovirt-engine/api/storagedomains/3d12969c-d21a-4a1d-8587-5425f5d031bd" id="3d12969c-d21a-4a1d-8587-5425f5d031bd"/> <storage_domains> <storage_domain href="/ovirt-engine/api/storagedomains/3d12969c-d21a-4a1d-8587-5425f5d031bd" id="3d12969c-d21a-4a1d-8587-5425f5d031bd"/> </storage_domains> <disk href="/ovirt-engine/api/disks/c2e70b36-cb46-4899-bc8d-29da86bf5306" id="c2e70b36-cb46-4899-bc8d-29da86bf5306"/> <parent href="/ovirt-engine/api/storagedomains/3d12969c-d21a-4a1d-8587-5425f5d031bd/disksnapshots/dc3070a5-eecb-45f7-b95a-ef4d918f5c70" id="dc3070a5-eecb-45f7-b95a-ef4d918f5c70"/> </disk_snapshot> <disk_snapshot href="/ovirt-engine/api/storagedomains/3d12969c-d21a-4a1d-8587-5425f5d031bd/disksnapshots/dc3070a5-eecb-45f7-b95a-ef4d918f5c70" id="dc3070a5-eecb-45f7-b95a-ef4d918f5c70"> <name>latest-rhel-guest-image-8.2-infra</name> <description>latest-rhel-guest-image-8.2-infra (91fc53b)</description> <actual_size>1073741824</actual_size> <alias>latest-rhel-guest-image-8.2-infra</alias> <backup>none</backup> <content_type>data</content_type> <format>cow</format> <propagate_errors>false</propagate_errors> <provisioned_size>10737418240</provisioned_size> <qcow_version>qcow2_v3</qcow_version> <shareable>false</shareable> <sparse>true</sparse> <status>ok</status> <storage_type>image</storage_type> <total_size>0</total_size> <wipe_after_delete>false</wipe_after_delete> <disk_profile href="/ovirt-engine/api/diskprofiles/372dd64a-4046-4f69-a8a1-a15bca4aaaef" id="372dd64a-4046-4f69-a8a1-a15bca4aaaef"/> <quota href="/ovirt-engine/api/datacenters/acecc970-b3ce-4907-ac74-0999e04b4549/quotas/ea96fcac-a1b5-4d1b-b20a-f5e625aa10a2" id="ea96fcac-a1b5-4d1b-b20a-f5e625aa10a2"/> <snapshot id="75025a3a-07d8-49be-86e4-78c72c1eeed3"/> <storage_domain href="/ovirt-engine/api/storagedomains/3d12969c-d21a-4a1d-8587-5425f5d031bd" id="3d12969c-d21a-4a1d-8587-5425f5d031bd"/> <storage_domains> <storage_domain href="/ovirt-engine/api/storagedomains/3d12969c-d21a-4a1d-8587-5425f5d031bd" id="3d12969c-d21a-4a1d-8587-5425f5d031bd"/> </storage_domains> <disk href="/ovirt-engine/api/disks/c2e70b36-cb46-4899-bc8d-29da86bf5306" id="c2e70b36-cb46-4899-bc8d-29da86bf5306"/> <parent href="/ovirt-engine/api/storagedomains/3d12969c-d21a-4a1d-8587-5425f5d031bd/disksnapshots/6666efad-1b32-4362-8bd4-64c72b17b1df" id="6666efad-1b32-4362-8bd4-64c72b17b1df"/> </disk_snapshot> <disk_snapshot href="/ovirt-engine/api/storagedomains/3d12969c-d21a-4a1d-8587-5425f5d031bd/disksnapshots/6666efad-1b32-4362-8bd4-64c72b17b1df" id="6666efad-1b32-4362-8bd4-64c72b17b1df"> <name>latest-rhel-guest-image-8.2-infra</name> <description>latest-rhel-guest-image-8.2-infra (91fc53b)</description> <actual_size>1073741824</actual_size> <alias>latest-rhel-guest-image-8.2-infra</alias> <backup>none</backup> <content_type>data</content_type> <format>cow</format> <propagate_errors>false</propagate_errors> <provisioned_size>10737418240</provisioned_size> <qcow_version>qcow2_v3</qcow_version> <shareable>false</shareable> <sparse>true</sparse> <status>ok</status> <storage_type>image</storage_type> <total_size>0</total_size> <wipe_after_delete>false</wipe_after_delete> <disk_profile href="/ovirt-engine/api/diskprofiles/372dd64a-4046-4f69-a8a1-a15bca4aaaef" id="372dd64a-4046-4f69-a8a1-a15bca4aaaef"/> <quota href="/ovirt-engine/api/datacenters/acecc970-b3ce-4907-ac74-0999e04b4549/quotas/ea96fcac-a1b5-4d1b-b20a-f5e625aa10a2" id="ea96fcac-a1b5-4d1b-b20a-f5e625aa10a2"/> <snapshot id="7aa882d8-ac0e-430f-8480-443541ff9ff2"/> <storage_domain href="/ovirt-engine/api/storagedomains/3d12969c-d21a-4a1d-8587-5425f5d031bd" id="3d12969c-d21a-4a1d-8587-5425f5d031bd"/> <storage_domains> <storage_domain href="/ovirt-engine/api/storagedomains/3d12969c-d21a-4a1d-8587-5425f5d031bd" id="3d12969c-d21a-4a1d-8587-5425f5d031bd"/> </storage_domains> <disk href="/ovirt-engine/api/disks/c2e70b36-cb46-4899-bc8d-29da86bf5306" id="c2e70b36-cb46-4899-bc8d-29da86bf5306"/> <parent href="/ovirt-engine/api/storagedomains/3d12969c-d21a-4a1d-8587-5425f5d031bd/disksnapshots/231bf9e8-6ee0-46d3-957a-6505a4162109" id="231bf9e8-6ee0-46d3-957a-6505a4162109"/> </disk_snapshot> <disk_snapshot href="/ovirt-engine/api/storagedomains/3d12969c-d21a-4a1d-8587-5425f5d031bd/disksnapshots/a724c554-79b9-47f8-8537-6cad2c56dc1f" id="a724c554-79b9-47f8-8537-6cad2c56dc1f"> <name>latest-rhel-guest-image-8.2-infra</name> <description>latest-rhel-guest-image-8.2-infra (91fc53b)</description> <actual_size>2684354560</actual_size> <alias>latest-rhel-guest-image-8.2-infra</alias> <backup>none</backup> <content_type>data</content_type> <format>cow</format> <propagate_errors>false</propagate_errors> <provisioned_size>10737418240</provisioned_size> <shareable>false</shareable> <sparse>true</sparse> <status>locked</status> <storage_type>image</storage_type> <total_size>0</total_size> <wipe_after_delete>false</wipe_after_delete> <disk_profile href="/ovirt-engine/api/diskprofiles/372dd64a-4046-4f69-a8a1-a15bca4aaaef" id="372dd64a-4046-4f69-a8a1-a15bca4aaaef"/> <quota href="/ovirt-engine/api/datacenters/acecc970-b3ce-4907-ac74-0999e04b4549/quotas/ea96fcac-a1b5-4d1b-b20a-f5e625aa10a2" id="ea96fcac-a1b5-4d1b-b20a-f5e625aa10a2"/> <storage_domain href="/ovirt-engine/api/storagedomains/3d12969c-d21a-4a1d-8587-5425f5d031bd" id="3d12969c-d21a-4a1d-8587-5425f5d031bd"/> <storage_domains> <storage_domain href="/ovirt-engine/api/storagedomains/3d12969c-d21a-4a1d-8587-5425f5d031bd" id="3d12969c-d21a-4a1d-8587-5425f5d031bd"/> </storage_domains> <disk href="/ovirt-engine/api/disks/dfad7cbe-f932-4bbf-b62f-338102510b37" id="dfad7cbe-f932-4bbf-b62f-338102510b37"/> </disk_snapshot> <disk_snapshot href="/ovirt-engine/api/storagedomains/3d12969c-d21a-4a1d-8587-5425f5d031bd/disksnapshots/ebd47707-f800-4fa7-98b9-de0a4432bb8d" id="ebd47707-f800-4fa7-98b9-de0a4432bb8d"> <name>latest-rhel-guest-image-8.2-infra</name> <description>latest-rhel-guest-image-8.2-infra (91fc53b)</description> <actual_size>1073741824</actual_size> <alias>latest-rhel-guest-image-8.2-infra</alias> <backup>none</backup> <content_type>data</content_type> <format>cow</format> <propagate_errors>false</propagate_errors> <provisioned_size>10737418240</provisioned_size> <qcow_version>qcow2_v3</qcow_version> <shareable>false</shareable> <sparse>true</sparse> <status>ok</status> <storage_type>image</storage_type> <total_size>0</total_size> <wipe_after_delete>false</wipe_after_delete> <disk_profile href="/ovirt-engine/api/diskprofiles/372dd64a-4046-4f69-a8a1-a15bca4aaaef" id="372dd64a-4046-4f69-a8a1-a15bca4aaaef"/> <quota href="/ovirt-engine/api/datacenters/acecc970-b3ce-4907-ac74-0999e04b4549/quotas/ea96fcac-a1b5-4d1b-b20a-f5e625aa10a2" id="ea96fcac-a1b5-4d1b-b20a-f5e625aa10a2"/> <storage_domain href="/ovirt-engine/api/storagedomains/3d12969c-d21a-4a1d-8587-5425f5d031bd" id="3d12969c-d21a-4a1d-8587-5425f5d031bd"/> <storage_domains> <storage_domain href="/ovirt-engine/api/storagedomains/3d12969c-d21a-4a1d-8587-5425f5d031bd" id="3d12969c-d21a-4a1d-8587-5425f5d031bd"/> </storage_domains> <disk href="/ovirt-engine/api/disks/b599d18c-78d0-4f7e-844b-dd50efa2881c" id="b599d18c-78d0-4f7e-844b-dd50efa2881c"/> <parent href="/ovirt-engine/api/storagedomains/3d12969c-d21a-4a1d-8587-5425f5d031bd/disksnapshots/231bf9e8-6ee0-46d3-957a-6505a4162109" id="231bf9e8-6ee0-46d3-957a-6505a4162109"/> </disk_snapshot> </disk_snapshots> The parent disksnapshot <parent href="/ovirt-engine/api/storagedomains/3d12969c-d21a-4a1d-8587-5425f5d031bd/disksnapshots/231bf9e8-6ee0-46d3-957a-6505a4162109" id="231bf9e8-6ee0-46d3-957a-6505a4162109"/> Is not included in the list, so the script cannot build the chain properly. We will have to fix engine API to report the templated disk snapshots or use another API to access them. Version-Release number of selected component (if applicable): rhv-release-4.4.4-2 python3-ovirt-engine-sdk4-4.4.7-1.el8ev.x86_64 How reproducible: 100% Steps to Reproduce: 1. Create VM from template by choosing the template in engine UI and then choose "thin" under "storage allocation" in the "resource allocation" tab 2. From a VDSM host run the script with command specified above Actual results: The script fails Expected results: Shouldnt fail
- What is the functionality impact for customers? Customer might use this functionality of listing existing chain of snapshots for a VM disks, without knowing why it will fail on a VM which wasnt cloned from template, but is based on template. The use cases for that can be various, and very important. - The Impact on QE currently is minor. As we are not testing any flows which include the usage of this example script. It was used only as a meaning for verifying "[RFE] Support transferring snapshots using raw format (NBD backend)", and this is where this bug was introduced at first. As a WA, a cloned VM was used for that RFE instead of "template based".
As mentioned in the bug description, in my reproduction I am also seeing that: GET …/api/storagedomains/xxx/disksnapshots?include_active returns the vm disk's snapshot but not the template-disk's snapshot. Trying to understand why, I've followed the code and saw that The API layer invokes: GetAllDiskSnapshotsByStorageDomainIdQuery and that the template disk's snapshot is absent in the result-set of this query. Is this somehow by design? Because it seems like this might be the source of this bug. Eyal can you opine on that? Note that interestingly, the template disk's snapshot is returned as a reference inside the vm disk's snapshot - as it is its parent. The link to it is valid and the template disk's snapshot is successfully fetched when using it.
(In reply to Ori Liel from comment #2) This is an old design decision, just like the hiding the active snapshot, hide the template disk. I'm not sure why these decisions were made, but we cannot change the behavior at this point, since it may break exiting code not expecting the active or template disk snapshots in the results by default. For backup purpose, the backup application must have access to all layers of the chain to create a correct backup. Currently backup application using snapshot based backup need to work really hard to find the disk snapshot of the template. Templates are special case, since you can create a backup that does not include the template, if you remember that the backup depends on the template, and also backup the template separately. In this case, you can restore the template to the storage domain, and then you can restore vm disks based on this template. I don't know if any backup application is doing this, since this makes backup and restore more complicated. I think the best way to solve this is to add another flag like include_template so backup application can get the entire chain in an easy way. This requires similar change that was done for include_active.
Patch which adds 'include_template' flag to the model: https://gerrit.ovirt.org/115601 WIP patch to the engine: https://gerrit.ovirt.org/115664
new patch: https://gerrit.ovirt.org/c/ovirt-engine-api-model/+/116025
Verified on nightly build: vdsm-4.50.0.7-1.el8ev.x86_64 ovirt-engine-4.5.0-743c0a787472.211.el8ev [root@storage-ge3-vdsm1 examples]# python3 list_disk_snapshots.py -c engine ebb84caf-f544-43f7-a31a-70d805e9a5fa [ { "actual_size": 3585912832, "format": "cow", "id": "9803db55-e461-4e9d-9186-f606f7b98eff", "parent": null, "status": "ok" }, { "actual_size": 1073741824, "format": "cow", "id": "05539745-1d77-4ac9-978d-9d234686fb74", "parent": "9803db55-e461-4e9d-9186-f606f7b98eff", "status": "ok" }, { "actual_size": 1073741824, "format": "cow", "id": "747e3bb4-4185-4a62-9072-066012f3bef6", "parent": "05539745-1d77-4ac9-978d-9d234686fb74", "status": "ok" }, { "actual_size": 1073741824, "format": "cow", "id": "a8a6b40e-4cad-45ba-96fe-54d9ab86ec45", "parent": "747e3bb4-4185-4a62-9072-066012f3bef6", "status": "ok" } ] list_disk_snapshots.py SDK script works as expected. Moving to 'Verified'.
This bugzilla is included in oVirt 4.5.0 release, published on April 20th 2022. Since the problem described in this bug report should be resolved in oVirt 4.5.0 release, it has been closed with a resolution of CURRENT RELEASE. If the solution does not work for you, please open a new bug report.