Bug 1381598

Summary: oVirt complains about not having a template in the destination datastore on a disk movement, but it does
Product: [oVirt] ovirt-engine Reporter: nicolas
Component: RestAPIAssignee: Fred Rolland <frolland>
Status: CLOSED WORKSFORME QA Contact: Pavel Stehlik <pstehlik>
Severity: unspecified Docs Contact:
Priority: unspecified    
Version: 4.0.3CC: amureini, bugs, frolland, juan.hernandez, nicolas
Target Milestone: ovirt-4.0.6Flags: amureini: ovirt-4.0.z?
rule-engine: planning_ack?
rule-engine: devel_ack?
rule-engine: testing_ack?
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2016-11-01 12:52:07 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: Storage RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Attachments:
Description Flags
Screenshot of the template none

Description nicolas 2016-10-04 14:03:19 UTC
Description of problem:

We've written a simple script that iterates over a list of VMs and moves their disks to a different backend.

The script is mostly this snippet:

    blocksd = api.storagedomains.get(name=BLOCKFS)

    for vmname in get_vmlist():
        vm = api.vms.get(name=vmname)

        for disk in vm.disks.list():
            log("Moving disk %s to different DS..." % (disk.get_id()))
            action = params.Action(storage_domain=blocksd, async=False)
            disk.move(action=action)
            poormans_wait4unlock(api, disk.get_id()) # This just waits until disk state is ok

When run over a list of VMs that belong to a VmPool, the following exception is thrown:

Traceback (most recent call last):
  File "migrate_vms.py", line 69, in <module>
    disk.move(action=action)
  File "/opt/venv/lib/python2.7/site-packages/ovirtsdk/infrastructure/brokers.py", line 31905, in move
    headers={"Correlation-Id":correlation_id}
  File "/opt/venv/lib/python2.7/site-packages/ovirtsdk/infrastructure/proxy.py", line 122, in request
    persistent_auth=self.__persistent_auth
  File "/opt/venv/lib/python2.7/site-packages/ovirtsdk/infrastructure/connectionspool.py", line 79, in do_request
    persistent_auth)
  File "/opt/venv/lib/python2.7/site-packages/ovirtsdk/infrastructure/connectionspool.py", line 162, in __do_request
    raise errors.RequestError(response_code, response_reason, response_body)
ovirtsdk.infrastructure.errors.RequestError: 
status: 400
reason: Bad Request
detail: Cannot move Virtual Disk. The selected Storage Domain does not contain the VM Template.

However, the destination datastore indeed contains the Template, as we copied it previously. More concretely, I can move a Vm's disk from the oVirt admin manually, but I cannot move the disk with the script.

Version-Release number of selected component (if applicable):

ovirt-engine-sdk-python: 3.6.9.2
ovirt: 4.0.3-1

How reproducible:

Always

Comment 1 Juan Hernández 2016-10-04 15:08:50 UTC
Are you completely sure that you copied the relevant disks of the template to the destination storage domain? How did you do that? Take into account that just copying the template won't work, you have to copy each disk, so they will effectively have the same id. To check in what storage domains are the disks available you can do something like this:

  for disk in template.disks.list():
      for sd in disk.get_storage_domains().get_storage_domain():
          sd = api.storagedomains.get(id=sd.get_id())
          print('%s %s' % (disk.get_id(), sd.get_name()))

That will print something like this:

  12db9d50-00a9-4caa-a519-5a9ffd39c046 mydata
  12db9d50-00a9-4caa-a519-5a9ffd39c046 yourdata

In order to correctly copy the template disks you can also use the API:

  for disk in template.disks.list():
      disk.copy(
         action=params.Action(
             storage_domain=params.StorageDomain(id=sd.get_id()),
         ),
     )

Note also that in your example you are sending the complete representation of the storage domain, when only the id is needed:

  action = params.Action(storage_domain=blocksd, async=False)

Try to do the following instead:

  action = params.Action(
      storage_domain=params.StorageDomain(id=blocksd.get_id()),
      async=False,
  )

Comment 2 nicolas 2016-10-04 16:10:35 UTC
Created attachment 1207258 [details]
Screenshot of the template

As per your description, I would say it's copied correctly. I'm attaching a screenshot of the template view to confirm, however, the template has only one disk and I did the following to copy it to the other datastore:

1) Choose the 'Templates' tab
2) Select the template of interest.
3) Click on the 'Disks' subtab.
4) Choose the origin datastore, and select 'Copy'.
5) Choose the destination datastore and click on 'OK'.

In the screenshot, iscsiceph01 is the origin and ceph02 the destination backends respectively.

I also run the code you included and the result is:

(venv)# python test.py 
77c7e959-3486-413e-af17-d509a41249fc iscsiceph01
77c7e959-3486-413e-af17-d509a41249fc ceph02

Also note that, as I initially said, I can do that operation from within the webadmin. It's just the Python-SDK that fails.

Comment 3 Juan Hernández 2016-10-05 07:44:06 UTC
Thanks for the detailed answer Nicolas, I think you are copying it correctly. I did test this with two NFS storage domains, and it worked correctly. It may be related to the fact that you are using iSCSI and Ceph.

Allon, can you take a look, please?

Comment 4 nicolas 2016-10-05 08:57:16 UTC
Note that this works when VMs are not inside a VmPool. I migrated lots of singular VMs without any issue - it's just VmPool machines that have this problem.

Comment 5 Allon Mureinik 2016-10-10 13:20:40 UTC
Fred, can you take a look at this please?

Comment 6 Fred Rolland 2016-10-31 15:03:06 UTC
Nicolas, are you working with cinder ?

Comment 7 nicolas 2016-10-31 17:21:19 UTC
No, all of them are iSCSI based datastores, some of them "pure" iSCSI and some of them acting as a gateway to a Ceph backend, but even those have copies of the templates.

Comment 8 nicolas 2016-11-01 12:15:36 UTC
I tried running the script again and now it works with no issues.

The only change we've done recently is upgrading from 4.0.3 to 4.0.4, however I'm unable to find some fix in the CHANGELOG referencing templates or vmpools that could affect (at least not directly). I'm 100% sure the migration script was not changed.

Comment 9 Allon Mureinik 2016-11-01 12:52:07 UTC
Closing for now, as neither side can reproduce. If this occurs again, plesae feel free to reopen!