Bug 1496704

Summary: Python-SDK doesn't seem to handle quota information on disks
Product: [oVirt] ovirt-engine Reporter: nicolas
Component: RestAPIAssignee: Andrej Krejcir <akrejcir>
Status: CLOSED CURRENTRELEASE QA Contact: Radim Hrazdil <rhrazdil>
Severity: medium Docs Contact:
Priority: medium    
Version: 4.1.0.2CC: akrejcir, amureini, bugs, juan.hernandez, lsvaty, mgoldboi
Target Milestone: ovirt-4.2.0Flags: rule-engine: ovirt-4.2+
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Cause: A quota does not have to be defined for every storage domain. When a disk is moved to a storage domain where its current quota is not defined and no new quota is provided, the default unlimited quota is assigned to the disk automatically. In the UI, the user has to choose a new quota to assign to the moved disk. But in the REST API a separate call had to be used to assign the new quota. Consequence: Users of the REST API could forget to assign a new quota after the disk was moved. Fix: The REST commands for moving and copying disks accept a quota and a disk profile as parameters. (Disk profiles have the same problem as quotas.) Result: Only one REST call is needed to move a disk and assign a new quota and disk profile.
Story Points: ---
Clone Of: Environment:
Last Closed: 2017-12-20 11:15:59 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: SLA RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description nicolas 2017-09-28 07:43:57 UTC
Description of problem:

When moving a disk, most of the times the correct quota is not shown when clicked on the 'Move' button, and the first one of the list is selected instead. I'm not sure of this, but I suspect that some disks don't have a quota assigned. The reason why I suspect this is because I've run a python script to get a quota of one of the affected disks and it returns NoneType:

vms_serv = sys_serv.vms_service()
vm = vms_serv.list(search='name=affected_machine')[0]
disks = conn.follow_link(vm.disk_attachments)[0]
disk = disks.disk

In [16]: disk.id
Out[16]: 'ba55afc6-e22d-4d2e-b532-833b7ccfd4c2'

In [17]: disk.quota

In [18]: type(disk.quota)
Out[18]: NoneType

Version-Release number of selected component (if applicable):

Just upgraded to 4.1.6.2.

How reproducible:

Steps to Reproduce:
1. Open the Virtual Machines tab
2. Choose a VM
3. Open the Disks subtab and choose a disk
4. Click on 'Move'

Actual results:

The disk shows the first quota available in the list, although I'm sure that when I deployed the VM it had a different quota.

Expected results:

The former quota should be shown.


Additional info:

Comment 1 nicolas 2017-09-28 11:49:34 UTC
I found a way to reproduce this. It seems that it happens when a VmPool is created with the Python-SDK and then you try to move the disk.

In [1]: sys_serv = conn.system_service()

In [2]: vmp_serv = sys_serv.vm_pools_service()

In [3]: clus_serv = sys_serv.clusters_service()

In [4]: tpl_serv = sys_serv.templates_service()

In [5]: tplsv = tpl_serv.template_service(id='023e4e99-5f18-...-...-b00b07ce9219')

In [6]: 

In [6]: cluster = clus_serv.list(search='name=...')[0]

In [7]: template = tplsv.get()

In [8]: newpool = types.VmPool(name='TEST',
   ...:                        cluster=cluster,
   ...:                        template=template,
   ...:                        max_user_vms=1,
   ...:                        size=1,
   ...:                        type=types.VmPoolType.MANUAL)

In [9]: vmp_serv.add(newpool)
Out[9]: <ovirtsdk4.types.VmPool at 0x18fd490>

This creates a VmPool with one VM.

If you then follow the steps described in the description, quota shows up unset. Seems like the VmPool.add() call didn't respect the quota set on the template. This is a big issue for us as around 80% of all machines (> 1000) are created this way.

However, this quota issue doesn't reproduce if you create the VmPool within the Webadmin. Then the quota is respected when moving the disk.

I'm not sure if this is the only case when this happens, I'll research a bit further and add new info if I find something new.

Comment 2 nicolas 2017-09-29 09:11:29 UTC
I tried to run another script, one that just migrates a disk to a different storage domain using Python-SDK, and the quota is lost after migration.

So this seems to be a Python-SDK issue which seems not to handle assigned quotas, losing any information when moving a disk or deploying a VmPool.

Not sure it this involves something else, but that would explain why we lose the quota information from time to time, as we use the Python-SDK for a lot of maintenance tasks (balancing disks betweek storage domains or deploy vmpools, for example).

Comment 3 nicolas 2017-09-29 09:19:21 UTC
ovirt-engine-sdk-python version is: 4.1.6

Comment 4 Ondra Machacek 2017-10-02 07:43:37 UTC
This is issue in REST API, it's where the default values are 'computed'. With Python SDK we can just specify values explicitly.

Comment 5 nicolas 2017-10-02 08:36:58 UTC
I knew I already notified this a time ago, just couldn't find it up until now.

Seems that a year ago I opened BZ #1379802 which is closed as 'CURRENTRELEASE' but evidently this isn't solved. I just wanted to make you know in terms of additional information FWIW.

Comment 6 Andrej Krejcir 2017-10-20 11:18:41 UTC
Currently the disk quota behaves this way: 

When a quota is defined only for some storage domains and not others and
the disk is moved to a storage domain where it is not defined, then no quota is assigned to the disk.
When moving a disk using the UI, the user has to select some quota from the list of available quotas in the new storage domain. 
But when using the SDK, it is up to the caller to correctly set a new quota on the moved disk after the move has finished.


I will change the API, so the move action accepts new quota as a parameter. The backend supports it and the UI uses it already.

Comment 7 nicolas 2017-10-20 11:58:53 UTC
Currently, how can you get/set the quota information of a disk using Python-SDK?

The 'quota' attribute of a types.Disk object is None although the disk has a quota set (at lease I can see it in the AdminUI).

In [1]: vm
Out[1]: <ovirtsdk4.types.Vm at 0x2b41c50>

In [2]: das = conn.follow_link(vm.disk_attachments)

In [3]: da = das[0]

In [4]: d = da.disk

In [5]: d
Out[5]: <ovirtsdk4.types.Disk at 0x2cefe10>

In [6]: d.quota

In [7]: type(d.quota)
Out[7]: NoneType

In [8]: conn.follow_link(d.quota)
---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
<ipython-input-23-90896641f840> in <module>()
----> 1 conn.follow_link(d.quota)

/opt/venv/lib/python2.7/site-packages/ovirtsdk4/__init__.pyc in follow_link(self, obj)
    706         # Check that the "href" attribute has a values, as it is needed
    707         # in order to retrieve the representation of the object:
--> 708         href = obj.href
    709         if href is None:
    710             raise Error(

AttributeError: 'NoneType' object has no attribute 'href'

Comment 8 Juan Hernández 2017-10-20 12:25:01 UTC
(In reply to nicolas from comment #7)
> In [4]: d = da.disk
> 
> In [5]: d
> Out[5]: <ovirtsdk4.types.Disk at 0x2cefe10>
> 
> In [6]: d.quota

At this ^ point "d" is a link to the disk, you need to follow that link before trying to use any attribute other than "id" and "href":

  disk = connection.follow_link(da.disk)

Once you have the disk, then you can try to follow the link to the quota:

  quota = connection.follow_link(disk.quota)

However I just saw that we aren't building that link correctly. This is what we return currently:

  <quota id="9d36e515-a2a4-483d-8746-b048725963ab"/>

As you can see there is no "href" attribute there, which is what the "follow_link" method needs to retrieve the information. Andrej, can you take care of this as well?

You can work around this by taking the id and retrieving the explicitly directly from the collection of quota:

  # Get the reference to the data center service:
  dc_service = ...

  # Get the reference to the service that manages the collection of
  # quotas of the data center:
  quotas_service = dc_service.quotas_service

  # Get the reference to the service that manages the quota of the
  # disk:
  quota_service = quotas_service(quota.id)

  # Retrieve the details of the quota:
  quota = quota_service.get()

Comment 9 Yaniv Kaul 2017-10-29 11:20:27 UTC
Target milestone?

Comment 12 Radim Hrazdil 2017-12-14 11:04:43 UTC
Verified that steps given in comment #7 doesn't lead to the error, therefore, as Juan says in comment #8, href link is provided. 

After following bug description and comment #1, the disk quota is by default set to unlimited, as is described in documentation. 

RHVM version 4.2.0-6, SDk 4.2.2

Comment 13 Sandro Bonazzola 2017-12-20 11:15:59 UTC
This bugzilla is included in oVirt 4.2.0 release, published on Dec 20th 2017.

Since the problem described in this bug report should be
resolved in oVirt 4.2.0 release, published on Dec 20th 2017, it has been closed with a resolution of CURRENT RELEASE.

If the solution does not work for you, please open a new bug report.