Bug 1496704 - Python-SDK doesn't seem to handle quota information on disks
Summary: Python-SDK doesn't seem to handle quota information on disks
Keywords:
Status: CLOSED CURRENTRELEASE
Alias: None
Product: ovirt-engine
Classification: oVirt
Component: RestAPI
Version: 4.1.0.2
Hardware: Unspecified
OS: Unspecified
medium
medium
Target Milestone: ovirt-4.2.0
: ---
Assignee: Andrej Krejcir
QA Contact: Radim Hrazdil
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2017-09-28 07:43 UTC by nicolas
Modified: 2017-12-20 11:15 UTC (History)
6 users (show)

Fixed In Version:
Clone Of:
Environment:
Last Closed: 2017-12-20 11:15:59 UTC
oVirt Team: SLA
Embargoed:
rule-engine: ovirt-4.2+


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
oVirt gerrit 83108 0 master MERGED core: Create QuotaValidator class and use it 2017-11-13 16:21:29 UTC
oVirt gerrit 83109 0 master MERGED core: Validate quota on disk move or copy 2017-11-13 16:21:32 UTC
oVirt gerrit 83110 0 master MERGED Add quota and diskProfile parameters to Move and Copy disk actions. 2017-11-07 13:10:35 UTC
oVirt gerrit 83220 0 master MERGED restapi: Move and Copy disk actions expect quota as a parameter 2017-11-13 16:21:35 UTC
oVirt gerrit 83706 0 model_4.2 MERGED Add quota and diskProfile parameters to Move and Copy disk actions. 2017-11-07 13:11:02 UTC
oVirt gerrit 83752 0 master MERGED restapi: Update to model 4.2.24 and metamodel 1.2.12 2017-11-08 11:02:43 UTC
oVirt gerrit 83762 0 master MERGED Update to model 4.2.24 2017-11-08 10:52:39 UTC

Description nicolas 2017-09-28 07:43:57 UTC
Description of problem:

When moving a disk, most of the times the correct quota is not shown when clicked on the 'Move' button, and the first one of the list is selected instead. I'm not sure of this, but I suspect that some disks don't have a quota assigned. The reason why I suspect this is because I've run a python script to get a quota of one of the affected disks and it returns NoneType:

vms_serv = sys_serv.vms_service()
vm = vms_serv.list(search='name=affected_machine')[0]
disks = conn.follow_link(vm.disk_attachments)[0]
disk = disks.disk

In [16]: disk.id
Out[16]: 'ba55afc6-e22d-4d2e-b532-833b7ccfd4c2'

In [17]: disk.quota

In [18]: type(disk.quota)
Out[18]: NoneType

Version-Release number of selected component (if applicable):

Just upgraded to 4.1.6.2.

How reproducible:

Steps to Reproduce:
1. Open the Virtual Machines tab
2. Choose a VM
3. Open the Disks subtab and choose a disk
4. Click on 'Move'

Actual results:

The disk shows the first quota available in the list, although I'm sure that when I deployed the VM it had a different quota.

Expected results:

The former quota should be shown.


Additional info:

Comment 1 nicolas 2017-09-28 11:49:34 UTC
I found a way to reproduce this. It seems that it happens when a VmPool is created with the Python-SDK and then you try to move the disk.

In [1]: sys_serv = conn.system_service()

In [2]: vmp_serv = sys_serv.vm_pools_service()

In [3]: clus_serv = sys_serv.clusters_service()

In [4]: tpl_serv = sys_serv.templates_service()

In [5]: tplsv = tpl_serv.template_service(id='023e4e99-5f18-...-...-b00b07ce9219')

In [6]: 

In [6]: cluster = clus_serv.list(search='name=...')[0]

In [7]: template = tplsv.get()

In [8]: newpool = types.VmPool(name='TEST',
   ...:                        cluster=cluster,
   ...:                        template=template,
   ...:                        max_user_vms=1,
   ...:                        size=1,
   ...:                        type=types.VmPoolType.MANUAL)

In [9]: vmp_serv.add(newpool)
Out[9]: <ovirtsdk4.types.VmPool at 0x18fd490>

This creates a VmPool with one VM.

If you then follow the steps described in the description, quota shows up unset. Seems like the VmPool.add() call didn't respect the quota set on the template. This is a big issue for us as around 80% of all machines (> 1000) are created this way.

However, this quota issue doesn't reproduce if you create the VmPool within the Webadmin. Then the quota is respected when moving the disk.

I'm not sure if this is the only case when this happens, I'll research a bit further and add new info if I find something new.

Comment 2 nicolas 2017-09-29 09:11:29 UTC
I tried to run another script, one that just migrates a disk to a different storage domain using Python-SDK, and the quota is lost after migration.

So this seems to be a Python-SDK issue which seems not to handle assigned quotas, losing any information when moving a disk or deploying a VmPool.

Not sure it this involves something else, but that would explain why we lose the quota information from time to time, as we use the Python-SDK for a lot of maintenance tasks (balancing disks betweek storage domains or deploy vmpools, for example).

Comment 3 nicolas 2017-09-29 09:19:21 UTC
ovirt-engine-sdk-python version is: 4.1.6

Comment 4 Ondra Machacek 2017-10-02 07:43:37 UTC
This is issue in REST API, it's where the default values are 'computed'. With Python SDK we can just specify values explicitly.

Comment 5 nicolas 2017-10-02 08:36:58 UTC
I knew I already notified this a time ago, just couldn't find it up until now.

Seems that a year ago I opened BZ #1379802 which is closed as 'CURRENTRELEASE' but evidently this isn't solved. I just wanted to make you know in terms of additional information FWIW.

Comment 6 Andrej Krejcir 2017-10-20 11:18:41 UTC
Currently the disk quota behaves this way: 

When a quota is defined only for some storage domains and not others and
the disk is moved to a storage domain where it is not defined, then no quota is assigned to the disk.
When moving a disk using the UI, the user has to select some quota from the list of available quotas in the new storage domain. 
But when using the SDK, it is up to the caller to correctly set a new quota on the moved disk after the move has finished.


I will change the API, so the move action accepts new quota as a parameter. The backend supports it and the UI uses it already.

Comment 7 nicolas 2017-10-20 11:58:53 UTC
Currently, how can you get/set the quota information of a disk using Python-SDK?

The 'quota' attribute of a types.Disk object is None although the disk has a quota set (at lease I can see it in the AdminUI).

In [1]: vm
Out[1]: <ovirtsdk4.types.Vm at 0x2b41c50>

In [2]: das = conn.follow_link(vm.disk_attachments)

In [3]: da = das[0]

In [4]: d = da.disk

In [5]: d
Out[5]: <ovirtsdk4.types.Disk at 0x2cefe10>

In [6]: d.quota

In [7]: type(d.quota)
Out[7]: NoneType

In [8]: conn.follow_link(d.quota)
---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
<ipython-input-23-90896641f840> in <module>()
----> 1 conn.follow_link(d.quota)

/opt/venv/lib/python2.7/site-packages/ovirtsdk4/__init__.pyc in follow_link(self, obj)
    706         # Check that the "href" attribute has a values, as it is needed
    707         # in order to retrieve the representation of the object:
--> 708         href = obj.href
    709         if href is None:
    710             raise Error(

AttributeError: 'NoneType' object has no attribute 'href'

Comment 8 Juan Hernández 2017-10-20 12:25:01 UTC
(In reply to nicolas from comment #7)
> In [4]: d = da.disk
> 
> In [5]: d
> Out[5]: <ovirtsdk4.types.Disk at 0x2cefe10>
> 
> In [6]: d.quota

At this ^ point "d" is a link to the disk, you need to follow that link before trying to use any attribute other than "id" and "href":

  disk = connection.follow_link(da.disk)

Once you have the disk, then you can try to follow the link to the quota:

  quota = connection.follow_link(disk.quota)

However I just saw that we aren't building that link correctly. This is what we return currently:

  <quota id="9d36e515-a2a4-483d-8746-b048725963ab"/>

As you can see there is no "href" attribute there, which is what the "follow_link" method needs to retrieve the information. Andrej, can you take care of this as well?

You can work around this by taking the id and retrieving the explicitly directly from the collection of quota:

  # Get the reference to the data center service:
  dc_service = ...

  # Get the reference to the service that manages the collection of
  # quotas of the data center:
  quotas_service = dc_service.quotas_service

  # Get the reference to the service that manages the quota of the
  # disk:
  quota_service = quotas_service(quota.id)

  # Retrieve the details of the quota:
  quota = quota_service.get()

Comment 9 Yaniv Kaul 2017-10-29 11:20:27 UTC
Target milestone?

Comment 12 Radim Hrazdil 2017-12-14 11:04:43 UTC
Verified that steps given in comment #7 doesn't lead to the error, therefore, as Juan says in comment #8, href link is provided. 

After following bug description and comment #1, the disk quota is by default set to unlimited, as is described in documentation. 

RHVM version 4.2.0-6, SDk 4.2.2

Comment 13 Sandro Bonazzola 2017-12-20 11:15:59 UTC
This bugzilla is included in oVirt 4.2.0 release, published on Dec 20th 2017.

Since the problem described in this bug report should be
resolved in oVirt 4.2.0 release, published on Dec 20th 2017, it has been closed with a resolution of CURRENT RELEASE.

If the solution does not work for you, please open a new bug report.


Note You need to log in before you can comment on or make changes to this bug.