Bug 1414997

[heat-admin@overcloud-controller-0 ~]$ rpm -qa |grep cinder
python-cinderclient-1.9.0-4.el7ost.noarch
puppet-cinder-9.4.1-2.el7ost.noarch
python-cinder-9.0.0-13.el7ost.noarch
openstack-cinder-9.0.0-13.el7ost.noarch

The gigabytes field is missing from upstream RDO horizon in newton.

Created attachment 1247709 [details]
Screen Shot of Horizon

Here is a screen shot of the horizon dashboard that is missing the field to enter the size of gigabytes. This value is a requirement to update quotas for any service

*** Bug 1431266 has been marked as a duplicate of this bug. ***

Description of problem:
Modified project information and members but unable to modify project quotas when modifying volume quota in horizon admin identity view

Go to Identity -> Identity -> Projects, modify the quotas of a project and set volumes to another value.

Spot check shows that:
- vCPUs and instances **do** update, but the error message still shows
- volumes do **not** update


Version-Release number of selected component (if applicable):
[root@overcloud-controller-0 ~]# rpm -qf /usr/share/openstack-dashboard/openstack_dashboard/dashboards/identity/projects/workflows.py
openstack-dashboard-10.0.1-5.el7ost.noarch

Actual results:
volumes quotas do not update, error message shows

Expected results:
update volume quotas, no error message

Additional info:

The problem in the code with integer / string conversion: 
~~~
==> /var/log/cinder/api.log <==
2017-03-09 17:55:41.871 103796 DEBUG eventlet.wsgi.server [-] (103796) accepted ('172.16.2.4', 45808) server /usr/lib/python2.7/site-packages/eventlet/wsgi.py:867
2017-03-09 17:55:41.878 103796 INFO cinder.api.openstack.wsgi [req-2e2563df-a8f4-436a-83b5-93f6a483a826 aaafa6c0557b4ed89b2803af8149743a d809ec86f152486981e1782e94fa7b2d - default default] PUT http://10.0.0.4:8776/v2/d809ec86f152486981e1782e94fa7b2d/os-quota-sets/d809ec86f152486981e1782e94fa7b2d
2017-03-09 17:55:41.879 103796 DEBUG cinder.api.openstack.wsgi [req-2e2563df-a8f4-436a-83b5-93f6a483a826 aaafa6c0557b4ed89b2803af8149743a d809ec86f152486981e1782e94fa7b2d - default default] Action: 'update', calling method: <bound method QuotaSetsController.update of <cinder.api.contrib.quotas.QuotaSetsController object at 0x689af50>>, body: {"quota_set": {"gigabytes": null, "tenant_id": "d809ec86f152486981e1782e94fa7b2d", "volumes": 21, "snapshots": null}} _process_stack /usr/lib/python2.7/site-packages/cinder/api/openstack/wsgi.py:868
2017-03-09 17:55:41.977 103796 INFO cinder.api.openstack.wsgi [req-2e2563df-a8f4-436a-83b5-93f6a483a826 aaafa6c0557b4ed89b2803af8149743a d809ec86f152486981e1782e94fa7b2d - default default] HTTP exception thrown: gigabytes must be an integer.
2017-03-09 17:55:41.978 103796 INFO cinder.api.openstack.wsgi [req-2e2563df-a8f4-436a-83b5-93f6a483a826 aaafa6c0557b4ed89b2803af8149743a d809ec86f152486981e1782e94fa7b2d - default default] http://10.0.0.4:8776/v2/d809ec86f152486981e1782e94fa7b2d/os-quota-sets/d809ec86f152486981e1782e94fa7b2d returned with HTTP 400
2017-03-09 17:55:41.980 103796 INFO eventlet.wsgi.server [req-2e2563df-a8f4-436a-83b5-93f6a483a826 aaafa6c0557b4ed89b2803af8149743a d809ec86f152486981e1782e94fa7b2d - default default] 172.16.2.4 "PUT /v2/d809ec86f152486981e1782e94fa7b2d/os-quota-sets/d809ec86f152486981e1782e94fa7b2d HTTP/1.1" status: 400  len: 335 time: 0.1074338
~~~

When saving the quotas from the web interface, the following happens:
~~~
POST /dashboard/identity/53f4466c084145e0b91296f68be517a6/update/ HTTP/1.1
Host: 10.0.0.4
User-Agent: Mozilla/5.0 (X11; Fedora; Linux x86_64; rv:51.0) Gecko/20100101 Firefox/51.0
Accept: */*
Accept-Language: en-US,en;q=0.5
Accept-Encoding: gzip, deflate
Content-Type: application/x-www-form-urlencoded; charset=UTF-8
X-Requested-With: XMLHttpRequest
Referer: http://10.0.0.4/dashboard/identity/
Content-Length: 1616
Cookie: login_region="http://172.16.2.10:5000/v2.0"; login_domain=; SERVERID=overcloud-controller-0; csrftoken=8DM2AJOQIfjEN9rHBic3BzZrxoPYvnnP; sessionid=wk7y07jc7938xe6zu0ushg3micrq3aus
Connection: keep-alive

csrfmiddlewaretoken=8DM2AJOQIfjEN9rHBic3BzZrxoPYvnnP&domain_id=default&domain_name=Default&name=service&description=Tenant+for+the+openstack+services&enabled=on&available_update_members_filter=&update_members_members_filter=&default_update_members_role=9fe2ff9ee4384b1894a90878d3e92bab&update_members_role_64f3a3e25453466cac4325df17b64694=1de8b8db8a224b4f972853d4787d7a63&update_members_role_cd3a65a0a92f4fdc8badb7d1c87395fe=b5104f5fc1f844979ddd7023d3ed38e2&update_members_role_cd3a65a0a92f4fdc8badb7d1c87395fe=8c6c981aa56f4ae1ad1b999bc94575a7&update_members_role_cd3a65a0a92f4fdc8badb7d1c87395fe=930ff1e6c81a4fedab4e61c5286c6667&update_members_role_cd3a65a0a92f4fdc8badb7d1c87395fe=043ea3a59f9f4f539866e4171f0dced8&update_members_role_cd3a65a0a92f4fdc8badb7d1c87395fe=cc6f8622d68644d8a02d915ecb731fa2&update_members_role_cd3a65a0a92f4fdc8badb7d1c87395fe=7028ebf145a34803826916968e68253a&update_members_role_cd3a65a0a92f4fdc8badb7d1c87395fe=1de8b8db8a224b4f972853d4787d7a63&update_members_role_cd3a65a0a92f4fdc8badb7d1c87395fe=31b239bbb61d421c8636a8a03343f1cc&update_members_role_cd3a65a0a92f4fdc8badb7d1c87395fe=b77fb7185f0b479abd286702e359953c&update_members_role_cd3a65a0a92f4fdc8badb7d1c87395fe=586416a8b2c34a94a88ee73035fb92e8&default_update_group_members_role=9fe2ff9ee4384b1894a90878d3e92bab&snapshots=&gigabytes=&floating_ips=&fixed_ips=&security_groups=&security_group_rules=&shares=&share_networks=&metadata_items=128&cores=20&instances=10&injected_files=5&injected_file_content_bytes=10240&volumes=11&ram=51200&security_group=10&security_group_rule=100&floatingip=50&network=10&port=50&router=10&subnet=10
~~~

Which means that, e.g. 'gigabytes=' which then gets set to NULL.

The answer to this post request is:
~~~
HTTP/1.1 200 OK
Date: Thu, 09 Mar 2017 18:19:26 GMT
Server: Apache
X-Horizon-Location: /dashboard/identity/
Vary: Cookie
Set-Cookie: messages="f07fa50570c8d673536fe69deb63255a5710eb9f$[[\"__json_message\"\0540\05440\054\"Modified project information and members\054 but unable to modify project quotas.\"]\054[\"__json_message\"\0540\05425\054\"Modified project \\\"service\\\".\"]]"; httponly; Path=/
Content-Length: 0
Content-Type: text/html; charset=utf-8
~~~

Which contains the error message from openstack_dashboard/dashboards/identity/projects/workflows.py
~~~
    def _update_project_quota(self, request, data, project_id):
        try:
            super(UpdateProject, self)._update_project_quota(
                request, data, project_id)
            return True
        except Exception:
            exceptions.handle(request, _('Modified project information and '
                                         'members, but unable to modify '
                                         'project quotas.'))
            return False
~~~



The error message comes from openstack_dashboard/dashboards/identity/projects/workflows.py
~~~
    def _update_project_quota(self, request, data, project_id):
        try:
            super(UpdateProject, self)._update_project_quota(
                request, data, project_id)
            return True
        except Exception:
            exceptions.handle(request, _('Modified project information and '
                                         'members, but unable to modify '
                                         'project quotas.'))
            return False
~~~

which calls update_project_quota from this class in the same file:
~~~
class CommonQuotaWorkflow(workflows.Workflow):
    def _update_project_quota(self, request, data, project_id):
        disabled_quotas = quotas.get_disabled_quotas(request)

        # Update the project quotas.
        if api.base.is_service_enabled(request, 'compute'):
            nova_data = {key: data[key] for key in
                         set(quotas.NOVA_QUOTA_FIELDS) - disabled_quotas}
            nova.tenant_quota_update(request, project_id, **nova_data)

        if cinder.is_volume_service_enabled(request):
            cinder_data = dict([(key, data[key]) for key in
                                quotas.CINDER_QUOTA_FIELDS])
            cinder.tenant_quota_update(request,
                                       project_id,
                                       **cinder_data)
~~~

I modified the following file:
~~~
[root@overcloud-controller-0 projects]# diff -c /usr/share/openstack-dashboard/openstack_dashboard/dashboards/identity/projects/workflows.py /usr/share/openstack-dashboard/openstack_dashboard/dashboards/identity/projects/workflows.py.patch 
*** /usr/share/openstack-dashboard/openstack_dashboard/dashboards/identity/projects/workflows.py	2017-03-09 21:42:05.078987027 +0000
--- /usr/share/openstack-dashboard/openstack_dashboard/dashboards/identity/projects/workflows.py.patch	2017-03-09 21:40:04.016155030 +0000
***************
*** 413,418 ****
--- 413,422 ----
          if cinder.is_volume_service_enabled(request):
              cinder_data = dict([(key, data[key]) for key in
                                  quotas.CINDER_QUOTA_FIELDS])
+             # remove any None / Null elements to avoid conversion errors
+             for key in cinder_data.keys():
+               if cinder_data[key] is None:
+                 cinder_data.pop(key, None)
              cinder.tenant_quota_update(request,
                                         project_id,
                                         **cinder_data)
~~~

which fixes the error message and lets you update the quotas. This may very well be the wrong point in the code to fix this, though.

The patch I'm suggesting fixes the issue. However, I think that it may treat a symptom only, and at the wrong place. Otherwise, I'd have pushed it upstream ...

I can't reproduce this issue with a recent build of OSP10. The "Total Size of Volumes and Snapshots (GiB)" field is visible and there is no error when saving this form.

Are there any additional conditions that have to be met to reproduce this?

The bug reported upstream at https://bugs.launchpad.net/horizon/+bug/1671911 also couldn't be reproduced by the upstream developers.

Hrmmm, I recently ran into this during a local deployment on my baremetal nodes, which was why I added myself to this BZ. Have you tried adding a project and then making the changes outside of the admin tenant? I'll look again today as I have an environment up and running that I could make changes to.

(In reply to Leif Madsen from comment #11)
> Hrmmm, I recently ran into this during a local deployment on my baremetal
> nodes, which was why I added myself to this BZ. Have you tried adding a
> project and then making the changes outside of the admin tenant? I'll look
> again today as I have an environment up and running that I could make
> changes to.

I just went back and looked, and it seems my overcloud-full.qcow2 is actually from March 11th, so it's possible this has been fixed in the last couple of weeks.

I installed an older build with python-django-horizon-10.0.1-5.el7ost.noarch but still can't reproduce this problem.

(In reply to Leif Madsen from comment #11)
> Hrmmm, I recently ran into this during a local deployment on my baremetal
> nodes, which was why I added myself to this BZ. Have you tried adding a
> project and then making the changes outside of the admin tenant? I'll look
> again today as I have an environment up and running that I could make
> changes to.

I tried on a demo project with the demo user -- it works fine.

it's with the admin user, and it's super easy to reproduce. let me take a look and get back to you.

This happens in RHEL OpenStack because opposed to upstream, we have the Gigabytes field by default disabled, and it then causes NULL to be sent.

I posted a patch upstream that should help with this problem: https://review.openstack.org/451679

However, I still can't figure out how it is possible that the fields don't appear in the tripleo horizon, but the cinder api is being called. The relevant fragments of code are:

In the form display, where the disabled fields are being hidden:

        disabled_quotas = quotas.get_disabled_quotas(request)
        for field in disabled_quotas:
            if field in self.fields:
                self.fields[field].required = False
                self.fields[field].widget = forms.HiddenInput()

That get_disabled_quotas function:

    if not cinder.is_volume_service_enabled(request):
        disabled_quotas.update(CINDER_QUOTA_FIELDS)

And finally the method that does the actual update:

        if cinder.is_volume_service_enabled(request):
            cinder_data = dict([(key, data[key]) for key in
                                quotas.CINDER_QUOTA_FIELDS])
            cinder.tenant_quota_update(request,
                                       project_id,
                                       **cinder_data)

The way I'm reading the code, the fields should be disabled IFF the cinder service is not available, but then the quota update API call shouldn't be made either.

I am working with a customer and we are running into the same error message in Horizon.

"Error: Modified project information and members, but unable to modify project quotas."

However we are not attempting to change quotas for a project, rather we are just trying to add/remove users from the project. 

We run into the error when. 

1. When attempting to edit project.
2. When attempting to modify quotas for a project.
3. When attempting to modify groups for a project.

*** Bug 1445890 has been marked as a duplicate of this bug. ***

Verified
========
python-django-horizon-10.0.3-6.el7ost.noarch

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2017:1598