Bug 1477706 - migration with block migration fails as disk_available_least is negative
migration with block migration fails as disk_available_least is negative
Status: CLOSED ERRATA
Product: Red Hat OpenStack
Classification: Red Hat
Component: openstack-nova (Show other bugs)
9.0 (Mitaka)
Unspecified Unspecified
high Severity high
: zstream
: 9.0 (Mitaka)
Assigned To: Matthew Booth
Archit Modi
: Triaged, ZStream
: 1496913 (view as bug list)
Depends On:
Blocks: 1485427 1530330 1533161 1533164
  Show dependency treegraph
 
Reported: 2017-08-02 12:28 EDT by Martin Schuppert
Modified: 2018-03-15 08:44 EDT (History)
18 users (show)

See Also:
Fixed In Version: openstack-nova-13.1.4-15.el7ost
Doc Type: Bug Fix
Doc Text:
Previously, a combination of circumstances could result in a failed live migration. This could arise when using block migration, with disk overcommit enabled, and using a client sending microversion < 2.25 (this version scope included the openstack client, but not the nova client). As a result of these circumstances, the live migration call could fail with the following error: Migration pre-check error: Unable to migrate [instance uuid]: Disk of instance is too large(available on destination host:[some negative number] < need:[disk space required]) With this update, the check has been updated to correctly consider overcommittal of existing instances. As a result, live migration succeeds as expected.
Story Points: ---
Clone Of:
: 1485427 1530330 (view as bug list)
Environment:
Last Closed: 2018-03-15 08:43:27 EDT
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)


External Trackers
Tracker ID Priority Status Summary Last Updated
Launchpad 1708572 None None None 2017-12-01 12:03 EST
OpenStack gerrit 524681 None None None 2017-12-01 12:04 EST
Red Hat Product Errata RHBA-2018:0538 None None None 2018-03-15 08:44 EDT

  None (edit)
Description Martin Schuppert 2017-08-02 12:28:32 EDT
Description of problem:

The issue is like what described in [1]. 

When disk_allocation_ratio config option > 1.0 and no shared storage is used, migration fails on checks with:
ERROR (BadRequest): Migration pre-check error: Unable to migrate 9f5360f3-282f-4a50-983c-7b0025d9509f: Disk of instance is too large(available on destination host:-727996956672 < need:65929216) (HTTP 400) (Request-ID: req-ef12922d-5032-4b59-8963-4ce124102662)

The root cause for that is that nova compares disk space, required by instance, with disk_available_least field, which is possible to be negative, due to overcommit.

[stack@undercloud-0 ~]$ openstack server migrate --live compute-1.localdomain --disk-overcommit test --debug
...
REQ: curl -g -i -X POST http://10.0.0.101:8774/v2.1/a126a6b887d6452bb4c8ae99774a07ef/servers/f78d0336-b874-4a81-855b-cb618b5e026d/action -H "User-Agent: python-novaclient" -H "Content-Type: application/json" -H "Accept: application/json" -H "X-Auth-Token: {SHA1}03a442295ebe0c3fe7548bfa74409f01a3063676" -d '{"os-migrateLive": {"disk_over_commit": true, "block_migration": true, "host": "compute-1.localdomain"}}'
"POST /v2.1/a126a6b887d6452bb4c8ae99774a07ef/servers/f78d0336-b874-4a81-855b-cb618b5e026d/action HTTP/1.1" 400 215
RESP: [400] X-Openstack-Nova-Api-Version: 2.1 Vary: X-OpenStack-Nova-API-Version Content-Type: application/json; charset=UTF-8 Content-Length: 215 X-Compute-Request-Id: req-877f7d28-295b-47ff-9e9e-430cbc39aeca Date: Wed, 02 Aug 2017 13:45:08 GMT
RESP BODY: {"badRequest": {"message": "Migration pre-check error: Unable to migrate f78d0336-b874-4a81-855b-cb618b5e026d: Disk of instance is too large(available on destination host:-48318382080 < need:2162688)", "code": 400}}

Migration pre-check error: Unable to migrate f78d0336-b874-4a81-855b-cb618b5e026d: Disk of instance is too large(available on destination host:-48318382080 < need:2162688) (HTTP 400) (Request-ID: req-877f7d28-295b-47ff-9e9e-430cbc39aeca)
Traceback (most recent call last):
  File "/usr/lib/python2.7/site-packages/cliff/app.py", line 346, in run_subcommand
    result = cmd.run(parsed_args)
  File "/usr/lib/python2.7/site-packages/openstackclient/common/command.py", line 38, in run
    return super(Command, self).run(parsed_args)
  File "/usr/lib/python2.7/site-packages/cliff/command.py", line 55, in run
    self.take_action(parsed_args)
  File "/usr/lib/python2.7/site-packages/openstackclient/compute/v2/server.py", line 978, in take_action
    parsed_args.disk_overcommit,
  File "/usr/lib/python2.7/site-packages/novaclient/v2/servers.py", line 433, in live_migrate
    disk_over_commit)   
  File "/usr/lib/python2.7/site-packages/novaclient/api_versions.py", line 370, in substitution
    return methods[-1].func(obj, *args, **kwargs)
  File "/usr/lib/python2.7/site-packages/novaclient/v2/servers.py", line 1515, in live_migrate
    'disk_over_commit': disk_over_commit})
  File "/usr/lib/python2.7/site-packages/novaclient/v2/servers.py", line 1682, in _action
    info=info, **kwargs)
  File "/usr/lib/python2.7/site-packages/novaclient/v2/servers.py", line 1693, in _action_return_resp_and_body
    return self.api.client.post(url, body=body)
  File "/usr/lib/python2.7/site-packages/keystoneauth1/adapter.py", line 179, in post
    return self.request(url, 'POST', **kwargs)
  File "/usr/lib/python2.7/site-packages/novaclient/client.py", line 94, in request
    raise exceptions.from_response(resp, body, url, method)
BadRequest: Migration pre-check error: Unable to migrate f78d0336-b874-4a81-855b-cb618b5e026d: Disk of instance is too large(available on destination host:-48318382080 < need:2162688) (HTTP 400) (Request-ID: req-877f7d28-295b-47ff-9e9e-430cbc39aeca)


[1] https://bugs.launchpad.net/mos/+bug/1659539

Version-Release number of selected component (if applicable):
python-nova-13.1.4-5.el7ost.noarch

How reproducible:
always

Steps to Reproduce:
1. set disk_allocation_ratio config option > 1.0 
2. qemu-img resize cirros-0.3.0-x86_64-disk.img +40G
3. glance image-create --disk-format qcow2 ...
4. boot VMs based on resized image
5. we see disk_available_least becomes negative
$ nova hypervisor-show compute-0.localdomain
+---------------------------+------------------------------------------+
| Property                  | Value                                    |
+---------------------------+------------------------------------------+
| cpu_info_arch             | x86_64                                   |
| cpu_info_features         | ["pge", "avx", "vmx", "clflush", "sep",  |
|                           | "syscall", "tsc-deadline", "tsc",        |
|                           | "xsave", "xsaveopt", "cmov", "nx",       |
|                           | "pat", "lm", "msr", "fpu", "fxsr",       |
|                           | "sse4.1", "pae", "sse4.2", "pclmuldq",   |
|                           | "pcid", "vme", "mmx", "osxsave", "cx8",  |
|                           | "mce", "de", "aes", "mca", "pse", "pni", |
|                           | "popcnt", "pdpe1gb", "apic", "sse",      |
|                           | "lahf_lm", "rdtscp", "sse2", "ss",       |
|                           | "hypervisor", "ssse3", "cx16", "pse36",  |
|                           | "mtrr", "x2apic"]                        |
| cpu_info_model            | SandyBridge                              |
| cpu_info_topology_cells   | 1                                        |
| cpu_info_topology_cores   | 1                                        |
| cpu_info_topology_sockets | 4                                        |
| cpu_info_topology_threads | 1                                        |
| cpu_info_vendor           | Intel                                    |
| current_workload          | 0                                        |
| disk_available_least      | -45                                      |


Actual results:
As a result migration to that host fails even if there is enough disk available.

Expected results:
Migration works as we specified disk_allocation_ratio and use not raw images

Additional info:
Comment 2 Sahid Ferdjaoui 2017-08-03 04:29:27 EDT
The launchpad bug you have noticed on the description is not related to Nova. (or it's related to fork or something else) anyway the fix is not merged upstream.

After a look at master it seems that the issue is fixed by that patch [0] but unfortunately i'm not sure it will be easily backportable since it comes with a large refactor.

Some investigations still need to be done...

[0] https://review.openstack.org/#/c/275585/
Comment 3 Martin Schuppert 2017-08-03 05:29:13 EDT
just realized, --disk-overcommit is deprecated since api version 2.25. And e.g. the need to pass disk-overcommit got removed:

https://specs.openstack.org/openstack/nova-specs/specs/mitaka/implemented/making_live_migration_api_friendly.html

The openstackclient uses v2 per default:

$ openstack server migrate --live compute-0.localdomain --disk-overcommit test2 --debug

defaults: {u'auth_type': 'password', u'compute_api_version': u'2', 'key': None, u'database_api_version': u'1.0', 'api_timeout': None, u'baremetal_api_version': u'1', u'image_api_version': u'2', 'cacert': None, u'image_api_use_tasks': False, u'floating_ip_source': u'neutron', u'orchestration_api_version': u'1', u'interface': None, u'network_api_version': u'2', u'image_format': u'qcow2', u'key_manager_api_version': u'v1', u'metering_api_version': u'2', 'verify': True, u'identity_api_version': u'2.0', u'volume_api_version': u'2', 'cert': None, u'secgroup_source': u'neutron', u'container_api_version': u'1', u'dns_api_version': u'2', u'object_store_api_version': u'1', u'disable_vendor_agent': {}}
cloud cfg: {'auth_type': 'password', 'tripleoclient_api_version': '1', u'compute_api_version': u'2', u'orchestration_api_version': '1', u'database_api_version': u'1.0', 'data_processing_api_version': '1.1', 'inspector_api_version': '1', u'network_api_version': u'2', u'image_format': u'qcow2', u'image_api_version': u'2', 'verify': True, u'dns_api_version': '2', u'object_store_api_version': u'1', 'verbose_level': 3, 'region_name': '', 'api_timeout': None, u'baremetal_api_version': '1.6', 'queues_api_version': '1.1', 'auth': {'username': 'admin', 'project_name': 'admin', 'password': '***', 'auth_url': 'http://10.0.0.101:5000/v2.0'}, 'default_domain': 'default', u'container_api_version': u'1', u'image_api_use_tasks': False, u'floating_ip_source': u'neutron', 'key': None, 'timing': False, 'cacert': None, u'key_manager_api_version': '1', u'metering_api_version': u'2', 'deferred_help': False, u'identity_api_version': u'2.0', u'volume_api_version': u'2', 'cert': None, u'secgroup_source': u'neutron', 'alarming_api_version': '2', 'debug': True, u'interface': None, u'disable_vendor_agent': {}}

Migration pre-check error: Unable to migrate 5ca04270-d047-4f6a-b09e-c6f11dd97079: Disk of instance is too large(available on destination host:-48318382080 < need:1573888) (HTTP 400) (Request-ID: req-6405a5af-3d09-4694-9f6e-ed27a2a76df9)

When we use the novaclient in OSP9, which per default use compute api version 2.25 :

$ nova --debug live-migration --block-migrate  test2 compute-0.localdomain --debug
...
DEBUG (session:248) REQ: curl -g -i -X POST http://10.0.0.101:8774/v2.1/a126a6b887d6452bb4c8ae99774a07ef/servers/90b887b5-0cd6-4145-a88d-8eba2dcbea12/action -H "User-Agent: python-novaclient" -H "Content-Type: application/json" -H "Accept: application/json" -H "X-OpenStack-Nova-API-Version: 2.25" -H "X-Auth-Token: {SHA1}c71f107636c6bbf12ade9b747bb259133c5fc6c0" -d '{"os-migrateLive": {"block_migration": true, "host": "compute-0.localdomain"}}'

$ nova show test2
+--------------------------------------+----------------------------------------------------------+
| Property                             | Value                                                    |
+--------------------------------------+----------------------------------------------------------+
| OS-DCF:diskConfig                    | MANUAL                                                   |
| OS-EXT-AZ:availability_zone          | nova                                                     |
| OS-EXT-SRV-ATTR:host                 | compute-0.localdomain                                    |
| OS-EXT-SRV-ATTR:hostname             | test2                                                    |
| OS-EXT-SRV-ATTR:hypervisor_hostname  | compute-0.localdomain                                    |

Seems with API version 2.25 and later where we do not have the disk_over_commit flag we then do not run the check for the DST host storage:

   5592             if 'disk_over_commit' in dest_check_data:
   5593                 self._assert_dest_node_has_enough_disk(context, instance,
   5594                                         dest_check_data.disk_available_mb,
   5595                                         dest_check_data.disk_over_commit,
   5596                                         block_device_info)

But when we specify the api version 2.25 with the openstackclient it fails as it seems disk_over_commit is provided per default in the API request.

$ openstack --os-compute-api-version 2.25 server migrate --live compute-0.localdomain  test2 
Setting 'disk_over_commit' argument is prohibited after microversion 2.25.


Maybe the correct would be to set the disk-overcommit default values to None

    945         disk_group.add_argument(
    946             '--disk-overcommit',
    947             action='store_true',
    948             default=None,
    949             help=_('Allow disk over-commit on the destination host'),
    950         )
    951         disk_group.add_argument(
    952             '--no-disk-overcommit',
    953             dest='disk_overcommit',
    954             action='store_false',
    955             default=None,
    956             help=_('Do not over-commit disk on the'
    957                    ' destination host (default)'),
    958         )

Then we can use the api version 2.25 also with openstackclient:
$ openstack --os-compute-api-version 2.25 server migrate --live compute-0.localdomain  test2
Comment 4 Martin Schuppert 2017-08-03 05:40:32 EDT
With this change when we use the default api version we fail with the disk too large error, but I'd say that is ok as we do not have specified disk-overcommit:

$ openstack server migrate --live compute-1.localdomain  test1
Migration pre-check error: Unable to migrate 90b887b5-0cd6-4145-a88d-8eba2dcbea12: Disk of instance is too large(available on destination host:38654705664 < need:85940472320) (HTTP 400) (Request-ID: req-2067b927-af13-494a-acd8-fe5226d17a30)

$ openstack server migrate --live compute-1.localdomain  --disk-overcommit test1

So maybe this bug needs to be moved to openstackclient component?
Comment 5 Sahid Ferdjaoui 2017-08-03 05:48:28 EDT
Yes so that seems to be an issue in openstackclient. Thanks for you investigation.
Comment 6 Matthew Booth 2017-08-04 08:28:24 EDT
I don't think this is an client issue. I think the bug is is the reporting of disk_available_least by the compute host. The disk_over_commit setting affects this code:

        necessary = 0
        if disk_over_commit:
            for info in disk_infos:
                necessary += int(info['disk_size'])
        else:
            for info in disk_infos:
                necessary += int(info['virt_disk_size'])

It's picking between checking that the available space at the destination is big enough for the actual [overcommit] or maximum [no overcommit] disk sizes. This makes sense. The problem is that the reported available space at the destination is negative, which is nonsense in both cases. I think that the only value we're interested in here is the actual amount of disk space at the destination, and overcommit is irrelevant. Perhaps this is why it has been removed from the newest value of the api.

We need to fix this flow such that the value we're comparing against is the actual current free disk space on the target.

Additionally, this code seems to imply that we're mixing overcommit and non-overcommit instances on the same host, with non-overcommit instances not fully allocated. If so, this is highly likely to be broken.
Comment 7 Martin Schuppert 2017-08-04 08:40:54 EDT
(In reply to Matthew Booth from comment #6)
> I don't think this is an client issue. I think the bug is is the reporting
> of disk_available_least by the compute host. The disk_over_commit setting
> affects this code:
> 
>         necessary = 0
>         if disk_over_commit:
>             for info in disk_infos:
>                 necessary += int(info['disk_size'])
>         else:
>             for info in disk_infos:
>                 necessary += int(info['virt_disk_size'])
> 
> It's picking between checking that the available space at the destination is
> big enough for the actual [overcommit] or maximum [no overcommit] disk
> sizes. This makes sense. The problem is that the reported available space at
> the destination is negative, which is nonsense in both cases. I think that
> the only value we're interested in here is the actual amount of disk space
> at the destination, and overcommit is irrelevant. Perhaps this is why it has
> been removed from the newest value of the api.
> 

yes this is correct, but we only hit it when we use API version < 2.25 as with
2.25 we do not have the overcommit setting any longer. So as a workaround you 
can use the novaclient and specify to use API v2.25 to not go that path in the 
code.

> We need to fix this flow such that the value we're comparing against is the
> actual current free disk space on the target.

Agreed, it would be good to fix it as e.g. in horizon there is no way to specify
 a minor API version to be used. You can just say API v1 or v2.

> 
> Additionally, this code seems to imply that we're mixing overcommit and
> non-overcommit instances on the same host, with non-overcommit instances not
> fully allocated. If so, this is highly likely to be broken.

Not sure I understand this correct. You mean you should not mix instances
with and without overcommit on a compute?
Comment 8 Martin Schuppert 2017-08-04 10:27:43 EDT
@mdbooth:

we only check the disk size when disk_over_commit is set and if I got it correct, we do not have it any more with API 2.25:

virt/libvirt/driver.py

   5592             if 'disk_over_commit' in dest_check_data:
   5593                 self._assert_dest_node_has_enough_disk(context, instance,
   5594                                         dest_check_data.disk_available_mb,
   5595                                         dest_check_data.disk_over_commit,
   5596                                         block_device_info)
Comment 9 Matthew Booth 2017-08-04 12:19:18 EDT
Sorry, Martin. I missed where you said this worked with nova client.

The only way I can see this would be a bug in openstack client is if the client wasn't setting the api version to 2.25 as requested. As you point out this test should be entirely skipped[1] if disk_over_commit isn't set. However, MigrateServerController._migrate_live won't attempt to pull disk_over_commit out of the request of api version >= 2.25. Also I don't think it would validate, as disk_over_commit is removed from migrate_server.migrate_live_v2_25.

Assuming you still have the reproducer system available, could you please run the command again with both openstack client and nova client, and check which api version request is actually received? I can't remember if we already log this, but if not could you please wedge some extra debug into MigrateServerController._migrate_live to capture it.

If we're receiving version >= 2.25 this is a bug in nova, because we should either be rejecting an invalid request, or ignoring the parameter. My guess would be that disk_over_commit=None accidentally becomes False or something like that at some point. Maybe something change in ovo which means the 'in' test doesn't work. I'm guessing.

If we're receiving version < 2.25 this is a bug in openstack client.

If the code responsible for setting the outgoing api version is in the nova plugin, the bug remains with us. Otherwise we need to punt it to whoever maintains the framework portion of the client.

[1] This seems like a different bug to me. Where does disk space checking happen?
Comment 10 Sylvain Bauza 2017-08-11 10:24:22 EDT
Martin, as far I know how OSC works, its default behaviour about microversions is different from the nova CLI.

By default, the Nova CLI asks for the latest microversion it knows (not 2.latest, but rather a specific maximum version related to the features it supports) while AFAICS openstackclient (OSC) asks for the stable v2.1 microversion.

Given the REST resource changed in 2.25 by not accepting the "disk_over_commit" flag, the problem was silently fixed for that version and later, but I guess the problem still remains for older microversions.

Since Nova needs to support 2.1 as a minimum, I think it's reasonable to do this :
 - first, could you please verify that if passing a specific microversion to Nova CLI that is older than 2.25, you still have the problem ?

 - second, if the problem is still there with old API microversions, it's worth discussing upstream about that and see how we can fix that from a Nova perspective.

HTH,
-Sylvain
Comment 11 Martin Schuppert 2017-08-22 03:52:36 EDT
(In reply to Sylvain Bauza from comment #10)

> Since Nova needs to support 2.1 as a minimum, I think it's reasonable to do
> this :
>  - first, could you please verify that if passing a specific microversion to
> Nova CLI that is older than 2.25, you still have the problem ?

yes when using an older minor version, e.g. 2.1 we see the problem also with the nova client:

$ nova --debug --os-compute-api-version 2.1 live-migration --block-migrate --disk_over_commit true test compute-0.localdomain
...
DEBUG (session:248) REQ: curl -g -i -X POST http://10.0.0.101:8774/v2.1/a126a6b887d6452bb4c8ae99774a07ef/servers/257bb249-3a2f-4ca3-95d0-f7a2d48406b5/action -H "User-Agent: python-novaclient" -H "Content-Type: application/json" -H "Accept: application/json" -H "X-OpenStack-Nova-API-Version: 2.1" -H "X-Auth-Token: {SHA1}07d5b8d3105a99288466337f2bca346a292271d6" -d '{"os-migrateLive": {"disk_over_commit": true, "block_migration": true, "host": "compute-0.localdomain"}}'

RESP BODY: {"badRequest": {"message": "Migration pre-check error: Unable to migrate 257bb249-3a2f-4ca3-95d0-f7a2d48406b5: Disk of instance is too large(available on destination host:-49392123904 < need:1573888)", "code": 400}}

DEBUG (shell:1082) Migration pre-check error: Unable to migrate 257bb249-3a2f-4ca3-95d0-f7a2d48406b5: Disk of instance is too large(available on destination host:-49392123904 < need:1573888) (HTTP 400) (Request-ID: req-2c960511-b24d-4db3-a50e-00fcdf2d9323)
Traceback (most recent call last):
  File "/usr/lib/python2.7/site-packages/novaclient/shell.py", line 1080, in main
    OpenStackComputeShell().main(argv)
  File "/usr/lib/python2.7/site-packages/novaclient/shell.py", line 1007, in main
    args.func(self.cs, args)
  File "/usr/lib/python2.7/site-packages/novaclient/v2/shell.py", line 3850, in do_live_migration
    args.disk_over_commit)
  File "/usr/lib/python2.7/site-packages/novaclient/v2/servers.py", line 433, in live_migrate
    disk_over_commit)   
  File "/usr/lib/python2.7/site-packages/novaclient/api_versions.py", line 370, in substitution
    return methods[-1].func(obj, *args, **kwargs)
  File "/usr/lib/python2.7/site-packages/novaclient/v2/servers.py", line 1515, in live_migrate
    'disk_over_commit': disk_over_commit})
  File "/usr/lib/python2.7/site-packages/novaclient/v2/servers.py", line 1682, in _action
    info=info, **kwargs)
  File "/usr/lib/python2.7/site-packages/novaclient/v2/servers.py", line 1693, in _action_return_resp_and_body
    return self.api.client.post(url, body=body)
  File "/usr/lib/python2.7/site-packages/keystoneauth1/adapter.py", line 179, in post
    return self.request(url, 'POST', **kwargs)
  File "/usr/lib/python2.7/site-packages/novaclient/client.py", line 94, in request
    raise exceptions.from_response(resp, body, url, method)
BadRequest: Migration pre-check error: Unable to migrate 257bb249-3a2f-4ca3-95d0-f7a2d48406b5: Disk of instance is too large(available on destination host:-49392123904 < need:1573888) (HTTP 400) (Request-ID: req-2c960511-b24d-4db3-a50e-00fcdf2d9323)
ERROR (BadRequest): Migration pre-check error: Unable to migrate 257bb249-3a2f-4ca3-95d0-f7a2d48406b5: Disk of instance is too large(available on destination host:-49392123904 < need:1573888) (HTTP 400) (Request-ID: req-2c960511-b24d-4db3-a50e-00fcdf2d9323)

>  - second, if the problem is still there with old API microversions, it's
> worth discussing upstream about that and see how we can fix that from a Nova
> perspective.
Comment 12 Matthew Booth 2017-08-25 09:11:26 EDT
Summary of the facts from above as I understand them:

* If client sends <2.25 we execute a broken overcommit check in Nova
* If client sends >=2.25 it's fine
* Nova client sends >=2.25
* OSC sends 2.1 by default
* Nova is supposed to support 2.1

Based on the above, I think the correct resolution is to fix (or at least bring some sanity to) the broken overcommit check in Nova.
Comment 13 Martin Schuppert 2017-08-25 09:19:52 EDT
(In reply to Matthew Booth from comment #12)
> Summary of the facts from above as I understand them:
> 
> * If client sends <2.25 we execute a broken overcommit check in Nova
> * If client sends >=2.25 it's fine
> * Nova client sends >=2.25
> * OSC sends 2.1 by default
> * Nova is supposed to support 2.1
> 
> Based on the above, I think the correct resolution is to fix (or at least
> bring some sanity to) the broken overcommit check in Nova.

yes, and OSC should not send overcommit flags, the current default is to set it to false which make it fail when specify >=2.25, as shown in comment 3.
Comment 14 Matthew Booth 2017-08-25 12:32:47 EDT
This bug will continue to track the Nova aspect of this bug described in comment 6. Specifically, Nova is expected to correctly handle api requests with microversion 2.1, but there is a bug in this code path.
Comment 15 Matthew Booth 2017-10-06 10:29:57 EDT
*** Bug 1496913 has been marked as a duplicate of this bug. ***
Comment 29 errata-xmlrpc 2018-03-15 08:43:27 EDT
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2018:0538

Note You need to log in before you can comment on or make changes to this bug.