Bug 1164764
Summary: | Appliance only inventories the first 25 images on OpenStack, pagination issue | |||
---|---|---|---|---|
Product: | Red Hat CloudForms Management Engine | Reporter: | Pete Savage <psavage> | |
Component: | Insight | Assignee: | Ladislav Smola <lsmola> | |
Status: | CLOSED ERRATA | QA Contact: | Dave Johnson <dajohnso> | |
Severity: | high | Docs Contact: | ||
Priority: | unspecified | |||
Version: | 5.3.0 | CC: | gblomqui, lsmola, obarenbo, xlecauch | |
Target Milestone: | GA | |||
Target Release: | 5.4.0 | |||
Hardware: | Unspecified | |||
OS: | Unspecified | |||
Whiteboard: | ||||
Fixed In Version: | 5.4.0.0.11 | Doc Type: | Bug Fix | |
Doc Text: | Story Points: | --- | ||
Clone Of: | ||||
: | 1164821 (view as bug list) | Environment: | ||
Last Closed: | 2015-06-16 12:44:51 UTC | Type: | Bug | |
Regression: | --- | Mount Type: | --- | |
Documentation: | --- | CRM: | ||
Verified Versions: | Category: | --- | ||
oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | ||
Cloudforms Team: | --- | Target Upstream Version: | ||
Embargoed: | ||||
Bug Depends On: | ||||
Bug Blocks: | 1164821 |
Description
Pete Savage
2014-11-17 11:29:30 UTC
Seems like FOG is not defining any limit. Default request is clean https://github.com/fog/fog/blob/f1500ba0003ac232d581ac8f1fdcf98f717d92ec/lib/fog/openstack/requests/image/list_public_images_detailed.rb#L9 Any chance you have defined limit in your /etc/glance/.. 'limit_param_default' or 'api_limit_max'? It's being used here https://github.com/openstack/glance/blob/12f12fb82ea78e23a276b2f0fb9faa3efa374ce3/glance/api/v2/images.py#L98 Even if you didn't set it, default should be 25, as stated here: http://docs.openstack.org/juno/config-reference/content/section_glance-registry.conf.html Seems like the fix for this bug is, that we need to work with pagination correctly, since we can't be sure pagination is disabled. This is also the way python-glanceclient does it. But, seems like FOG has a bug when sending parameter, e.g: svc.send(:list_public_images_detailed, 'limit', '20') URI::InvalidComponentError Exception: bad component(expected absolute path component): /v1.1/images/detail?limit=20 Plus it allows to send only one parametere now, which probable won't be enough. I suspect we also need to specify limit, because without that I don't see body['next'] which should have the next marker. Yes, that is defined in my environment I believe. btw. seems like this is the FOG bug I am seeing, the url with ? doesn't pass URI::parser.regexp[:ABS_PATH] (byebug) v='/v1.1/images/detail?limit=20' "/v1.1/images/detail?limit=20" (byebug) URI::parser.regexp[:ABS_PATH] !~ v true (byebug) v='/v1.1/images/detail' "/v1.1/images/detail" (byebug) URI::parser.regexp[:ABS_PATH] !~ v false Any idea how to define the URL correctly? I've pushed fix to FOG, the followup CFME fix will depend on this https://github.com/fog/fog/pull/3321 Another FOG patch dependency. We need to be able to pass the query to the images model https://github.com/fog/fog/pull/3322 Fix is here: https://github.com/ManageIQ/manageiq/pull/1198 Anybody has experience with the FOG CI? Seems like the gate is failing on some internal stuff, can I just recheck it somehow? Ladislav, I have a little experience with FOG CI. I also have seen PRs merged in FOG with failing tests. :/ New commit detected on manageiq/master: https://github.com/ManageIQ/manageiq/commit/0ac11e0b57fb077cffabbc101c813ea3eef7cc8d commit 0ac11e0b57fb077cffabbc101c813ea3eef7cc8d Author: Ladislav Smola <lsmola> AuthorDate: Tue Dec 9 13:07:42 2014 +0100 Commit: Ladislav Smola <lsmola> CommitDate: Tue Dec 16 15:41:02 2014 +0100 Fix OpenStack images pagination Pagination is always forced by Glance conf file. So we need to always obtain all pages of images in cycle. Depends-On: https://github.com/fog/fog/pull/3321 Depends-On: https://github.com/fog/fog/pull/3322 https://bugzilla.redhat.com/show_bug.cgi?id=1164764 lib/openstack/openstack_handle/image_delegate.rb | 17 +- .../openstack_refresher_rhos_havana_spec.rb | 26 +- .../openstack_refresher_rhos_grizzly.yml | 1286 +++++++++++--------- .../refreshers/openstack_refresher_rhos_havana.yml | 879 +++++++------ 4 files changed, 1226 insertions(+), 982 deletions(-) Just a quick note on verification There are three values that control the glance API limit: 1) the limit parameter passed in via the API call We need to patch Fog (being tracked in the upstream clone of this bug) to allow CFME to pass this value to Glance via the API. Once we have this ability, we can control how we gather images with pagination enabled in Glance. 2. the limit_param_default glance-registry.conf file setting If no parameter is passed in the API call, the limit_param_default is used. 3. the api_limit_max glance-registry.conf file setting The api_limit_max setting is used as an upper bound on the number of images returned via the Glance API call. The logic for setting the image count limit looks like: limit = value_from_api_call || limit_param_default limit = min(api_limit_max, limit) When attempting to verify this fix, make sure that there are enough images to cause pagination based on the above settings. Verified in 5.4.0.0.11 Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://rhn.redhat.com/errata/RHBA-2015-1100.html |