Bug 1552583 - [Glance] Interoperable image import should be disabled by default on 3 node HA setup
Summary: [Glance] Interoperable image import should be disabled by default on 3 node H...
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat OpenStack
Classification: Red Hat
Component: openstack-tripleo-heat-templates
Version: 13.0 (Queens)
Hardware: Unspecified
OS: Unspecified
urgent
urgent
Target Milestone: rc
: 13.0 (Queens)
Assignee: Pranali Deore
QA Contact: Mike Abrams
Kim Nylander
URL:
Whiteboard:
Depends On:
Blocks: 1552584
TreeView+ depends on / blocked
 
Reported: 2018-03-07 12:03 UTC by Christian Schwede (cschwede)
Modified: 2018-06-27 13:35 UTC (History)
12 users (show)

Fixed In Version: openstack-tripleo-heat-templates-8.0.2-21.el7ost puppet-glance-12.5.0-2.el7ost
Doc Type: Bug Fix
Doc Text:
The glance-direct method requires a shared staging area when used in a HA configuration. Image uploads using the 'glance-direct' method may fail in an HA environment if a common staging area is not present. Incoming requests to the controller nodes are distributed across the available controller nodes. One controller handles the first step and another controller handles the second request with both controllers writing the image to different staging areas. The second controller will not have access to the same staging area used by the controller handling the first step. Glance supports multiple image import methods, including the 'glance-direct' method. This method uses a three-step approach: creating an image record, uploading the image to a staging area, and then transferring the image from the staging area to the storage backend so the image becomes available. In an HA setup (i.e., with 3 controller nodes), the glance-direct method requires a common staging area using a shared file system across the controller nodes. The list of enabled Glance import methods can now be configured. The default configuration does not enable the 'glance-direct' method (web-download is enabled by default). To avoid the issue and reliably import images to Glance in an HA environment, do not enable the 'glance-direct' method.
Clone Of:
Environment:
Last Closed: 2018-06-27 13:35:05 UTC
Target Upstream Version:


Attachments (Terms of Use)


Links
System ID Priority Status Summary Last Updated
OpenStack gerrit 567662 None MERGED Add ability to configure enabled_import_methods 2020-04-24 06:25:23 UTC
OpenStack gerrit 567667 None MERGED Add ability to control Glance's enabled_import_methods 2020-04-24 06:25:23 UTC
OpenStack gerrit 567694 None MERGED Add ability to configure enabled_import_methods 2020-04-24 06:25:23 UTC
OpenStack gerrit 568366 None MERGED Fix syntax for setting enabled_import_methods 2020-04-24 06:25:23 UTC
Red Hat Product Errata RHEA-2018:2086 None None None 2018-06-27 13:35:52 UTC

Description Christian Schwede (cschwede) 2018-03-07 12:03:30 UTC
Glance added the support for interoperable image imports. Image uploads are done using a two-step approach: first uploading it to the Glance API using a local staging area, and then triggering an upload from the staging area to the storage backend.

In the case of an HA setup (ie with 3 controller nodes) this requires a shared consistent filesystem across the controller nodes. Otherwise it might happen that the image is uploaded to one node, and triggering the import is executed on a different node - where the image does not exist.

Therefore we need to disable this by default, thus operators need to enable it explicitly if they have a shared filesystem across the controller nodes (for example using NFS).

We need to do:
1. Set enabled_import_methods = web-download by default in glance-api.conf
2. Make this option changeable in t-h-t
3. Document this

Comment 6 Alan Bishop 2018-05-11 11:45:37 UTC
Moving to POST because upstream reviews have been approved on master (although one has not merged due to CI gate issues). Patches have been proposed to stable/queens.

Comment 8 Paul Grist 2018-05-11 20:43:48 UTC
Adding some more detail on the description. We definitely need to get the proper default in because there is another implication of the out of the box settings.

The shared filesystem will only come into picture if customer wants to use a new feature (image import with glance-direct method). It doesn't have any impact with traditional way of creating the image.


In the HA setup, there is default support to use the root disk for staging. So if a customer does use the image import features, there is a clear risk that the root disk will fill and cause issues.  The intended approach for OSP-13 is to disable it and customers can opt-in and will see the docs for adding the shared file system for HA configs.

Comment 9 Alan Bishop 2018-05-14 12:14:24 UTC
All (3 of 3) patches have been proposed downstream.

Comment 10 Alan Bishop 2018-05-14 20:59:05 UTC
One of the upstream patches was incorrect, and an update has been posted.

Comment 26 errata-xmlrpc 2018-06-27 13:35:05 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHEA-2018:2086


Note You need to log in before you can comment on or make changes to this bug.