Red Hat Bugzilla – Bug 1192862
Glance fails to start after RHEL-OSP6 install with Ceph backend (missing known_stores)
Last modified: 2016-11-07 23:08:30 EST
Description of problem: After deployment of RHEL-OSP6 and manual installation of Ceph backend, set "Include Glance" manually to true (in Advanced Configuration/Controller) but glance fails to start. Version-Release number of selected component (if applicable): How reproducible: Steps to Reproduce: 1. Deploy RHEL-OSP6 in HA on 3 controllers 2. Manually setup Ceph (HEALTH_OK) 3. Set the flag "Install glance" to true 4. Watch puppet fails Actual results: Expected results: glance-api starting regularly. Additional info: After successful deployment (RHEL-OSP6 in HA, on hardware), I proceed to install Ceph manually. After successful Ceph cluster creation, I went back to Foreman and set "Install Glance" to true manually, run puppet on the controllers and observed glance failing. I then run glance manually: sudo -u glance -H glance-api and got the error ERROR: Store for scheme rbd not found in glance-api.conf, the line for stores is commented out; however, setting it manually to: stores=glance.store.rbd.Store gets reverted by puppet: Notice: /Stage[main]/Glance::Api/Glance_api_config[glance_store/stores]/ensure: removed Info: /Stage[main]/Glance::Api/Glance_api_config[glance_store/stores]: Scheduling refresh of Exec[glance-manage db_sync] Info: /Stage[main]/Glance::Api/Glance_api_config[glance_store/stores]: Scheduling refresh of Service[glance-api] Notice: /Stage[main]/Glance::Registry/Exec[glance-manage db_sync]: Triggered 'refresh' from 1 events Info: /Stage[main]/Glance::Registry/Exec[glance-manage db_sync]: Scheduling refresh of Service[glance-api] Info: /Stage[main]/Glance::Registry/Exec[glance-manage db_sync]: Scheduling refresh of Service[glance-registry] Notice: /Stage[main]/Glance::Registry/Service[glance-registry]: Triggered 'refresh' from 1 events Error: Could not start Service[glance-api]: Execution of '/usr/bin/systemctl start openstack-glance-api' returned 1: Job for openstack-glance-api.service failed. See 'systemctl status openstack-glance-api.service' and 'journalctl -xn' for details. Wrapped exception: Execution of '/usr/bin/systemctl start openstack-glance-api' returned 1: Job for openstack-glance-api.service failed. See 'systemctl status openstack-glance-api.service' and 'journalctl -xn' for details. Error: /Stage[main]/Glance::Api/Service[glance-api]/ensure: change from stopped to running failed: Could not start Service[glance-api]: Execution of '/usr/bin/systemctl start openstack-glance-api' returned 1: Job for openstack-glance-api.service failed. See 'systemctl status openstack-glance-api.service' and 'journalctl -xn' for details. The YAML file of the controller doesn't contain a reference to $known_stores in glance::api (see https://review.openstack.org/#/c/82699/). Related to https://bugzilla.redhat.com/show_bug.cgi?id=1183946
I did the following to work around the issue: I changed /etc/puppet/environments/production/modules/glance/manifests/api.pp to include the rbd store: root@idospinstaller manifests]# diff -Nur api.pp.orig api.pp --- api.pp.orig 2015-02-17 16:28:06.837029944 +0100 +++ api.pp 2015-02-17 16:28:27.555821296 +0100 @@ -197,7 +197,7 @@ $cert_file = false, $key_file = false, $ca_file = false, - $known_stores = false, + $known_stores = ['glance.store.rbd.Store'], $database_connection = 'sqlite:///var/lib/glance/glance.sqlite', $database_idle_timeout = 3600, $image_cache_dir = '/var/lib/glance/image-cache', This allowed me to work through the installation w/o further issues (so far ;) Cheers, Alex
This is a bug in foreman-quickstart-puppet-module ( astapor). The openstack-puppet-module class glance::api is expecting the known_stores to be passed by foreman-quickstart-puppet-module. Basically the real fix is to check the backend and set the $known_stores variable in the quickstart glance module. File to be fixed(quickstart): openstack-foreman-installer/puppet/modules/quickstack/manifests/glance.pp
*** Bug 1196921 has been marked as a duplicate of this bug. ***
(In reply to Alex Stellwag from comment #5) > I did the following to work around the issue: > > I changed > /etc/puppet/environments/production/modules/glance/manifests/api.pp to > include the rbd store: > > root@idospinstaller manifests]# diff -Nur api.pp.orig api.pp > --- api.pp.orig 2015-02-17 16:28:06.837029944 +0100 > +++ api.pp 2015-02-17 16:28:27.555821296 +0100 > @@ -197,7 +197,7 @@ > $cert_file = false, > $key_file = false, > $ca_file = false, > - $known_stores = false, > + $known_stores = ['glance.store.rbd.Store'], > $database_connection = 'sqlite:///var/lib/glance/glance.sqlite', > $database_idle_timeout = 3600, > $image_cache_dir = '/var/lib/glance/image-cache', > > This allowed me to work through the installation w/o further issues (so far > ;) > > Cheers, Alex I am currently testing a patch that does exactly this, though it will substitute in the value of $backend parameter from the glance manifest that Rajini mentioned (still using the long name for the store though). the only possible issue I see in this is that the short name file seems to have the long name with 'filesystem', so I may have to munge the value for that case.
Patch posted: https://github.com/redhat-openstack/astapor/pull/483
*** Bug 1183946 has been marked as a duplicate of this bug. ***
Merged by Crag
what is the workaround till A2? What is the way to remove glance-api.conf from puppet control after initial install?
The only 2 workarounds I can think of are to add the puppet change to an existing deployment, or to not rerun puppet after configuration, manually setting the config file value as desired.
the Staypuft didn't installed Glance: Bug 1207158
A new bug is filed for that issue which is not a bug. moving back to ON_QA
The parameter 'stores' is been set: stores=glance.store.rbd.Store,glance.store.http.Store in staypuft version: ruby193-rubygem-staypuft-0.5.22-1.el7ost.noarch
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://rhn.redhat.com/errata/RHSA-2015-0791.html