Bug 1447859

Summary: Control plane service restart during compute node scaleout
Product: Red Hat OpenStack Reporter: VIKRANT <vaggarwa>
Component: puppet-novaAssignee: Emilien Macchi <emacchi>
Status: CLOSED ERRATA QA Contact: Gurenko Alex <agurenko>
Severity: high Docs Contact:
Priority: high    
Version: 11.0 (Ocata)CC: aschultz, jjoyce, jschluet, mburns, mnadeem, rhel-osp-director-maint, slinaber, tvignaud
Target Milestone: z1Keywords: Triaged, ZStream
Target Release: 11.0 (Ocata)   
Hardware: x86_64   
OS: Linux   
Whiteboard:
Fixed In Version: puppet-nova-10.4.0-6.el7ost Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of:
: 1455175 (view as bug list) Environment:
Last Closed: 2017-07-19 17:04:56 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On: 1472142    
Bug Blocks: 1455175    
Attachments:
Description Flags
controller sosreport before scaleout
none
controller sosreport after scaleout
none
compute sosreport before scaleout
none
compute sosreport after scaleout
none
undercloud none

Description VIKRANT 2017-05-04 05:25:46 UTC
Description of problem:

Services are getting restarted on existing controller and compute nodes during the scaleout of compute nodes. 

~~~
Controller:

Not restarted: neutron, nova, glance, rabbitmq, cinder, swift, ceilometer,
redis, heat

Restarted: httpd, keystone (as httpd was restarted hence keystone restart
was expected)

Compute:

Not restarted: neutron,

Restarted: nova-compute
~~~


Version-Release number of selected component (if applicable):
RHEL OSP 11

How reproducible:
Everytime.

Steps to Reproduce:
1. Deploy openstack setup with 1 controller and 1 compute.
2. Capture the sosreport from the overlcoud nodes. 
2. Try to perform scaleout from 1 to 2 compute node. 
3. Capture the sosreport again and compare the services which got restarted after scaleout. 

Actual results:
Services are getting restarted. 

Expected results:
No service should get restarted

Additional info:

More information in coming in next comment.

Comment 7 VIKRANT 2017-05-06 01:47:25 UTC
Created attachment 1276727 [details]
controller sosreport before scaleout

Comment 8 VIKRANT 2017-05-06 01:49:09 UTC
Created attachment 1276728 [details]
controller sosreport after scaleout

Comment 9 VIKRANT 2017-05-06 01:49:49 UTC
Created attachment 1276729 [details]
compute sosreport before scaleout

Comment 10 VIKRANT 2017-05-06 01:50:44 UTC
Created attachment 1276730 [details]
compute sosreport after scaleout

Comment 11 VIKRANT 2017-05-06 01:52:14 UTC
Created attachment 1276731 [details]
undercloud

Comment 12 Alex Schultz 2017-05-15 20:02:56 UTC
For the controllers, the httpd was restarted due to the gnocchi/ceilometer the configuration being removed in step3 but being reapplied in step4. This needs to be pulled downstream.


So for the compute node, the services restarted due to libvirt/enabled_perf_events being 'created'. But if you look in the configuration, it's not actually set in the configuration file. Need to track down why this is happening, usually it's because of an issue handling '' or [].  This may not be fixed yet.

May  4 15:57:25 host-192-168-24-12 os-collect-config: #033[mNotice: /Stage[main]/Nova::Compute::Libvirt/Nova_config[libvirt/enabled_perf_events]/ensure: created#033[0m
May  4 15:57:25 host-192-168-24-12 os-collect-config: #033[mNotice: /Stage[main]/Nova::Deps/Anchor[nova::config::end]: Triggered 'refresh' from 1 events#033[0m
May  4 15:57:25 host-192-168-24-12 os-collect-config: #033[mNotice: /Stage[main]/Nova::Deps/Anchor[nova::service::begin]: Triggered 'refresh' from 1 events#033[0m
May  4 15:57:25 host-192-168-24-12 os-collect-config: #033[mNotice: /Stage[main]/Nova::Compute/Nova::Generic_service[compute]/Service[nova-compute]: Triggered 'refresh' from 1 events#033[0m
May  4 15:57:25 host-192-168-24-12 os-collect-config: #033[mNotice: /Stage[main]/Nova::Deps/Anchor[nova::service::end]: Triggered 'refresh' from 1 events#033[0m

Comment 14 Alex Schultz 2017-07-18 19:00:01 UTC
Verified nova-compute is not restarted on a compute scaleout action with puppet-nova-10.4.1-1.el7ost.noarch

Comment 16 errata-xmlrpc 2017-07-19 17:04:56 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2017:1778