Note: This bug is displayed in read-only format because the product is no longer active in Red Hat Bugzilla.

Bug 1641020

Summary: Undercloud and overcloud become unusable after some time
Product: Red Hat OpenStack Reporter: Raoul Scarazzini <rscarazz>
Component: rhosp-directorAssignee: Emilien Macchi <emacchi>
Status: CLOSED INSUFFICIENT_DATA QA Contact: Gurenko Alex <agurenko>
Severity: high Docs Contact:
Priority: high    
Version: 14.0 (Rocky)CC: bcafarel, bdobreli, chjones, dbecker, emacchi, mburns, michele, morazi
Target Milestone: ---Keywords: Triaged
Target Release: ---   
Hardware: x86_64   
OS: Linux   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2018-12-14 09:50:55 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On: 1641568    
Bug Blocks:    

Description Raoul Scarazzini 2018-10-19 12:54:31 UTC 
Description of problem:

After successfully deploying an environment, while keeping it untouched for more than one day, both undercloud and overcloud becomes unusable and strange issues start to happen.
For example, after sourcing stackrc file, it's impossible even to show the Nova hosts related to the overcloud, getting "ERROR (ClientException)".
Talking about the overcloud it's impossible for example to deploy a new heat stack on the overcloud, the error is:

ERROR: HEAT-E99001 Service neutron is not available for resource type OS::Neutron::SecurityGroup, reason: <html><body><h1>503 Service Unavailable</h1>
No server is available to handle this request.
</body></html>

Version-Release number of selected component (if applicable):

Puddle 2018-10-10.3

How reproducible:

I am working to understand how consistent is the failure.

Steps to Reproduce:
1. Deploy an OSP1 environment
2. Wait for at least more than one day
3. Try to use the undercloud or the overcloud for normal operations

Actual results:

Errors.

Expected results:

Everything should respond as expected.

Additional info:

The only workaround I found was to do a global restart of all the containers on the undercloud and every overcloud node.