Bug 1417668
Summary: | podfying cfme: evmserver.service restart would cause cfme container to be killed and re-start | ||
---|---|---|---|
Product: | Red Hat CloudForms Management Engine | Reporter: | Dafna Ron <dron> |
Component: | cfme-openshift-app | Assignee: | Franco Bladilo <fbladilo> |
Status: | CLOSED CURRENTRELEASE | QA Contact: | Einat Pacifici <epacific> |
Severity: | medium | Docs Contact: | Red Hat CloudForms Documentation <cloudforms-docs> |
Priority: | medium | ||
Version: | 5.7.0 | CC: | dron, fbladilo, jhardy, lavenel |
Target Milestone: | GA | ||
Target Release: | cfme-future | ||
Hardware: | x86_64 | ||
OS: | Linux | ||
Whiteboard: | container | ||
Fixed In Version: | Doc Type: | If docs needed, set a value | |
Doc Text: | Story Points: | --- | |
Clone Of: | Environment: | ||
Last Closed: | 2018-07-02 09:15:06 UTC | Type: | Bug |
Regression: | --- | Mount Type: | --- |
Documentation: | --- | CRM: | |
Verified Versions: | Category: | --- | |
oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |
Cloudforms Team: | Container Management | Target Upstream Version: | |
Embargoed: |
Description
Dafna Ron
2017-01-30 15:44:12 UTC
Franco, Why is this happening, Is it the liveness check ? Yes it is the liveness probe, now, we might have solved this issue (as a bonus) via : https://github.com/ManageIQ/manageiq-pods/pull/112 I haven't tested it but I must also admit I'm still unsure about the use case of restarting EVM service within container. Well the use case of restarting EVM service is not so common, but I am sure it will be required for debugging in a complex customer environment (happens sometimes on high profile customers), so this needs to be solved. The question is do we need to make sure that the POD is not rescheduled when we bring the EVM service down for a long perios of time ? Given comment #3 Dafna - can you please check whether the initial behavior you reported in this bug is still happening (evm service restart) ? There are some changes in cfme side and on podfied side but basically it can be reproduced by disabling the evm-watchdog.service and restart the mserverd.service you can see that the pod becomes 0/1 ready once we restart the evm [root@dafna-pods-master ~]# oc get pods NAME READY STATUS RESTARTS AGE cloudforms-0 1/1 Running 0 4d memcached-1-bby93 1/1 Running 0 4d postgresql-1-9y9hd 1/1 Running 0 4d [root@dafna-pods-master ~]# oc get pods NAME READY STATUS RESTARTS AGE cloudforms-0 1/1 Running 0 4d memcached-1-bby93 1/1 Running 0 4d postgresql-1-9y9hd 1/1 Running 0 4d [root@dafna-pods-master ~]# watch 'co get pods' [root@dafna-pods-master ~]# [root@dafna-pods-master ~]# watch 'oc get pods' [root@dafna-pods-master ~]# oc get pods NAME READY STATUS RESTARTS AGE cloudforms-0 0/1 Running 0 4d memcached-1-bby93 1/1 Running 0 4d postgresql-1-9y9hd 1/1 Running 0 4d [root@dafna-pods-master ~]# oc get pods NAME READY STATUS RESTARTS AGE cloudforms-0 0/1 Running 0 4d memcached-1-bby93 1/1 Running 0 4d postgresql-1-9y9hd 1/1 Running 0 4d [root@dafna-pods-master ~]# oc get pods NAME READY STATUS RESTARTS AGE cloudforms-0 0/1 Running 0 4d memcached-1-bby93 1/1 Running 0 4d postgresql-1-9y9hd 1/1 Running 0 4d [root@dafna-pods-master ~]# oc get pods NAME READY STATUS RESTARTS AGE cloudforms-0 0/1 Running 0 4d memcached-1-bby93 1/1 Running 0 4d postgresql-1-9y9hd 1/1 Running 0 4d [root@dafna-pods-master ~]# oc get pods NAME READY STATUS RESTARTS AGE cloudforms-0 0/1 Running 0 4d memcached-1-bby93 1/1 Running 0 4d postgresql-1-9y9hd 1/1 Running 0 4d [root@dafna-pods-master ~]# [root@dafna-pods-master ~]# [root@dafna-pods-master ~]# oc get pods NAME READY STATUS RESTARTS AGE cloudforms-0 0/1 Running 0 4d memcached-1-bby93 1/1 Running 0 4d postgresql-1-9y9hd 1/1 Running 0 4d [root@dafna-pods-master ~]# oc get pods NAME READY STATUS RESTARTS AGE cloudforms-0 1/1 Running 0 4d memcached-1-bby93 1/1 Running 0 4d postgresql-1-9y9hd 1/1 Running 0 4d Dafna, That is the intended behavior, as your restart EVM server, the pod would become not ready as it is not responding successfully. The most important piece if you notice is that the POD is not getting KILLED/RESTARTED by the liveness probe which was the case before. Barak, I must advice that for debugging purposes I would turn off the liveness check, as it could potentially kill the container in the middle of a debugging session and become counter productive. |