Bug 1683081 - when candlepin is in "SUSPEND" mode, `hammer ping` still reports "ok" on candlepin
Summary: when candlepin is in "SUSPEND" mode, `hammer ping` still reports "ok" on cand...
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat Satellite
Classification: Red Hat
Component: Hammer - Content
Version: 6.5.0
Hardware: Unspecified
OS: Unspecified
unspecified
medium
Target Milestone: 6.5.0
Assignee: Jonathon Turel
QA Contact: Roman Plevka
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2019-02-26 08:39 UTC by Jan Hutař
Modified: 2019-11-05 22:46 UTC (History)
9 users (show)

Fixed In Version: tfm-rubygem-katello-3.10.0.28-1
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2019-05-14 12:40:17 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Foreman Issue Tracker 26228 0 Normal Closed when candlepin is in "SUSPEND" mode, `hammer ping` still reports "ok" on candlepin 2020-04-23 15:20:48 UTC
Red Hat Product Errata RHSA-2019:1222 0 None None None 2019-05-14 12:40:25 UTC

Description Jan Hutař 2019-02-26 08:39:30 UTC
Description of problem:
When candlepin is in "SUSPEND" mode, `hammer ping` still reports "ok" on candlepin


Version-Release number of selected component (if applicable):
satellite-6.5.0-6.beta.el7sat.noarch


How reproducible:
always


Steps to Reproduce:
1. # curl -k https://$HOSTNAME:8443/candlepin/status
{"mode":"SUSPEND","modeReason":"QPID_DOWN","modeChangeTime":"2019-02-23T21:33:14+0000","result":false,"version":"2.5.8","rulesVersion":"5.30.1","release":"1","standalone":true,"timeUTC":"2019-02-26T08:25:05+0000","rulesSource":"database","managerCapabilities":["instance_multiplier","derived_product","vcpu","cert_v3","remove_by_pool_id","syspurpose","storage_band","cores","hypervisors_async","org_level_content_access","guest_limit","ram","batch_bind"]}
    I assume it is possible to get into this state by stopping qpidd or
    qdrouterd or both
2. # hammer ping
candlepin:      
    Status:          ok
    Server Response: Duration: 21ms
candlepin_auth: 
    Status:          ok
    Server Response: Duration: 16ms
pulp:           
    Status:          ok
    Server Response: Duration: 64ms
pulp_auth:      
    Status:          ok
    Server Response: Duration: 38ms
foreman_tasks:  
    Status:          ok
    Server Response: Duration: 6ms


Actual results:
`hammer ping` is still happy although Candlepin is not operating:

[root@some_client ~]# subscription-manager unregister
Unregistering from: caps1.satperf.local:8443/rhsm
HTTP error code 503: Candlepin is in Suspend mode, please check /status resource to get more details


Expected results:
`hammer ping` (or `foreman-maintain health check` should fail on this)

Comment 2 Jonathon Turel 2019-03-04 22:38:24 UTC
Created redmine issue https://projects.theforeman.org/issues/26228 from this bug

Comment 3 Bryan Kearney 2019-03-11 16:07:26 UTC
Moving this bug to POST for triage into Satellite 6 since the upstream issue https://projects.theforeman.org/issues/26228 has been resolved.

Comment 5 Roman Plevka 2019-03-19 15:24:45 UTC
VERIFIED
on sat6.5.0-20

it takes a while, but the cnadlepin service status switches to FAIL finally:

[root@sat-6-5-qa-rhel7 foreman]# systemctl status qpidd.service
● qpidd.service - An AMQP message broker daemon.
   Loaded: loaded (/usr/lib/systemd/system/qpidd.service; enabled; vendor preset: disabled)
  Drop-In: /etc/systemd/system/qpidd.service.d
           └─wait-for-port.conf
   Active: inactive (dead) since Tue 2019-03-19 15:21:54 UTC; 1min 9s ago
     Docs: man:qpidd(1)
           http://qpid.apache.org/
  Process: 8116 ExecStartPost=/bin/bash -c while ! nc -z 127.0.0.1 5671; do sleep 1; done (code=exited, status=0/SUCCESS)
  Process: 8115 ExecStart=/usr/sbin/qpidd --config /etc/qpid/qpidd.conf (code=exited, status=0/SUCCESS)
 Main PID: 8115 (code=exited, status=0/SUCCESS)
...
[root@sat-6-5-qa-rhel7 foreman]# curl -k https://$HOSTNAME:8443/candlepin/status
{"mode":"SUSPEND","modeReason":"QPID_DOWN","modeChangeTime":"2019-03-19T15:22:13+0000","result":false,"version":"2.5.11","rulesVersion":"5.30.1","release":"1","standalone":true,"timeUTC":"2019-03-19T15:23:26+0000","rulesSource":"database","managerCapabilities":["instance_multiplier","derived_product","vcpu","cert_v3","remove_by_pool_id","syspurpose","storage_band","cores","hypervisors_async","org_level_content_access","guest_limit","ram","batch_bind"]}[root@sat-6-5-qa-rhel7 foreman]# 
[root@sat-6-5-qa-rhel7 foreman]# hammer ping
candlepin:      
    Status:          FAIL
    Server Response:
candlepin_auth: 
    Status:          FAIL
    Server Response:
pulp:           
    Status:          FAIL
    Server Response:
pulp_auth:      
    Status: FAIL
foreman_tasks:  
    Status:          ok
    Server Response: Duration: 2ms


[root@sat-6-5-qa-rhel7 foreman]# systemctl start qpidd.service
[root@sat-6-5-qa-rhel7 foreman]# hammer ping
candlepin:      
    Status:          ok
    Server Response: Duration: 15ms
candlepin_auth: 
    Status:          ok
    Server Response: Duration: 15ms
pulp:           
    Status:          ok
    Server Response: Duration: 42ms
pulp_auth:      
    Status:          ok
    Server Response: Duration: 35ms
foreman_tasks:  
    Status:          ok
    Server Response: Duration: 4ms

Comment 7 errata-xmlrpc 2019-05-14 12:40:17 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2019:1222


Note You need to log in before you can comment on or make changes to this bug.