Bug 2055790

Summary: satellite-maintain service enable fails to enable all services previously disabled by satellite-maintain service disable
Product: Red Hat Satellite Reporter: Sam Wachira <swachira>
Component: Satellite MaintainAssignee: Evgeni Golov <egolov>
Status: CLOSED ERRATA QA Contact: Radek Mynar <rmynar>
Severity: high Docs Contact:
Priority: high    
Version: 6.10.2CC: ahumbe, apatel, dgupte, egolov, ehelms, gpayelka, jan3, jbreitwe, kgaikwad, pbadguja, pcreech, pdwyer, pmoravec, rlavi, rmynar, satellite6-bugs
Target Milestone: 6.14.0Keywords: Triaged
Target Release: Unused   
Hardware: x86_64   
OS: All   
Whiteboard:
Fixed In Version: foreman-maintain-1.3.2 Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of:
: 2211960 (view as bug list) Environment:
Last Closed: 2023-11-08 14:17:47 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Sam Wachira 2022-02-17 16:19:30 UTC
Description of problem:
After disabling Satellite services using the 'satellite-maintain service disable' command, not all services are re-enabled after running the 'satellite-maintain service enable' command.

Version-Release number of selected component (if applicable):
Satellite 6.10.2

How reproducible:
100%

Steps to Reproduce:
1. On a functional Satellite instance, check the status of services (should return Status: ok) 
# hammer ping

2. Disable applicable services:
# satellite-maintain service disable

3. Enable applicable services:
# satellite-maintain service enable

4. Reboot Satellite server

5. Check the status of services 
# hammer ping


Actual results:
Before making any changes, Satellite services return Status: ok.
~~~
[root@sat610 ~]# hammer ping
database:         
	Status:          ok
	Server Response: Duration: 0ms
katello_agent:    
	Status:          ok
	message:         0 Processed, 0 Failed
	Server Response: Duration: 0ms
candlepin:        
	Status:          ok
	Server Response: Duration: 27ms
candlepin_auth:   
	Status:          ok
	Server Response: Duration: 37ms
candlepin_events: 
	Status:          ok
	message:         84 Processed, 0 Failed
	Server Response: Duration: 0ms
katello_events:   
	Status:          ok
	message:         9 Processed, 0 Failed
	Server Response: Duration: 1ms
pulp3:            
	Status:          ok
	Server Response: Duration: 94ms
pulp3_content:    
	Status:          ok
	Server Response: Duration: 99ms
foreman_tasks:    
	Status:          ok
	Server Response: Duration: 10ms
~~~

Disable applicable services:
~~~
# satellite-maintain service disable
Disabling the following service(s):
rh-redis5-redis, postgresql, pulpcore-api, pulpcore-content, qdrouterd, qpidd, pulpcore-worker, pulpcore-worker, pulpcore-worker, pulpcore-worker, pulpcore-worker, pulpcore-worker, tomcat, dynflow-sidekiq@orchestrator, foreman, httpd, puppetserver, dynflow-sidekiq@worker-1, dynflow-sidekiq@worker-hosts-queue-1, foreman-proxy

Skipping disable for pulpcore-worker, pulpcore-worker, pulpcore-worker, pulpcore-worker, pulpcore-worker, pulpcore-worker.

| All services disabled                                               [OK]    
~~~

Enable applicable services. Note: The 'foreman, pulpcore-api, pulpcore-content' services were disabled but not enabled.
~~~
# satellite-maintain service enable
Enabling the following service(s):
rh-redis5-redis, postgresql, qdrouterd, qpidd, tomcat, dynflow-sidekiq@orchestrator, httpd, puppetserver, dynflow-sidekiq@worker-1, dynflow-sidekiq@worker-hosts-queue-1, foreman-proxy

- All services enabled                                                [OK]     
~~~~

After rebooting the Satellite server, pulp3_content and foreman_tasks services failed.
~~~
[root@sat610 ~]# hammer ping
database:         
	Status:          ok
	Server Response: Duration: 0ms
katello_agent:    
	Status:          ok
	message:         0 Processed, 0 Failed
	Server Response: Duration: 0ms
candlepin:        
	Status:          ok
	Server Response: Duration: 25ms
candlepin_auth:   
	Status:          ok
	Server Response: Duration: 30ms
candlepin_events: 
	Status:          ok
	message:         0 Processed, 0 Failed
	Server Response: Duration: 0ms
katello_events:   
	Status:          ok
	message:         0 Processed, 0 Failed
	Server Response: Duration: 0ms
pulp3:            
	Status:          ok
	Server Response: Duration: 6104ms
pulp3_content:    
	Status:          FAIL
	Server Response: Message: No pulpcore content apps are running at https://sat610.example.org/pulp/api/v3.
foreman_tasks:    
	Status:          FAIL
	Server Response: Message: some executors are not responding, check /foreman_tasks/dynflow/status
~~~


The 'foreman' and 'pulpcore-api' services started but remained disabled. The 'pulpcore-content' service remained disabled and inactive.
~~~
# systemctl status foreman
 foreman.service - Foreman
   Loaded: loaded (/usr/lib/systemd/system/foreman.service; disabled; vendor preset: disabled)
  Drop-In: /etc/systemd/system/foreman.service.d
   Active: active (running) since Thu 2022-02-17 15:36:12 GMT; 9min ago

# systemctl status pulpcore-api
 pulpcore-api.service - Pulp API Server
   Loaded: loaded (/etc/systemd/system/pulpcore-api.service; disabled; vendor preset: disabled)
   Active: active (running) since Thu 2022-02-17 15:36:14 GMT; 9min ago

# systemctl status pulpcore-content
 pulpcore-content.service - Pulp Content App
   Loaded: loaded (/etc/systemd/system/pulpcore-content.service; disabled; vendor preset: disabled)
   Active: inactive (dead)
~~~


After starting the pulpcore-content service manually, all Satellite services return Status: ok. 
~~~
# hammer ping
database:         
	Status:          ok
	Server Response: Duration: 0ms
katello_agent:    
	Status:          ok
	message:         0 Processed, 0 Failed
	Server Response: Duration: 0ms
candlepin:        
	Status:          ok
	Server Response: Duration: 33ms
candlepin_auth:   
	Status:          ok
	Server Response: Duration: 24ms
candlepin_events: 
	Status:          ok
	message:         0 Processed, 0 Failed
	Server Response: Duration: 0ms
katello_events:   
	Status:          ok
	message:         0 Processed, 0 Failed
	Server Response: Duration: 1ms
pulp3:            
	Status:          ok
	Server Response: Duration: 197ms
pulp3_content:    
	Status:          ok
	Server Response: Duration: 246ms
foreman_tasks:    
	Status:          ok
	Server Response: Duration: 3ms
~~~


Expected results:
All services disabled by 'satellite-maintain service disable' should be enabled  'satellite-maintain service enable'.
If enabling the services fails, a reason should be given for the failed enable.


Additional info:

Comment 4 Pavel Moravec 2022-07-22 13:04:05 UTC
*** Bug 2058269 has been marked as a duplicate of this bug. ***

Comment 7 Evgeni Golov 2023-06-02 06:35:04 UTC
Created redmine issue https://projects.theforeman.org/issues/36464 from this bug

Comment 8 Bryan Kearney 2023-06-02 08:02:16 UTC
Upstream bug assigned to egolov

Comment 9 Bryan Kearney 2023-06-02 08:02:19 UTC
Moving this bug to POST for triage into Satellite since the upstream issue https://projects.theforeman.org/issues/36464 has been resolved.

Comment 10 Radek Mynar 2023-06-20 11:58:46 UTC
'satellite-maintain service list' reports same status before and after disabling and re-enabling services (satellite-maintain service disable; satellite-maintain service enable; reboot). Also 'hammer ping' reports all ok.

VERIFIED with Satellite 6.14 SNAP4.0 @RHEL8.8

Comment 11 Eric Helms 2023-09-06 00:37:45 UTC
*** Bug 2095912 has been marked as a duplicate of this bug. ***

Comment 14 errata-xmlrpc 2023-11-08 14:17:47 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (Important: Satellite 6.14 security and bug fix update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2023:6818