Bug 1487559

Summary:	[RFE] foreman-maintain maintenance-mode command
Product:	Red Hat Satellite	Reporter:	Ivan Necas <inecas>
Component:	Satellite Maintain	Assignee:	Kavita <kgaikwad>
Status:	CLOSED CURRENTRELEASE	QA Contact:	Jameer Pathan <jpathan>
Severity:	medium	Docs Contact:
Priority:	unspecified
Version:	Unspecified	CC:	ahumbe, aupadhye, bbuckingham, bkearney, inecas, kgaikwad, lzap, mmccune, mvanderw, ofalk, peter.vreman, sthirugn, swadeley
Target Milestone:	Unspecified	Keywords:	FutureFeature
Target Release:	Unused
Hardware:	Unspecified
OS:	Unspecified
Whiteboard:
Fixed In Version:		Doc Type:	If docs needed, set a value
Doc Text:		Story Points:	---
Clone Of:		Environment:
Last Closed:	2020-03-05 16:17:26 UTC	Type:	Bug
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:
Bug Depends On:
Bug Blocks:	1122832, 1496794

Description Ivan Necas 2017-09-01 09:27:51 UTC

As part of the pre-migrations phase, we perform steps to temporary disable functionality to make sure the upgrades are run in isolation from outside, e.g.

* Disable enabled sync plans, stores information about the disabled sync plans to file to be able to restore it in SyncPlans::Enable

* Disable external traffic on port 443 to prevent the interference with the system while the uprade runs

The commands to work with the maintenance-mode:

1. foreman-maintain maintenance-mode status 
   Shows the status of the maintenance mode. Possible values:

   * off - the maintenance mode is off
   * on - the maintenance mode is on
   * partially on - there were some steps of the maintenance mode already set,  while some parts were not
   * not available - since foreman-maintain can run not only the satellite, on some installations (such as clients), this might not be avaiable

2. foreman-maintain maintenance-mode on
   Runs the steps to enable the maintenance mode

3. foreman-maintain maintenance-mode off
   Runs the steps to disable the maintenance mode


The exact steps to turn the maintenance mode on/off might differ from different versions and server vs proxy.

Comment 6 Bryan Kearney 2017-11-02 16:54:04 UTC

*** Bug 1314742 has been marked as a duplicate of this bug. ***

Comment 7 Ivan Necas 2018-01-09 15:09:15 UTC

Created redmine issue http://projects.theforeman.org/issues/22198 from this bug

Comment 8 Anurag Patel 2018-01-10 13:53:26 UTC

*** Bug 1526749 has been marked as a duplicate of this bug. ***

Comment 9 Satellite Program 2018-05-16 20:10:34 UTC

Upstream bug assigned to kgaikwad

Comment 10 Peter Vreman 2018-11-02 15:48:28 UTC

With Sat6.4 the system requirements are more reasonable now with 20GB memory and ~800GB disk space.

For me this BZ can be closed as delivered with Sat6.4

Comment 11 Bryan Kearney 2018-12-17 18:03:14 UTC

Ivan, how are you with comment 10?

Comment 12 Peter Vreman 2018-12-18 14:27:31 UTC

Looks like i typed a comment in the wrong BZ, https://bugzilla.redhat.com/show_bug.cgi?id=1487559#c10 was meant for BZ https://bugzilla.redhat.com/show_bug.cgi?id=1408380 and has nothing to do with maintenance mode.

Sorry for the confusion,
Peter

Comment 15 Bryan Kearney 2019-02-21 13:01:57 UTC

Moving this bug to POST for triage into Satellite 6 since the upstream issue https://projects.theforeman.org/issues/22198 has been resolved.

Comment 21 Jameer Pathan 2019-07-02 11:37:11 UTC

Verified

-Satellite version: satellite-6.6.0-5.beta.el7sat.noarch
-foreman-maintain version: rubygem-foreman_maintain-0.4.3-2.el7sat.noarch

Steps:
1. "maintenance-mode" command
# foreman-maintain maintenance-mode --help
Usage:
    foreman-maintain maintenance-mode [OPTIONS] SUBCOMMAND [ARG] ...

Parameters:
    SUBCOMMAND                    subcommand
    [ARG] ...                     subcommand arguments

Subcommands:
    start                         Start maintenance-mode
    stop                          Stop maintenance-mode
    status                        Get maintenance-mode status
    is-enabled                    Get maintenance-mode status code

Options:
    -h, --help                    print help

2. "maintenance-mode status" command
# foreman-maintain maintenance-mode status
Running Status of maintenance-mode
================================================================================
Check of maintenance-mode consistency: 
/ Running status of maintenance-mode                                            

Status of maintenance-mode: Off
- Iptables chain: absent
- sync plans: empty data
- cron jobs: not managed                                              [OK]
--------------------------------------------------------------------------------

2. "maintenance-mode is-enabled" command
# foreman-maintain maintenance-mode is-enabled
Running Show only status code of maintenance-mode
================================================================================
Showing status code for maintenance_mode: 
Maintenance mode is Off                                               [OK]
--------------------------------------------------------------------------------

1

3. "maintenance-mode start" command
- Create few sync-plans and add ":manage_crond: true" in "/etc/foreman-maintain/foreman_maintain.yml" 
file to manage cron service.

# foreman-maintain maintenance-mode start
Running Start maintenance-mode
================================================================================
disable active sync plans: 
- Total 3 sync plans are now disabled.                                [OK]      
--------------------------------------------------------------------------------
Add maintenance_mode chain to iptables:                               [OK]
--------------------------------------------------------------------------------
Stop cron service: 
Stopping the following service(s):

crond
\ All services stopped                                                [OK]      
--------------------------------------------------------------------------------

- Information about the disabled sync plans is stored in "/var/lib/foreman-maintain/data.yml" file to be able to use it to re-enable sync plans.
# cat /var/lib/foreman-maintain/data.yml 
---
:default:
  :sync_plans:
    :enabled: []
    :disabled:
    - 1
    - 2
    - 3

4. Check status of maintenace-mode
# foreman-maintain maintenance-mode status
Running Status of maintenance-mode
================================================================================
Check of maintenance-mode consistency: 
/ Running status of maintenance-mode                                            

Status of maintenance-mode: On
- Iptables chain: present
- sync plans: disabled
- cron jobs: not running                                              [OK]
--------------------------------------------------------------------------------

# foreman-maintain maintenance-mode is-enabled
Running Show only status code of maintenance-mode
================================================================================
Showing status code for maintenance_mode: 
Maintenance mode is On                                                [OK]
--------------------------------------------------------------------------------

0

5. "maintenance-mode stop" command
# foreman-maintain maintenance-mode stop
Running Stop maintenance-mode
================================================================================
re-enable sync plans: 
\ Total 3 sync plans are now enabled.                                 [OK]      
--------------------------------------------------------------------------------
Remove maintenance_mode chain from iptables:                          [OK]
--------------------------------------------------------------------------------
Start cron service: 
Starting the following service(s):

crond
\ All services started                                                [OK]      
--------------------------------------------------------------------------------

# foreman-maintain maintenance-mode status
Running Status of maintenance-mode
================================================================================
Check of maintenance-mode consistency: 
/ Running status of maintenance-mode                                            

Status of maintenance-mode: Off
- Iptables chain: absent
- sync plans: enabled
- cron jobs: running                                                  [OK]
--------------------------------------------------------------------------------


Observation:
1. "maintenance-mode status" gives status of:
- maintenance-mode is On or Off.
- Sync-plans enabled/disabled.
- Whether maintenance_mode chain is added to iptables.
- state of cron service

2. "maintenance-mode is-enabled" gives maintenance-mode status code,
0 when maintenance-mode is On and 1 if it's Off.

3. "maintenance-mode start" command:
- Adds maintenance_mode chain to iptables.
- Disables active sync-plans
- stops cron service. (if manage_crond is set to true in "/etc/foreman-maintain/foreman_maintain.yml")

4. "maintenance-mode stop" command:
- Removes maintenance_mode chain from iptables.
- re-enables sync-plans which got disabled during "maintenance-mode start"
- starts cron service.

Additional info:
- maintenance-mode subcommand is also supported on capsule for backup scenario as cron start/stop and iptables chain rule for 443 will work on capsule.
Foreman Maintain won't run sync-plan procedures on capsule as it checks for feature katello. (Reference comment #20 )

Comment 22 Bryan Kearney 2020-03-05 16:17:26 UTC

The current version of foreman_maintain includes this fix.

Comment 23 Bryan Kearney 2020-03-05 16:17:49 UTC

The current version of foreman_maintain includes this fix.