Red Hat Satellite engineering is moving the tracking of its product development work on Satellite to Red Hat Jira (issues.redhat.com). If you're a Red Hat customer, please continue to file support cases via the Red Hat customer portal. If you're not, please head to the "Satellite project" in Red Hat Jira and file new tickets here. Individual Bugzilla bugs will be migrated starting at the end of May. If you cannot log in to RH Jira, please consult article #7032570. That failing, please send an e-mail to the RH Jira admins at rh-issues@redhat.com to troubleshoot your issue as a user management inquiry. The email creates a ServiceNow ticket with Red Hat. Individual Bugzilla bugs that are migrated will be moved to status "CLOSED", resolution "MIGRATED", and set with "MigratedToJIRA" in "Keywords". The link to the successor Jira issue will be found under "Links", have a little "two-footprint" icon next to it, and direct you to the "Satellite project" in Red Hat Jira (issue links are of type "https://issues.redhat.com/browse/SAT-XXXX", where "X" is a digit). This same link will be available in a blue banner at the top of the page informing you that that bug has been migrated.
Bug 1524279 - [RFE] Foreman-maintain should provide an option to clean its cache as we store previously completed checks.
Summary: [RFE] Foreman-maintain should provide an option to clean its cache as we stor...
Keywords:
Status: CLOSED WONTFIX
Alias: None
Product: Red Hat Satellite
Classification: Red Hat
Component: Satellite Maintain
Version: 6.3.0
Hardware: x86_64
OS: Linux
high
high
Target Milestone: Unspecified
Assignee: Anurag Patel
QA Contact: Nikhil Kathole
URL:
Whiteboard:
Depends On:
Blocks: 1496794
TreeView+ depends on / blocked
 
Reported: 2017-12-11 06:42 UTC by Ashish Humbe
Modified: 2019-08-01 18:34 UTC (History)
7 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2019-08-01 18:34:58 UTC
Target Upstream Version:
Embargoed:

Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Foreman Issue Tracker 23855 0 Normal New [RFE] Foreman-maintain should provide an option to clean data file that contains state data 2021-02-18 04:25:50 UTC

Description Ashish Humbe 2017-12-11 06:42:40 UTC 
Description of problem:
Foreman-maintain should provide an option to clean its cache as we store previously completed checks.

This will be helpful in the situations where upgrade failed in between while running "satellite-installer", if we want to start the upgrade from scratch there is no way we could see. 

The challenge we faced is :

First upgrade attempt failed with below error: 

Upgrade Step: Running installer...
 pulp-manage-db && touch /var/lib/pulp/init.flag returned 70 instead of one of [0]
 /Stage[main]/Pulp::Database/Exec[migrate_pulp_db]/returns: change from notrun to 0 failed: pulp-manage-db && touch /var/lib/pulp/init.flag returned 70 instead of one of [0]

While diagnosing issue we flushed iptable rules along with fixing pulp issue and that caused failure with upgrade via foreman maintain, upgrade failed with below error: 



Running Procedures after migrating to Satellite 6.2.z
================================================================================
katello-service start: 
/ No katello service to start                                         [OK]      
--------------------------------------------------------------------------------
Turn off maintenance mode:                                            [FAIL]
Failed executing iptables -D INPUT -j FOREMAN_MAINTAIN, exit status 1:
 iptables: No chain/target/match by that name.
--------------------------------------------------------------------------------
Scenario [Procedures after migrating to Satellite 6.2.z] failed.

The following steps ended up in failing state:

  [maintenance-mode-disable]

Resolve the failed steps and rerun
the command. In case the failures are false positives,
use --whitelist="maintenance-mode-disable"


To get the firewall rule created we tried to run the installer with " --phase pre_migrations " but in multiple attempts, it failed with error "Illegal quoting in line 3. while disabling sync plan." 

# foreman-maintain upgrade run  --target-version 6.2.z --whitelist="disk-io" --phase pre_migrations 
Running Procedures before migrating to Satellite 6.2.z
================================================================================
Turn on maintenance mode:                                             [OK]
--------------------------------------------------------------------------------
disable active sync plans: 
/ disabling sync plans                                                [FAIL]    
Illegal quoting in line 3.
--------------------------------------------------------------------------------
Scenario [Procedures before migrating to Satellite 6.2.z] failed.

The following steps ended up in failing state:

  [sync-plans-disable]

Resolve the failed steps and rerun
the command. In case the failures are false positives,
use --whitelist="sync-plans-disable"

On the final attempt of executing foreman-maintain it failed with an error and got the message "The upgrade failed and system was restored to pre-upgrade state."

******************************************************************************************
# foreman-maintain upgrade run  --target-version 6.2.z --whitelist="disk-io"
Running Checks before upgrading to Satellite 6.2.z
================================================================================
Skipping pre_upgrade_checks phase as it was already run before.
To enforce to run the phase, use `upgrade run --phase pre_upgrade_checks`

Scenario [Checks before upgrading to Satellite 6.2.z] failed.

The following steps ended up in failing state:

  [disk-io]

Resolve the failed steps and rerun
the command. In case the failures are false positives,
use --whitelist="disk-io"



Running Procedures before migrating to Satellite 6.2.z
================================================================================
Turn on maintenance mode:                                             [OK]
--------------------------------------------------------------------------------
disable active sync plans: 
/ disabling sync plans                                                [FAIL]    
Illegal quoting in line 3.
--------------------------------------------------------------------------------
Scenario [Procedures before migrating to Satellite 6.2.z] failed.

The following steps ended up in failing state:

  [sync-plans-disable]

Resolve the failed steps and rerun
the command. In case the failures are false positives,
use --whitelist="sync-plans-disable"



Running Procedures after migrating to Satellite 6.2.z
================================================================================
katello-service start: 
| Starting katello services..                                         [OK]      
--------------------------------------------------------------------------------
Turn off maintenance mode:                                            [OK]
--------------------------------------------------------------------------------
re-enable sync plans: 
/ Total 1 sync plans are now enabled.                                 [OK]      
--------------------------------------------------------------------------------


Running Checks after upgrading to Satellite 6.2.z
================================================================================
Check for paused tasks:                                               [OK]
--------------------------------------------------------------------------------
Check whether all services are running using hammer ping:             [OK]
--------------------------------------------------------------------------------


The upgrade failed and system was restored to pre-upgrade state.
******************************************************************************************

In the next execution of "foreman-maintain upgrade run  --target-version 6.2.z --whitelist="disk-io"  " upgrade was completed successfully without any problem, there were no extra steps/commands executed in between earlier and this successful attempt.
Comment 3 Anurag Patel 2017-12-15 12:09:34 UTC 
Discussed this issue with Ashish. This BZ can be broken down into the following themes:

1. Cleaning data files - /var/lib/foreman-maintain/data.yml is the file that contains state data. Renaming this file would clear previously completed check state.
2. Turning maintenance mode ON/OFF can be better done by the tool itself. 
3. 'Illegal quoting in line 3.' This seems to be an error (improper escaping?) that we should investigate and fix.

We'll assign BZs.
Comment 4 Ashish Humbe 2017-12-17 06:36:40 UTC 
Thank you Anurag,

We can track 1st issue in this bz for other 2 points we have different bz created. 

Turning maintenance mode ON/OFF can be better done by the tool itself.
https://bugzilla.redhat.com/show_bug.cgi?id=1526749

While running foreman-maintain it fails with "Illegal quoting in line 3."
https://bugzilla.redhat.com/show_bug.cgi?id=1523947
Comment 5 Anurag Patel 2018-01-10 13:46:06 UTC 
Ashish, would providing clear instructions in docs on how to clear state data by renaming data.yml be sufficient to address point #1?
Comment 6 Ashish Humbe 2018-01-11 08:25:03 UTC 
Hi Anurag,

Yeah, I think we should add it in two places:

1. In the formeman-maintain code, maybe yes/no option for the customer to clear old data at "Running Checks before upgrading to Satellite" step or at the end where upgrade fail message is displayed. 

2. In the official product guide.

Many time customers do not read the product guide properly so having an option to clean data.yml will make a good impact. 

What do you think?
Comment 7 Mike McCune 2018-01-29 18:24:07 UTC 
Moving to 1.1 Milestone as we are out of time for 1.0
Comment 8 Kavita 2018-04-03 12:38:21 UTC 
We can track first point here & second point, will track in other BZ under Documentation.


(In reply to Ashish Humbe from comment #6)
> Hi Anurag,
> 
> Yeah, I think we should add it in two places:
> 
> 1. In the formeman-maintain code, maybe yes/no option for the customer to
> clear old data at "Running Checks before upgrading to Satellite" step or at
> the end where upgrade fail message is displayed. 
> 
> 2. In the official product guide.
> 
> Many time customers do not read the product guide properly so having an
> option to clean data.yml will make a good impact. 
> 
> What do you think?
Comment 9 Bryan Kearney 2019-07-02 18:01:43 UTC 
The Satellite Team is attempting to provide an accurate backlog of bugzilla requests which we feel will be resolved in the next few releases. We do not believe this bugzilla will meet that criteria, and have plans to close it out in 1 month. This is not a reflection on the validity of the request, but a reflection of the many priorities for the product. If you have any concerns about this, feel free to contact Red Hat Technical Support or your account team. If we do not hear from you, we will close this bug out. Thank you.
Comment 10 Bryan Kearney 2019-08-01 18:34:58 UTC 
Thank you for your interest in Satellite 6. We have evaluated this request, and while we recognize that it is a valid request, we do not expect this to be implemented in the product in the foreseeable future. This is due to other priorities for the product, and not a reflection on the request itself. We are therefore closing this out as WONTFIX. If you have any concerns about this, please do not reopen. Instead, feel free to contact Red Hat Technical Support. Thank you.
Note You need to log in before you can comment on or make changes to this bug.