Bug 2040650
| Summary: | Upgrade or offline backup fails on RHEL8 due to missing iptables command | ||
|---|---|---|---|
| Product: | Red Hat Satellite | Reporter: | Vladimír Sedmík <vsedmik> |
| Component: | Satellite Maintain | Assignee: | Amit Upadhye <aupadhye> |
| Status: | CLOSED ERRATA | QA Contact: | Vladimír Sedmík <vsedmik> |
| Severity: | high | Docs Contact: | |
| Priority: | unspecified | ||
| Version: | 6.11.0 | CC: | apatel, aupadhye, ehelms, gtalreja, kgaikwad, lpramuk, sganar, vijsingh, zhunting |
| Target Milestone: | 6.11.0 | Keywords: | AutomationBlocker, Triaged |
| Target Release: | Unused | ||
| Hardware: | x86_64 | ||
| OS: | Linux | ||
| Whiteboard: | |||
| Fixed In Version: | rubygem-foreman_maintain-1.0.9 | Doc Type: | If docs needed, set a value |
| Doc Text: | Story Points: | --- | |
| Clone Of: | Environment: | ||
| Last Closed: | 2022-07-05 14:31:52 UTC | Type: | Bug |
| Regression: | --- | Mount Type: | --- |
| Documentation: | --- | CRM: | |
| Verified Versions: | Category: | --- | |
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |
| Cloudforms Team: | --- | Target Upstream Version: | |
| Embargoed: | |||
| Bug Depends On: | 2079357 | ||
| Bug Blocks: | 1693733 | ||
Created redmine issue https://projects.theforeman.org/issues/34282 from this bug *** Bug 2055208 has been marked as a duplicate of this bug. *** This happens also for 7.0.z upgrades on RHEL8:
# satellite-maintain upgrade run --target-version 7.0.z -w repositories-validate,repositories-setup -y
Checking for new version of satellite-maintain...
Nothing to update, can't find new version of satellite-maintain.
Running preparation steps required to run the next scenarios
================================================================================
Check whether system has any non Red Hat repositories (e.g.: EPEL) enabled:
| Checking repositories enabled on the system [OK]
--------------------------------------------------------------------------------
Running Checks before upgrading to Satellite 7.0.z
================================================================================
Clean old Kernel and initramfs files from tftp-boot: [OK]
--------------------------------------------------------------------------------
Check number of fact names in database: [OK]
--------------------------------------------------------------------------------
Check whether all services are running: [OK]
--------------------------------------------------------------------------------
Check whether all services are running using the ping call: [OK]
--------------------------------------------------------------------------------
Check for paused tasks: [OK]
--------------------------------------------------------------------------------
Check whether system is self-registered or not: [OK]
--------------------------------------------------------------------------------
Check to make sure root(/) partition has enough space: [OK]
--------------------------------------------------------------------------------
Check to make sure /var/lib/candlepin has enough space: [OK]
--------------------------------------------------------------------------------
Check to validate candlepin database: [OK]
--------------------------------------------------------------------------------
Check for running tasks: [OK]
--------------------------------------------------------------------------------
Check for old tasks in paused/stopped state: [OK]
--------------------------------------------------------------------------------
Check for pending tasks which are safe to delete: [OK]
--------------------------------------------------------------------------------
Check for tasks in planning state: [OK]
--------------------------------------------------------------------------------
Check to verify if any hotfix installed on system:
| Checking for presence of hotfix(es). It may take some time to verify.
[OK]
--------------------------------------------------------------------------------
Check whether system has any non Red Hat repositories (e.g.: EPEL) enabled:
| Checking repositories enabled on the system [OK]
--------------------------------------------------------------------------------
Check if TMOUT environment variable is set: [OK]
--------------------------------------------------------------------------------
Check if any upstream repositories are enabled on system:
/ Checking for presence of upstream repositories [OK]
--------------------------------------------------------------------------------
Check for roles that have filters with multiple resources attached: [OK]
--------------------------------------------------------------------------------
Check for duplicate permissions from database: [OK]
--------------------------------------------------------------------------------
Check if system has any non Red Hat RPMs installed (e.g.: Fedora): [OK]
--------------------------------------------------------------------------------
Check whether reports have correct associations: [OK]
--------------------------------------------------------------------------------
Check to validate yum configuration before upgrade: [OK]
--------------------------------------------------------------------------------
Validate availability of repositories: [SKIPPED]
--------------------------------------------------------------------------------
The pre-upgrade checks indicate that the system is ready for upgrade.
It's recommended to perform a backup at this stage.
Confirm to continue with the modification part of the upgrade (assuming yes)
Running Procedures before migrating to Satellite 7.0.z
================================================================================
disable active sync plans:
| Total 0 sync plans are now disabled. [OK]
--------------------------------------------------------------------------------
Add maintenance_mode chain to iptables: [FAIL]
Failed executing iptables -N FOREMAN_MAINTAIN, exit status 127:
sh: iptables: command not found
--------------------------------------------------------------------------------
Scenario [Procedures before migrating to Satellite 7.0.z] failed.
The following steps ended up in failing state:
[iptables-add-maintenance-mode-chain]
Moving this bug to POST for triage into Satellite since the upstream issue https://projects.theforeman.org/issues/34282 has been resolved. @aupadhye When the maintenance mode is lifted ? I get that maintenance mode is lifted after installer upgrade run I see following error in 6.11.z upgrade on RHEL8: # satellite-maintain upgrade run --target-version 6.11.z -w repositories-validate,repositories-setup,non-rh-packages -y ... Add maintenance_mode tables/chain to nftables/iptables: [OK] ... Running Migration scripts to Satellite 6.11.z ================================================================================ Setup repositories: [SKIPPED] -------------------------------------------------------------------------------- Unlock packages: [OK] -------------------------------------------------------------------------------- Update package(s) : [OK] -------------------------------------------------------------------------------- Procedures::Installer::Upgrade: [FAIL] Failed executing LANG=en_US.utf-8 satellite-installer --disable-system-checks, exit status 6: 2022-04-19 06:04:00 [NOTICE] [root] Loading installer configuration. This will take some time. 2022-04-19 06:04:07 [NOTICE] [root] Running installer with log based terminal output at level NOTICE. 2022-04-19 06:04:07 [NOTICE] [root] Use -l to set the terminal output log level to ERROR, WARN, NOTICE, INFO, or DEBUG. See --full-help for definitions. 2022-04-19 06:04:14 [WARN ] [pre] Skipping system checks. 2022-04-19 06:04:14 [WARN ] [pre] Skipping system checks. 2022-04-19 06:04:27 [NOTICE] [configure] Starting system configuration. 2022-04-19 06:04:48 [NOTICE] [configure] 250 configuration steps out of 2067 steps complete. 2022-04-19 06:05:06 [NOTICE] [configure] 500 configuration steps out of 2067 steps complete. 2022-04-19 06:05:06 [NOTICE] [configure] 750 configuration steps out of 2070 steps complete. 2022-04-19 06:05:14 [NOTICE] [configure] 1000 configuration steps out of 2075 steps complete. 2022-04-19 06:05:16 [NOTICE] [configure] 1250 configuration steps out of 2079 steps complete. 2022-04-19 06:06:23 [NOTICE] [configure] 1500 configuration steps out of 2079 steps complete. 2022-04-19 06:07:15 [NOTICE] [configure] 1750 configuration steps out of 2883 steps complete. 2022-04-19 06:07:15 [NOTICE] [configure] 2000 configuration steps out of 2883 steps complete. 2022-04-19 06:07:16 [NOTICE] [configure] 2250 configuration steps out of 2883 steps complete. 2022-04-19 06:07:38 [NOTICE] [configure] 2500 configuration steps out of 2883 steps complete. 2022-04-19 06:09:13 [NOTICE] [configure] 2750 configuration steps out of 2883 steps complete. 2022-04-19 06:09:14 [ERROR ] [configure] /Stage[main]/Foreman::Register/Foreman_host[foreman-sat.example.com]: Could not evaluate: Exception Failed to open TCP connection to sat.example.com:443 (Connection refused - connect(2) for "sat.example.com" port 443) in get request to: https://sat.example.com/api/v2/hosts?search=name%3D%22sat.example.com%22 2022-04-19 06:09:14 [ERROR ] [configure] Wrapped exception: 2022-04-19 06:09:14 [ERROR ] [configure] Failed to open TCP connection to sat.example.com:443 (Connection refused - connect(2) for "sat.example.com" port 443) 2022-04-19 06:10:24 [ERROR ] [configure] /Stage[main]/Foreman_proxy::Register/Foreman_host[foreman-proxy-sat.example.com]: Could not evaluate: Exception Failed to open TCP connection to sat.example.com:443 (Connection refused - connect(2) for "sat.example.com" port 443) in get request to: https://sat.example.com/api/v2/hosts?search=name%3D%22sat.example.com%22 >>> it looks like satellite-installer now goes against FW maintenance mode and fails (or maintenance mode fails to be removed?) Hello, I misunderstood the nft, we need to have rule to allow the localhost traffic. With the pr https://github.com/theforeman/foreman_maintain/pull/608 I can see its able to connect without connection refused messages, 2022-04-26 04:41:12 [DEBUG ] [configure] Foreman_host[foreman-dhcp-3-118.vms.sat.rdu2.redhat.com](provider=rest_v3): Received response 200 from request to https://dhcp-3-118.vms.sat.rdu2.redhat.com/api/v2/hosts?search=name%3D%22dhcp-3-118.vms.sat.rdu2.redhat.com%22 2022-04-26 04:41:12 [DEBUG ] [configure] /Stage[main]/Foreman::Register/Foreman_host[foreman-dhcp-3-118.vms.sat.rdu2.redhat.com]: Evaluated in 3.77 seconds 2022-04-26 04:41:12 [DEBUG ] [configure] /Stage[main]/Foreman::Register/Foreman_instance_host[foreman-dhcp-3-118.vms.sat.rdu2.redhat.com]: Starting to evaluate the resource (2408 of 2522) 2022-04-26 04:41:12 [DEBUG ] [configure] Foreman_instance_host[foreman-dhcp-3-118.vms.sat.rdu2.redhat.com](provider=rest_v3): Making get request to https://dhcp-3-118.vms.sat.rdu2.redhat.com/api/v2/hosts?search=name%3D%22dhcp-3-118.vms.sat.rdu2.redhat.com%22 2022-04-26 04:41:12 [DEBUG ] [configure] Foreman_instance_host[foreman-dhcp-3-118.vms.sat.rdu2.redhat.com](provider=rest_v3): Received response 200 from request to https://dhcp-3-118.vms.sat.rdu2.redhat.com/api/v2/hosts?search=name%3D%22dhcp-3-118.vms.sat.rdu2.redhat.com%22 Hello, `foreman-maintain maintenance-mode status` and `foreman-maintain maintenance-mode is-enabled` are failing with `undefined method `maintenance_mode_status?' for nil:NilClass` for which I have filed a BZ --> https://bugzilla.redhat.com/show_bug.cgi?id=2079357 Also, nftables is not pre-installed snap18, PR https://github.com/theforeman/foreman-packaging/pull/7807/files is not included in snap18. Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory (Moderate: Satellite 6.11 Release), and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHSA-2022:5498 The needinfo request[s] on this closed bug have been removed as they have been unresolved for 365 days |
Description of problem: Offline backup fails on RHEL8 due to missing iptables command, please add some dependency (move the component to Packaging or Installer) or update the f-m to avoid it's usage. Version-Release number of selected component (if applicable): Satellite 7.0.0 snap 5 RHEL 8.5 How reproducible: always Steps to Reproduce: 1. Have a fresh Satellite on RHEL 8.5 2. Run offline backup. Actual results: # foreman-maintain backup offline /tmp Starting backup: 2022-01-14 05:42:52 -0500 Running preparation steps required to run the next scenarios ================================================================================ Make sure Foreman DB is up: / Checking connection to the Foreman DB [OK] -------------------------------------------------------------------------------- Running Backup ================================================================================ Confirm turning off services is allowed: WARNING: This script will stop your services. Do you want to proceed?, [y(yes), q(quit)] y [OK] -------------------------------------------------------------------------------- Prepare backup Directory: Creating backup folder /tmp/satellite-backup-2022-01-14-05-42-52 [OK] -------------------------------------------------------------------------------- Check if the directory exists and is writable: [OK] -------------------------------------------------------------------------------- Generate metadata: - Saving metadata to metadata.yml [OK] -------------------------------------------------------------------------------- Detect features available in the local proxy: [OK] -------------------------------------------------------------------------------- disable active sync plans: / Total 0 sync plans are now disabled. [OK] -------------------------------------------------------------------------------- Add maintenance_mode chain to iptables: [FAIL] Failed executing iptables -N FOREMAN_MAINTAIN, exit status 127: sh: iptables: command not found -------------------------------------------------------------------------------- Scenario [Backup] failed. The following steps ended up in failing state: [iptables-add-maintenance-mode-chain] Resolve the failed steps and rerun the command. In case the failures are false positives, use --whitelist="iptables-add-maintenance-mode-chain" Running Failed backup cleanup ================================================================================ Start applicable services: Starting the following service(s): redis, postgresql, pulpcore-api, pulpcore-content, pulpcore-worker, pulpcore-worker, pulpcore-worker, pulpcore-worker, pulpcore-worker, pulpcore-worker, tomcat, dynflow-sidekiq@orchestrator, foreman, httpd, dynflow-sidekiq@worker-1, dynflow-sidekiq@worker-hosts-queue-1, foreman-proxy / All services started [OK] -------------------------------------------------------------------------------- re-enable sync plans: - Total 0 sync plans are now enabled. [OK] -------------------------------------------------------------------------------- Remove maintenance_mode chain from iptables: [OK] -------------------------------------------------------------------------------- Clean up backup directory: [OK] -------------------------------------------------------------------------------- Done with backup: 2022-01-14 05:43:30 -0500 Backup didn't finish. Incomplete backup was removed. Expected results: Successful backup Additional info: `yum install iptables` fixes the issue