Description of problem: Satellite maintain upgrade check failing when we have an epel repository enabled on satellite 6.7 Beta Version-Release number of selected component (if applicable): satellite-6.7.0-5.beta.el7sat.noarch satellite-maintain-0.0.1-1.el7sat.noarch How reproducible: New installation Steps to Reproduce: 1. Install new Satellite 6.7 beta 2. Try to perform minor update steps with # satellite-maintain upgrade check 3. Actual results: # satellite-maintain upgrade check --target-version 6.7.z --whitelist="disk-performance" Running preparation steps required to run the next scenarios ================================================================================ Install packages: Loaded plugins: foreman-protector, product-id, search-disabled-repos, subscription-manager ansible-runner | 2.9 kB 00:00:00 rhel-7-server-ansible-2.8-rpms | 4.0 kB 00:00:00 rhel-7-server-rh-common-rpms | 3.8 kB 00:00:00 rhel-7-server-rpms | 3.5 kB 00:00:00 rhel-7-server-satellite-maintenance-6-beta-rpms | 3.8 kB 00:00:00 rhel-server-7-satellite-6-beta-rpms | 4.0 kB 00:00:00 rhel-server-rhscl-7-rpms | 3.4 kB 00:00:00 (1/4): rhel-7-server-rpms/7Server/x86_64/updateinfo | 3.5 MB 00:00:01 (2/4): rhel-7-server-rh-common-rpms/7Server/x86_64/updateinfo | 34 kB 00:00:02 (3/4): rhel-7-server-rh-common-rpms/7Server/x86_64/primary_db | 119 kB 00:00:02 (4/4): rhel-7-server-rpms/7Server/x86_64/primary_db | 66 MB 00:00:07 WARNING: Excluding 12248 updates due to foreman-protector. Use foreman-maintain packages install/update <package> to safely install packages without restrictions. Use foreman-maintain upgrade run for full upgrade. Resolving Dependencies --> Running transaction check ---> Package fio.x86_64 0:3.7-1.el7 will be installed --> Processing Dependency: libibverbs.so.1(IBVERBS_1.0)(64bit) for package: fio-3.7-1.el7.x86_64 --> Processing Dependency: libibverbs.so.1(IBVERBS_1.1)(64bit) for package: fio-3.7-1.el7.x86_64 --> Processing Dependency: libpmem.so.1(LIBPMEM_1.0)(64bit) for package: fio-3.7-1.el7.x86_64 --> Processing Dependency: libpmemblk.so.1(LIBPMEMBLK_1.0)(64bit) for package: fio-3.7-1.el7.x86_64 --> Processing Dependency: librdmacm.so.1(RDMACM_1.0)(64bit) for package: fio-3.7-1.el7.x86_64 --> Processing Dependency: libibverbs.so.1()(64bit) for package: fio-3.7-1.el7.x86_64 --> Processing Dependency: libpmem.so.1()(64bit) for package: fio-3.7-1.el7.x86_64 --> Processing Dependency: libpmemblk.so.1()(64bit) for package: fio-3.7-1.el7.x86_64 --> Processing Dependency: librados.so.2()(64bit) for package: fio-3.7-1.el7.x86_64 --> Processing Dependency: librbd.so.1()(64bit) for package: fio-3.7-1.el7.x86_64 --> Processing Dependency: librdmacm.so.1()(64bit) for package: fio-3.7-1.el7.x86_64 --> Running transaction check ---> Package libibverbs.x86_64 0:22.1-3.el7 will be installed --> Processing Dependency: rdma-core(x86-64) = 22.1-3.el7 for package: libibverbs-22.1-3.el7.x86_64 ---> Package libpmem.x86_64 0:1.5.1-2.1.el7 will be installed ---> Package libpmemblk.x86_64 0:1.5.1-2.1.el7 will be installed --> Processing Dependency: libdaxctl.so.1(LIBDAXCTL_2)(64bit) for package: libpmemblk-1.5.1-2.1.el7.x86_64 --> Processing Dependency: libndctl.so.6(LIBNDCTL_1)(64bit) for package: libpmemblk-1.5.1-2.1.el7.x86_64 --> Processing Dependency: libndctl.so.6(LIBNDCTL_14)(64bit) for package: libpmemblk-1.5.1-2.1.el7.x86_64 --> Processing Dependency: libndctl.so.6(LIBNDCTL_3)(64bit) for package: libpmemblk-1.5.1-2.1.el7.x86_64 --> Processing Dependency: libdaxctl.so.1()(64bit) for package: libpmemblk-1.5.1-2.1.el7.x86_64 --> Processing Dependency: libndctl.so.6()(64bit) for package: libpmemblk-1.5.1-2.1.el7.x86_64 ---> Package librados2.x86_64 1:10.2.5-4.el7 will be installed ---> Package librbd1.x86_64 1:10.2.5-4.el7 will be installed ---> Package librdmacm.x86_64 0:22.1-3.el7 will be installed --> Running transaction check ---> Package daxctl-libs.x86_64 0:64.1-2.el7 will be installed ---> Package ndctl-libs.x86_64 0:64.1-2.el7 will be installed ---> Package rdma-core.x86_64 0:22.1-3.el7 will be installed --> Finished Dependency Resolution Dependencies Resolved =================================================================================================================================================================================================================== Package Arch Version Repository Size =================================================================================================================================================================================================================== Installing: fio x86_64 3.7-1.el7 rhel-7-server-rpms 467 k Installing for dependencies: daxctl-libs x86_64 64.1-2.el7 rhel-7-server-rpms 26 k libibverbs x86_64 22.1-3.el7 rhel-7-server-rpms 267 k libpmem x86_64 1.5.1-2.1.el7 rhel-7-server-rpms 59 k libpmemblk x86_64 1.5.1-2.1.el7 rhel-7-server-rpms 80 k librados2 x86_64 1:10.2.5-4.el7 rhel-7-server-rpms 1.8 M librbd1 x86_64 1:10.2.5-4.el7 rhel-7-server-rpms 2.4 M librdmacm x86_64 22.1-3.el7 rhel-7-server-rpms 63 k ndctl-libs x86_64 64.1-2.el7 rhel-7-server-rpms 64 k rdma-core x86_64 22.1-3.el7 rhel-7-server-rpms 50 k Transaction Summary =================================================================================================================================================================================================================== Install 1 Package (+9 Dependent packages) Total download size: 5.3 M Installed size: 16 M Is this ok [y/d/N]: y Downloading packages: (1/10): daxctl-libs-64.1-2.el7.x86_64.rpm | 26 kB 00:00:00 (2/10): libibverbs-22.1-3.el7.x86_64.rpm | 267 kB 00:00:00 (3/10): libpmem-1.5.1-2.1.el7.x86_64.rpm | 59 kB 00:00:00 (4/10): fio-3.7-1.el7.x86_64.rpm | 467 kB 00:00:02 (5/10): librados2-10.2.5-4.el7.x86_64.rpm | 1.8 MB 00:00:00 (6/10): librbd1-10.2.5-4.el7.x86_64.rpm | 2.4 MB 00:00:00 (7/10): librdmacm-22.1-3.el7.x86_64.rpm | 63 kB 00:00:00 (8/10): libpmemblk-1.5.1-2.1.el7.x86_64.rpm | 80 kB 00:00:01 (9/10): ndctl-libs-64.1-2.el7.x86_64.rpm | 64 kB 00:00:00 (10/10): rdma-core-22.1-3.el7.x86_64.rpm | 50 kB 00:00:00 ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Total 1.1 MB/s | 5.3 MB 00:00:04 Running transaction check Running transaction test Transaction test succeeded Running transaction Installing : libpmem-1.5.1-2.1.el7.x86_64 1/10 Installing : rdma-core-22.1-3.el7.x86_64 2/10 Installing : libibverbs-22.1-3.el7.x86_64 3/10 Installing : 1:librados2-10.2.5-4.el7.x86_64 4/10 Installing : daxctl-libs-64.1-2.el7.x86_64 5/10 Installing : ndctl-libs-64.1-2.el7.x86_64 6/10 Installing : libpmemblk-1.5.1-2.1.el7.x86_64 7/10 Installing : 1:librbd1-10.2.5-4.el7.x86_64 8/10 Installing : librdmacm-22.1-3.el7.x86_64 9/10 Installing : fio-3.7-1.el7.x86_64 10/10 Verifying : 1:librbd1-10.2.5-4.el7.x86_64 1/10 Verifying : daxctl-libs-64.1-2.el7.x86_64 2/10 Verifying : libibverbs-22.1-3.el7.x86_64 3/10 Verifying : fio-3.7-1.el7.x86_64 4/10 Verifying : 1:librados2-10.2.5-4.el7.x86_64 5/10 Verifying : libpmemblk-1.5.1-2.1.el7.x86_64 6/10 Verifying : librdmacm-22.1-3.el7.x86_64 7/10 Verifying : rdma-core-22.1-3.el7.x86_64 8/10 Verifying : libpmem-1.5.1-2.1.el7.x86_64 9/10 Verifying : ndctl-libs-64.1-2.el7.x86_64 10/10 Installed: fio.x86_64 0:3.7-1.el7 Dependency Installed: daxctl-libs.x86_64 0:64.1-2.el7 libibverbs.x86_64 0:22.1-3.el7 libpmem.x86_64 0:1.5.1-2.1.el7 libpmemblk.x86_64 0:1.5.1-2.1.el7 librados2.x86_64 1:10.2.5-4.el7 librbd1.x86_64 1:10.2.5-4.el7 librdmacm.x86_64 0:22.1-3.el7 ndctl-libs.x86_64 0:64.1-2.el7 rdma-core.x86_64 0:22.1-3.el7 Complete! [OK] -------------------------------------------------------------------------------- Running Checks before upgrading to Satellite 6.7.z ================================================================================ Check number of fact names in database: [OK] -------------------------------------------------------------------------------- Check for verifying syntax for ISP DHCP configurations: [SKIPPED] DHCP feature is not enabled -------------------------------------------------------------------------------- Check whether all services are running: [OK] -------------------------------------------------------------------------------- Check whether all services are running using the ping call: [OK] -------------------------------------------------------------------------------- Check for paused tasks: [OK] -------------------------------------------------------------------------------- Check to verify no empty CA cert requests exist: [OK] -------------------------------------------------------------------------------- Check whether system is self-registered or not: [OK] -------------------------------------------------------------------------------- Check to make sure root(/) partition has enough space: [OK] -------------------------------------------------------------------------------- Check to validate candlepin database: [OK] -------------------------------------------------------------------------------- Check if EPEL repository enabled on system: \ Checking for presence of EPEL repository [FAIL] System is subscribed to EPEL repository -------------------------------------------------------------------------------- Check for running tasks: [OK] -------------------------------------------------------------------------------- Check for old tasks in paused/stopped state: [OK] -------------------------------------------------------------------------------- Check for pending tasks which are safe to delete: [OK] -------------------------------------------------------------------------------- Check for tasks in planning state: [OK] -------------------------------------------------------------------------------- Check to verify if any hotfix installed on system: / Checking for presence of hotfix(es). It may take some time to verify. [OK] -------------------------------------------------------------------------------- Check if TMOUT environment variable is set: [OK] -------------------------------------------------------------------------------- Check if any upstream repositories are enabled on system: | Checking for presence of upstream repositories [OK] -------------------------------------------------------------------------------- Check if yum exclude list is configured: [OK] -------------------------------------------------------------------------------- Check for roles that have filters with multiple resources attached: [OK] -------------------------------------------------------------------------------- Check for recommended disk speed of pulp, mongodb, pgsql dir.: [SKIPPED] -------------------------------------------------------------------------------- Check whether reports have correct associations: [OK] -------------------------------------------------------------------------------- Validate availability of repositories: / Validating availability of repositories for 6.7 [OK] -------------------------------------------------------------------------------- Scenario [Checks before upgrading to Satellite 6.7.z] failed. The following steps ended up in failing state: [check-epel-repository] Resolve the failed steps and rerun the command. In case the failures are false positives, use --whitelist="check-epel-repository" Expected results: It seems that this is happening due to we have enabled "ansible-runner" repos on satellite server. # cat /etc/yum.repos.d/ansible-runner.repo [ansible-runner] name=Ansible runner baseurl=https://releases.ansible.com/ansible-runner/rpm/epel-7-$basearch/ enabled=1 gpgcheck=True gpgkey=https://releases.ansible.com/keys/RPM-GPG-KEY-ansible-release.pub If this is an expected behavior then we should have added in the documentation that we should use --whitelist="check-epel-repository" parameter in SM Additional info: I, [2020-02-17 22:29:48+0530 #16427] INFO -- : --- Execution step 'Check if EPEL repository enabled on system' [check-epel-repository] started --- D, [2020-02-17 22:29:48+0530 #16427] DEBUG -- : Running command yum repolist enabled -d 6 -e 0| grep -E 'Repo-baseurl|Repo-id' with stdin nil D, [2020-02-17 22:30:30+0530 #16427] DEBUG -- : output of the command: Repo-id : ansible-runner/x86_64 Repo-baseurl : https://releases.ansible.com/ansible-runner/rpm/epel-7-x86_64/ Repo-id : rhel-7-server-ansible-2.8-rpms/x86_64 Repo-baseurl : https://cdn.redhat.com/content/dist/rhel/server/7/7Server/x86_64/ansible/2.8/os Repo-id : rhel-7-server-rpms/7Server/x86_64 Repo-baseurl : https://cdn.redhat.com/content/dist/rhel/server/7/7Server/x86_64/os Repo-id : rhel-7-server-satellite-maintenance-6-beta-rpms/x86_64 Repo-baseurl : https://cdn.redhat.com/content/beta/rhel/server/7/x86_64/sat-maintenance/6/os Repo-id : rhel-server-7-satellite-6-beta-rpms/x86_64 Repo-baseurl : https://cdn.redhat.com/content/beta/rhel/server/7/x86_64/satellite/6/os Repo-id : rhel-server-rhscl-7-rpms/7Server/x86_64 Repo-baseurl : https://cdn.redhat.com/content/dist/rhel/server/7/7Server/x86_64/rhscl/1/os I, [2020-02-17 22:30:30+0530 #16427] INFO -- : --- Execution step 'Check if EPEL repository enabled on system' finished --- satellite.log:[DEBUG 2020-02-15T02:36:59 main] /Package[ansible-runner]: Provider yum does not support features targetable; not managing attribute command satellite.log:[ INFO 2020-02-15T02:37:01 main] MONITOR_RESOURCE Yumrepo[ansible-runner] satellite.log:[ INFO 2020-02-15T02:37:01 main] MONITOR_RESOURCE Package[ansible-runner] satellite.log:[ WARN 2020-02-15T02:37:25 main] /Stage[main]/Foreman_proxy::Plugin::Ansible::Runner/Yumrepo[ansible-runner]/ensure: created satellite.log:[ INFO 2020-02-15T02:37:25 main] Yumrepo[ansible-runner](provider=inifile): changing mode of /etc/yum.repos.d/ansible-runner.repo from 600 to 644 satellite.log:[DEBUG 2020-02-15T02:37:25 main] /Stage[main]/Foreman_proxy::Plugin::Ansible::Runner/Yumrepo[ansible-runner]: The container Class[Foreman_proxy::Plugin::Ansible::Runner] will propagate my refresh event satellite.log:[DEBUG 2020-02-15T02:37:25 main] Executing: '/usr/bin/rpm -q ansible-runner --nosignature --nodigest --qf %{NAME} %|EPOCH?{%{EPOCH}}:{0}| %{VERSION} %{RELEASE} %{ARCH}\n' satellite.log:[DEBUG 2020-02-15T02:37:25 main] Executing: '/usr/bin/rpm -q ansible-runner --nosignature --nodigest --qf %{NAME} %|EPOCH?{%{EPOCH}}:{0}| %{VERSION} %{RELEASE} %{ARCH}\n --whatprovides' satellite.log:[DEBUG 2020-02-15T02:37:25 main] Package[ansible-runner](provider=yum): Ensuring => present satellite.log:[DEBUG 2020-02-15T02:37:25 main] Executing: '/usr/bin/yum -d 0 -e 0 -y install ansible-runner' satellite.log:[ WARN 2020-02-15T02:37:49 main] /Stage[main]/Foreman_proxy::Plugin::Ansible::Runner/Package[ansible-runner]/ensure: created satellite.log:[DEBUG 2020-02-15T02:37:49 main] /Package[ansible-runner]: The container Class[Foreman_proxy::Plugin::Ansible::Runner] will propagate my refresh event foreman_proxy::plugin::ansible: enabled: true listen_on: https ansible_dir: /usr/share/foreman-proxy working_dir: /tmp host_key_checking: false stdout_callback: yaml roles_path: - /etc/ansible/roles - /usr/share/ansible/roles
Hi Ganesh, This is expected, we have this check so that we only upgrade packages provided from official satellite repos during upgrade. We shouldn't whitelist this kind of checks, they are there to keep satellite in working condition. If we whitelist this check there is chance that we might end up updating packages to version greater than what was expected.
As this bug shows, the upgrade check is properly finding non-Red Hat repositories that are enabled are failing but it is confusing to the user as it said it found an EPEL respository, when in fact, it was ansible.com repositories. We should rename this check to something like 'Check for repositories not hosted at cdn.redhat.com' so it is super-clear to our users what is failing. As a user with the above error, I would not know how to resolve this as there was no EPEL repo enabled.
(In reply to Mike McCune from comment #5) > As this bug shows, the upgrade check is properly finding non-Red Hat > repositories that are enabled are failing but it is confusing to the user as > it said it found an EPEL respository, when in fact, it was ansible.com > repositories. > > We should rename this check to something like 'Check for repositories not > hosted at cdn.redhat.com' so it is super-clear to our users what is failing. This is fixed in version rubygem-foreman_maintain-0.6.4. Instead of 'cdn.redhat.com', changed it to 'non Red Hat repositories'. Now, check description will look like - "Check to verify no any non Red Hat repositories(Eg: EPEL) enabled"
Verified: Verified with: - Satellite 6.8.0 snap 9 - rubygem-foreman_maintain-0.6.8-1.el7sat.noarch Test steps: - Run "foreman-maintain health check --label check-non-redhat-repository" Observation: - check name is changed from "check-epel-repository" to "check-non-redhat-repository" - Check description changed to "Check whether system has any non Red Hat repositories (e.g.: EPEL) enabled". # foreman-maintain health check --label check-non-redhat-repository Running ForemanMaintain::Scenario::FilteredScenario ================================================================================ Check whether system has any non Red Hat repositories (e.g.: EPEL) enabled: | Checking repositories enabled on the system [FAIL] System is subscribed to non Red Hat repositories -------------------------------------------------------------------------------- Scenario [ForemanMaintain::Scenario::FilteredScenario] failed. The following steps ended up in failing state: [check-non-redhat-repository] Resolve the failed steps and rerun the command. In case the failures are false positives, use --whitelist="check-non-redhat-repository" # foreman-maintain health check --label check-non-redhat-repository Running ForemanMaintain::Scenario::FilteredScenario ================================================================================ Check whether system has any non Red Hat repositories (e.g.: EPEL) enabled: - Checking repositories enabled on the system [OK] --------------------------------------------------------------------------------
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory (Satellite 6.8 Satellite Maintenance Release), and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHBA-2020:4365