Bug 1847309

Description of problem:
While testing the Capsule upgrade from 6.7 to 6.8 via foreman-maintain (6.8 feature), I bumped into an installer issue which seems to be solved here: https://github.com/theforeman/foreman-installer/pull/472

However, on Capsule 6.7 I do still face it, although the foreman-installer-2.1.0-0.3.1.rc2 is present and I can see the fix as well:

# satellite-maintain upgrade list-versions
6.8
# satellite-maintain upgrade run --assumeyes --target-version=6.8 --whitelist="repositories-validate,repositories-setup,disk-performance"
Checking for new version of satellite-maintain...
Nothing to update, can't find new version of satellite-maintain.
Running Checks before upgrading to Capsule 6.8
================================================================================
Skipping pre_upgrade_checks phase as it was already run before.
To enforce to run the phase, use `upgrade run --phase pre_upgrade_checks`



Running Procedures before migrating to Capsule 6.8
================================================================================
Skipping pre_migrations phase as it was already run before.
To enforce to run the phase, use `upgrade run --phase pre_migrations`



Running preparation steps required to run the next scenarios
================================================================================
Check if tooling for package locking is installed:                    [OK]
--------------------------------------------------------------------------------


Running Migration scripts to Capsule 6.8
================================================================================
Setup repositories:                                                   [ALREADY RUN]
The step was skipped as it was already run and it is marked as run_once. Use --force to enforce the execution.
--------------------------------------------------------------------------------
Unlock packages:                                                      [OK]
--------------------------------------------------------------------------------
Update package(s) :                                                   [OK]
--------------------------------------------------------------------------------
Procedures::Installer::Upgrade:                                       [FAIL]
Failed executing LANG=en_US.utf-8 foreman-installer, exit status 6:
 Executing: foreman-maintain packages is-locked --assumeyes
Info: Loading facts                               : 0 00:00:00
    id => "capsule",                              : 0 0.0/s 00:00:01
      Using configuration "/usr/share/foreman-inst: 0 0.0/s 00:00:02
Debug: template[/usr/share/foreman-installer/modul: 0 0.0/s 00:00:03
Debug: Automatic Parameter Lookup of 'apache::mod:: 0 0.0/s 00:00:04
Debug: template[/usr/share/foreman-installer/modul: 0 0.0/s 00:00:05
Debug: template[/usr/share/foreman-installer/modul: 0 0.0/s 00:00:06
Debug: /File[/etc/pki/katello/certs/katello-defaul: 0 0.0/s 00:00:07
Debug: /File[dav.load]/seltype: Found seltype defa: 0 0.0/s 00:00:08
Debug: /File[/etc/qpid-dispatch/qdrouterd.conf]/se: 0 0.0/s 00:00:09
Info: Kafo_configure::Puppet_version_semver[hercul: 0 0.0/s 00:00:10
Debug: Prefetching yum resources for package      : 0 0.0/s 00:00:10
Info: Class[Apache::Params]: Starting to evaluate : 0 0.0/s 00:00:16
Info: Certs::Keypair[foreman_proxy_client]: Evalua: 0 0.0/s 00:00:17
Info: /Stage[main]/Qpid::Service/Systemd::Dropin_f: 0 0.0/s 00:00:18
Debug: Class[Qpid::Service]: The container Class[Q: 0 0.0/s 00:00:20
Debug: Prefetching semanage resources for selinux_: 0 0.0/s 00:00:20
Debug: Prefetching semanage resources for selinux_: 0 0.0/s 00:00:20
Debug: /Stage[main]/Pulp::Database/Exec[migrate_pu: 0 0.0/s 00:00:22
Debug: /Stage[main]/Pulp::Database/Exec[migrate_pu: 0 0.0/s 00:00:24
Debug: /Stage[main]/Pulp::Database/Exec[migrate_pu: 0 0.0/s 00:00:25
Debug: /Stage[main]/Pulp::Database/Exec[migrate_pu: 0 0.0/s 00:00:27
Debug: /Stage[main]/Pulp::Database/Exec[migrate_pu: 0 0.0/s 00:00:29
Info: Class[Pulp::Apache]: Starting to evaluate th: 0 0.0/s 00:00:31
Debug: /File[/var/lib/tftpboot/grub2/i386-pc/raid5: 0 0.0/s 00:00:32
Debug: /File[/var/lib/tftpboot/grub2/i386-efi/part: 0 0.0/s 00:00:33
Debug: /File[/var/lib/tftpboot/grub2/x86_64-efi/fs: 0 0.0/s 00:00:34
Info: /Stage[main]/Foreman_proxy::Tftp/File[/var/l: 0 0.0/s 00:00:35
Info: /Stage[main]/Foreman_proxy::Tftp/File[/var/l: 0 0.0/s 00:00:36
Info: Concat::Fragment[/var/www/html/pub/katello-r: 0 0.0/s 00:00:37
Info: /Package[tfm-rubygem-smart_proxy_dynflow]: S: 0 0.0/s 00:00:39
Debug: Prefetching crontab resources for cron     : 0 0.0/s 00:00:39
 Could not find a suitable provider for foreman_smartproxy
Done                                              : 1 0.0/s 00:00:43
Done                                              : 1 0.0/s 00:00:43
  Something went wrong! Check the log for ERROR-level output
  The full log is at /var/log/foreman-installer/capsule.log
--------------------------------------------------------------------------------
Scenario [Migration scripts to Capsule 6.8] failed.

The following steps ended up in failing state:

  [installer-upgrade]

Resolve the failed steps and rerun
the command. In case the failures are false positives,
use --whitelist="installer-upgrade"


# rpm -qa | grep installer
satellite-installer-6.8.0.5-1.el7sat.noarch
foreman-installer-katello-2.1.0-0.3.1.rc2.el7sat.noarch
foreman-installer-2.1.0-0.3.1.rc2.el7sat.noarch
ansiblerole-satellite-receptor-installer-0.6.6-1.el7sat.noarch


# cat /usr/share/foreman-installer/hooks/pre_validations/31-puppet-agent-oauth.rb
unless app_value(:noop)
  if File.exist?('/opt/puppetlabs/puppet/bin/ruby') && execute_command("rpm -q puppet-agent-oauth", false, false)
    unless execute_command("/opt/puppetlabs/puppet/bin/ruby -e \"require 'oauth'\"", false, false)
      execute("yum -y reinstall puppet-agent-oauth", false, true)
    end
  end
end


Running 'yum reinstall puppet-agent-oauth' as a workaround works, but we will need some fix for Capsule as well.

Thank you.


Version-Release number of selected component (if applicable):
foreman-installer-2.1.0-0.3.1.rc2


How reproducible:
always


Steps to Reproduce:
1. Install a 6.7 capsule, register it to a Satellite
2. Add and enable Satellite 6.8 repos
3. Run # satellite-maintain upgrade run --assumeyes --target-version=6.8 --whitelist="repositories-validate,repositories-setup,disk-performance"


Actual results:
installer::upgrade fails


Expected results:
successful run