Bug 1889320

Summary:	Installer fails with NoMethodError: undefined method `permissions' for nil:NilClass
Product:	Red Hat Satellite	Reporter:	Lukas Pramuk <lpramuk>
Component:	Infrastructure	Assignee:	satellite6-bugs <satellite6-bugs>
Status:	CLOSED ERRATA	QA Contact:	Lukas Pramuk <lpramuk>
Severity:	high	Docs Contact:
Priority:	high
Version:	6.8.0	CC:	ahumbe, egolov, inecas, tbrisker, zhunting
Target Milestone:	6.9.0	Keywords:	PrioBumpGSS, Triaged
Target Release:	Unused
Hardware:	Unspecified
OS:	Unspecified
Whiteboard:
Fixed In Version:		Doc Type:	If docs needed, set a value
Doc Text:		Story Points:	---
Clone Of:
Clones:	1890923 (view as bug list)		Environment:
Last Closed:	2021-04-21 13:18:20 UTC	Type:	---
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:

Description Lukas Pramuk 2020-10-19 11:32:51 UTC

This bug was initially created as a copy of Bug #1882276

I am copying this bug because: 
There is another issue with the same symptom but different cause 


Description of problem:
Satellite-installer fails at execution of '/usr/sbin/foreman-rake -- config -k 'remote_execution_cockpit_url' -v '/webcon/=%{host}''

Version-Release number of selected component (if applicable):
Satellite 6.8.0 Snap18 (upto Snap17 there was Bug #1882276) 

How reproducible:
intermittent

Steps to Reproduce:
1. Try installing satellite with cockpit enabled

Actual results:
Fails at Execution of '/usr/sbin/foreman-rake -- config -k 'remote_execution_cockpit_url' -v '/webcon/=%{host}'' returned 1: rake aborted!

Expected results:
Satellite 6.8.0 installed successfully.

Additional info:
deep down the traceback is a bit different now:

 /opt/theforeman/tfm/root/usr/share/gems/gems/foreman_discovery-16.1.2/lib/foreman_discovery/engine.rb:152:in `block (2 levels) in <class:Engine>'

vs.

 /opt/theforeman/tfm/root/usr/share/gems/gems/foreman_openscap-4.0.3/lib/foreman_openscap/engine.rb:127:in `block (2 levels) in <class:Engine>'

Comment 2 Lukas Pramuk 2020-10-19 11:44:39 UTC

New traceback: 

[ERROR 2020-10-15T11:38:35 verbose]  Execution of '/usr/sbin/foreman-rake -- config -k 'remote_execution_cockpit_url' -v '/webcon/=%{host}'' returned 1: rake aborted!
[ERROR 2020-10-15T11:38:35 verbose] NoMethodError: undefined method `permissions' for nil:NilClass
[ERROR 2020-10-15T11:38:35 verbose] /usr/share/foreman/app/models/role.rb:161:in `block (2 levels) in add_permissions'
[ERROR 2020-10-15T11:38:35 verbose] /usr/share/foreman/app/models/role.rb:160:in `each'
[ERROR 2020-10-15T11:38:35 verbose] /usr/share/foreman/app/models/role.rb:160:in `block in add_permissions'
[ERROR 2020-10-15T11:38:35 verbose] /usr/share/foreman/app/models/role.rb:157:in `each'
[ERROR 2020-10-15T11:38:35 verbose] /usr/share/foreman/app/models/role.rb:157:in `add_permissions'
[ERROR 2020-10-15T11:38:35 verbose] /usr/share/foreman/app/models/role.rb:207:in `add_permissions!'
[ERROR 2020-10-15T11:38:35 verbose] /usr/share/foreman/app/services/foreman/plugin/role_lock.rb:54:in `block in update_plugin_role_permissions'
[ERROR 2020-10-15T11:38:35 verbose] /usr/share/foreman/app/models/role.rb:86:in `ignore_locking'
[ERROR 2020-10-15T11:38:35 verbose] /usr/share/foreman/app/services/foreman/plugin/role_lock.rb:52:in `update_plugin_role_permissions'
[ERROR 2020-10-15T11:38:35 verbose] /usr/share/foreman/app/services/foreman/plugin/role_lock.rb:23:in `process_role'
[ERROR 2020-10-15T11:38:35 verbose] /usr/share/foreman/app/services/foreman/plugin/role_lock.rb:12:in `block in register_role'
[ERROR 2020-10-15T11:38:35 verbose] /usr/share/foreman/app/models/concerns/foreman/thread_session.rb:108:in `as'
[ERROR 2020-10-15T11:38:35 verbose] /usr/share/foreman/app/models/concerns/foreman/thread_session.rb:114:in `as_anonymous_admin'
[ERROR 2020-10-15T11:38:35 verbose] /usr/share/foreman/app/services/foreman/plugin/role_lock.rb:11:in `register_role'
[ERROR 2020-10-15T11:38:35 verbose] /usr/share/foreman/app/registries/foreman/plugin.rb:302:in `block (2 levels) in role'
[ERROR 2020-10-15T11:38:35 verbose] /opt/theforeman/tfm/root/usr/share/gems/gems/audited-4.9.0/lib/audited/auditor.rb:376:in `without_auditing'
[ERROR 2020-10-15T11:38:35 verbose] /usr/share/foreman/app/registries/foreman/plugin.rb:301:in `block in role'
[ERROR 2020-10-15T11:38:35 verbose] /opt/theforeman/tfm/root/usr/share/gems/gems/audited-4.9.0/lib/audited/auditor.rb:376:in `without_auditing'
[ERROR 2020-10-15T11:38:35 verbose] /usr/share/foreman/app/registries/foreman/plugin.rb:300:in `role'
[ERROR 2020-10-15T11:38:35 verbose] /opt/theforeman/tfm/root/usr/share/gems/gems/foreman_discovery-16.1.2/lib/foreman_discovery/engine.rb:152:in `block (2 levels) in <class:Engine>'
[ERROR 2020-10-15T11:38:35 verbose] /usr/share/foreman/app/registries/foreman/plugin.rb:78:in `instance_eval'
[ERROR 2020-10-15T11:38:35 verbose] /usr/share/foreman/app/registries/foreman/plugin.rb:78:in `register'
[ERROR 2020-10-15T11:38:35 verbose] /opt/theforeman/tfm/root/usr/share/gems/gems/foreman_discovery-16.1.2/lib/foreman_discovery/engine.rb:45:in `block in <class:Engine>'


Former (fixed) traceback:

[ERROR 2020-09-23T13:10:59 verbose]  Execution of '/usr/sbin/foreman-rake -- config -k 'remote_execution_cockpit_url' -v '/webcon/=%{host}'' returned 1: rake aborted!
[ERROR 2020-09-23T13:10:59 verbose] NoMethodError: undefined method `permissions' for nil:NilClass
[ERROR 2020-09-23T13:10:59 verbose] /usr/share/foreman/app/models/role.rb:161:in `block (2 levels) in add_permissions'
[ERROR 2020-09-23T13:10:59 verbose] /usr/share/foreman/app/models/role.rb:160:in `each'
[ERROR 2020-09-23T13:10:59 verbose] /usr/share/foreman/app/models/role.rb:160:in `block in add_permissions'
[ERROR 2020-09-23T13:10:59 verbose] /usr/share/foreman/app/models/role.rb:157:in `each'
[ERROR 2020-09-23T13:10:59 verbose] /usr/share/foreman/app/models/role.rb:157:in `add_permissions'
[ERROR 2020-09-23T13:10:59 verbose] /usr/share/foreman/app/models/role.rb:207:in `add_permissions!'
[ERROR 2020-09-23T13:10:59 verbose] /usr/share/foreman/app/services/foreman/plugin/role_lock.rb:54:in `block in update_plugin_role_permissions'
[ERROR 2020-09-23T13:10:59 verbose] /usr/share/foreman/app/models/role.rb:86:in `ignore_locking'
[ERROR 2020-09-23T13:10:59 verbose] /usr/share/foreman/app/services/foreman/plugin/role_lock.rb:52:in `update_plugin_role_permissions'
[ERROR 2020-09-23T13:10:59 verbose] /usr/share/foreman/app/services/foreman/plugin/role_lock.rb:23:in `process_role'
[ERROR 2020-09-23T13:10:59 verbose] /usr/share/foreman/app/services/foreman/plugin/role_lock.rb:12:in `block in register_role'
[ERROR 2020-09-23T13:10:59 verbose] /usr/share/foreman/app/models/concerns/foreman/thread_session.rb:108:in `as'
[ERROR 2020-09-23T13:10:59 verbose] /usr/share/foreman/app/models/concerns/foreman/thread_session.rb:114:in `as_anonymous_admin'
[ERROR 2020-09-23T13:10:59 verbose] /usr/share/foreman/app/services/foreman/plugin/role_lock.rb:11:in `register_role'
[ERROR 2020-09-23T13:10:59 verbose] /usr/share/foreman/app/registries/foreman/plugin.rb:302:in `block (2 levels) in role'
[ERROR 2020-09-23T13:10:59 verbose] /opt/theforeman/tfm/root/usr/share/gems/gems/audited-4.9.0/lib/audited/auditor.rb:376:in `without_auditing'
[ERROR 2020-09-23T13:10:59 verbose] /usr/share/foreman/app/registries/foreman/plugin.rb:301:in `block in role'
[ERROR 2020-09-23T13:10:59 verbose] /opt/theforeman/tfm/root/usr/share/gems/gems/audited-4.9.0/lib/audited/auditor.rb:376:in `without_auditing'
[ERROR 2020-09-23T13:10:59 verbose] /usr/share/foreman/app/registries/foreman/plugin.rb:300:in `role'
[ERROR 2020-09-23T13:10:59 verbose] /opt/theforeman/tfm/root/usr/share/gems/gems/foreman_openscap-4.0.3/lib/foreman_openscap/engine.rb:127:in `block (2 levels) in <class:Engine>'
[ERROR 2020-09-23T13:10:59 verbose] /usr/share/foreman/app/registries/foreman/plugin.rb:78:in `instance_eval'
[ERROR 2020-09-23T13:10:59 verbose] /usr/share/foreman/app/registries/foreman/plugin.rb:78:in `register'
[ERROR 2020-09-23T13:10:59 verbose] /opt/theforeman/tfm/root/usr/share/gems/gems/foreman_openscap-4.0.3/lib/foreman_openscap/engine.rb:49:in `block in <class:Engine>'

Comment 3 Tomer Brisker 2020-10-20 12:57:59 UTC

This may possibly be fixed by https://projects.theforeman.org/issues/30821. 
Additionally, it isn't specific to cockpit, in fact the stack trace points at discovery in this case, updating the title accordingly.
Most likely the rake task is running before the migrations finished and the db fully seeded, which causes the plugins to expect roles to already be there.

Comment 4 Evgeni Golov 2020-10-30 10:27:46 UTC

(In reply to Tomer Brisker from comment #3)
> This may possibly be fixed by https://projects.theforeman.org/issues/30821. 
> Additionally, it isn't specific to cockpit, in fact the stack trace points
> at discovery in this case, updating the title accordingly.
> Most likely the rake task is running before the migrations finished and the
> db fully seeded, which causes the plugins to expect roles to already be
> there.

I've seen this on 2.2/6.9 based installs too :(

Comment 5 Lukas Pramuk 2021-01-28 22:35:02 UTC

VERIFIED.

@Satellite 6.9.0 Snap11
foreman-2.3.1.4-1.el7sat.noarch

Tried 5 fresh Satellite installations with cockpit feature enabled
All 5 installation finished successfully

2021-01-28 17:22:19 [DEBUG ] [configure] Executing with uid=foreman gid=foreman: '/usr/sbin/foreman-rake -- config -k 'remote_execution_cockpit_url' -v '/webcon/=%{host}''
2021-01-28 17:23:18 [INFO  ] [configure] /Stage[main]/Foreman::Plugin::Remote_execution::Cockpit/Foreman_config_entry[remote_execution_cockpit_url]/value: value changed '' to '/webcon/=%{host}'

Comment 8 errata-xmlrpc 2021-04-21 13:18:20 UTC

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (Moderate: Satellite 6.9 Release), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2021:1313