Bug 1928264

Summary: Receptor service fails to run on Satellite 6.9 Beta
Product: Red Hat Satellite Reporter: Mike McCune <mmccune>
Component: RH Cloud - Cloud ConnectorAssignee: Adam Ruzicka <aruzicka>
Status: CLOSED ERRATA QA Contact: Lukáš Hellebrandt <lhellebr>
Severity: high Docs Contact:
Priority: high    
Version: 6.9.0CC: aruzicka, jaudet, jhutar, zhunting
Target Milestone: 6.9.0Keywords: Triaged
Target Release: Unused   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: python3-receptor-satellite-1.3.1 Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2021-04-21 13:10:34 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Mike McCune 2021-02-12 19:24:31 UTC
1) Install Satellite 6.9 beta

2) Configure Cloud Connector

3) Check on service status, note stack trace:

Feb 12 14:21:34 sat-r220-07.lab.eng.rdu2.redhat.com receptor[17900]: ERROR 2021-02-12 14:21:34,279 6ebe5e0a-318a-49cc-98a4-6d82515597ca __main__ main: an error occured while running receptor
Feb 12 14:21:34 sat-r220-07.lab.eng.rdu2.redhat.com receptor[17900]: Traceback (most recent call last):
Feb 12 14:21:34 sat-r220-07.lab.eng.rdu2.redhat.com receptor[17900]: File "/usr/lib/python3.6/site-packages/receptor/entrypoints.py", line 43, in run_as_node
Feb 12 14:21:34 sat-r220-07.lab.eng.rdu2.redhat.com receptor[17900]: controller = Controller(config)
Feb 12 14:21:34 sat-r220-07.lab.eng.rdu2.redhat.com receptor[17900]: File "/usr/lib/python3.6/site-packages/receptor/controller.py", line 19, in __init__
Feb 12 14:21:34 sat-r220-07.lab.eng.rdu2.redhat.com receptor[17900]: self.receptor = Receptor(config)
Feb 12 14:21:34 sat-r220-07.lab.eng.rdu2.redhat.com receptor[17900]: File "/usr/lib/python3.6/site-packages/receptor/receptor.py", line 46, in __init__
Feb 12 14:21:34 sat-r220-07.lab.eng.rdu2.redhat.com receptor[17900]: self.work_manager = (work_manager_cls or WorkManager)(self)
Feb 12 14:21:34 sat-r220-07.lab.eng.rdu2.redhat.com receptor[17900]: File "/usr/lib/python3.6/site-packages/receptor/work.py", line 20, in __init__
Feb 12 14:21:34 sat-r220-07.lab.eng.rdu2.redhat.com receptor[17900]: work_info.info(dict(plugins=str(self.get_capabilities())))
Feb 12 14:21:34 sat-r220-07.lab.eng.rdu2.redhat.com receptor[17900]: File "/usr/lib/python3.6/site-packages/receptor/work.py", line 36, in get_capabilities
Feb 12 14:21:34 sat-r220-07.lab.eng.rdu2.redhat.com receptor[17900]: for x in pkg_resources.iter_entry_points('receptor.worker')
Feb 12 14:21:34 sat-r220-07.lab.eng.rdu2.redhat.com receptor[17900]: File "/usr/lib/python3.6/site-packages/receptor/work.py", line 36, in <dictcomp>
Feb 12 14:21:34 sat-r220-07.lab.eng.rdu2.redhat.com receptor[17900]: for x in pkg_resources.iter_entry_points('receptor.worker')
Feb 12 14:21:34 sat-r220-07.lab.eng.rdu2.redhat.com receptor[17900]: File "/usr/lib/python3.6/site-packages/pkg_resources/__init__.py", line 2324, in resolve
Feb 12 14:21:34 sat-r220-07.lab.eng.rdu2.redhat.com receptor[17900]: module = __import__(self.module_name, fromlist=['__name__'], level=0)
Feb 12 14:21:34 sat-r220-07.lab.eng.rdu2.redhat.com receptor[17900]: File "/usr/lib/python3.6/site-packages/receptor_satellite/worker.py", line 9, in <module>
Feb 12 14:21:34 sat-r220-07.lab.eng.rdu2.redhat.com receptor[17900]: from .run import Run
Feb 12 14:21:34 sat-r220-07.lab.eng.rdu2.redhat.com receptor[17900]: File "/usr/lib/python3.6/site-packages/receptor_satellite/run.py", line 3, in <module>
Feb 12 14:21:34 sat-r220-07.lab.eng.rdu2.redhat.com receptor[17900]: from . import playbook_verifier_adapter
Feb 12 14:21:34 sat-r220-07.lab.eng.rdu2.redhat.com receptor[17900]: File "/usr/lib/python3.6/site-packages/receptor_satellite/playbook_verifier_adapter.py", line 1, in <module>
Feb 12 14:21:34 sat-r220-07.lab.eng.rdu2.redhat.com receptor[17900]: from insights.client.apps.ansible.playbook_verifier import (  # noqa: F401
Feb 12 14:21:34 sat-r220-07.lab.eng.rdu2.redhat.com receptor[17900]: ModuleNotFoundError: No module named 'insights'
Feb 12 14:21:34 sat-r220-07.lab.eng.rdu2.redhat.com receptor[17900]: During handling of the above exception, another exception occurred:
Feb 12 14:21:34 sat-r220-07.lab.eng.rdu2.redhat.com receptor[17900]: Traceback (most recent call last):
Feb 12 14:21:34 sat-r220-07.lab.eng.rdu2.redhat.com receptor[17900]: File "/usr/lib/python3.6/site-packages/receptor/__main__.py", line 59, in main
Feb 12 14:21:34 sat-r220-07.lab.eng.rdu2.redhat.com receptor[17900]: config.go()
Feb 12 14:21:34 sat-r220-07.lab.eng.rdu2.redhat.com receptor[17900]: File "/usr/lib/python3.6/site-packages/receptor/config.py", line 547, in go
Feb 12 14:21:34 sat-r220-07.lab.eng.rdu2.redhat.com receptor[17900]: self._parsed_args.func(self)
Feb 12 14:21:34 sat-r220-07.lab.eng.rdu2.redhat.com receptor[17900]: File "/usr/lib/python3.6/site-packages/receptor/entrypoints.py", line 62, in run_as_node
Feb 12 14:21:34 sat-r220-07.lab.eng.rdu2.redhat.com receptor[17900]: cleanup_tmpdir(controller)
Feb 12 14:21:34 sat-r220-07.lab.eng.rdu2.redhat.com receptor[17900]: UnboundLocalError: local variable 'controller' referenced before assignment
Feb 12 14:21:34 sat-r220-07.lab.eng.rdu2.redhat.com systemd[1]: receptor: main process exited, code=exited, status=1/FAILURE
Feb 12 14:21:34 sat-r220-07.lab.eng.rdu2.redhat.com systemd[1]: Unit receptor entered failed state.
Feb 12 14:21:34 sat-r220-07.lab.eng.rdu2.redhat.com systemd[1]: receptor failed.

Comment 3 Adam Ruzicka 2021-02-18 12:09:23 UTC
Fix was merged in upstream and a release is out

Comment 4 Lukáš Hellebrandt 2021-02-22 15:37:02 UTC
Still reproducible on Sat 6.9 snap 13, workaround: `pip install insights-core`.

Comment 5 jaudet 2021-02-22 16:53:58 UTC
How does one execute {{pip install insights-core}}? Is that command executed system-wide as root? Or in a venv?

Comment 6 Lukáš Hellebrandt 2021-02-23 10:37:49 UTC
I did it system-wide as root and it worked for me.

Comment 7 jaudet 2021-02-24 19:12:03 UTC
I executed `pip install insights-core && systemctl start 'receptor@*'` as root on a Satellite 6.9 snap 11 system. The receptor units successfully started, according to `systemctl status 'receptor@*'` and `journalctl --unit 'receptor@*' --follow`. The "Configure Cloud Connector" Ansible playbook was also successfully executed. Further testing will be needed to discover whether the Satellite is truly in working order, but it's looking good right now.

Comment 8 jaudet 2021-02-24 20:56:33 UTC
End-to-end tests indicate that the workaround fails on snap 11. I'll continue work with a newer snap of Satellite 6.9.

Comment 9 Lukáš Hellebrandt 2021-03-09 15:12:37 UTC
Verified with Sat 6.9 snap 16. Receptor installation is successful. Service starts successfuly. After a few hours of running, I don't see any related issues. Running receptor locally also doesn't show any issues:

# receptor --debug --node-id controller -d /tmp/controller node --listen=receptor://0.0.0.0:8842
INFO 2021-03-09 10:10:44,775 controller entrypoints Running as Receptor node with ID: controller
INFO 2021-03-09 10:10:44,776 controller controller Serving on receptor://0.0.0.0:8842

Comment 12 errata-xmlrpc 2021-04-21 13:10:34 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (Moderate: Satellite 6.9 Release), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2021:1313