Bug 2091407

Summary: insights-client.service fails to start
Product: Red Hat Enterprise Linux 8 Reporter: Tony <tony>
Component: selinux-policyAssignee: Zdenek Pytela <zpytela>
Status: CLOSED DUPLICATE QA Contact: BaseOS QE Security Team <qe-baseos-security>
Severity: high Docs Contact:
Priority: medium    
Version: 8.6CC: bschulz, cmarinea, fjansen, lvrabec, mmalik, pakotvan, prgutier, ssekidde, stomsa
Target Milestone: rcKeywords: Triaged
Target Release: 8.7Flags: pm-rhel: mirror+
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: No Doc Update
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2022-05-30 15:28:41 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Tony 2022-05-29 15:10:03 UTC
Description of problem:


Version-Release number of selected component (if applicable):


How reproducible:


Steps to Reproduce:
1. Restart system or attempt to start insights-client from the CLI
2. Wait for it to connect
3. Watch it fail instead

Actual results:
May 29 10:55:14 skinny.msnomer.com systemd[1]: Started Insights Client.
May 29 10:55:30 skinny.msnomer.com insights-client[95670]: Fatal error
May 29 10:55:30 skinny.msnomer.com insights-client[95670]: Traceback (most recent call last):
May 29 10:55:30 skinny.msnomer.com insights-client[95670]:   File "/usr/lib/python3.6/site-packages/urllib3/connection.py", line 162, in _new_conn
May 29 10:55:30 skinny.msnomer.com insights-client[95670]:     (self._dns_host, self.port), self.timeout, **extra_kw)
May 29 10:55:30 skinny.msnomer.com insights-client[95670]:   File "/usr/lib/python3.6/site-packages/urllib3/util/connection.py", line 57, in create_connection
May 29 10:55:30 skinny.msnomer.com insights-client[95670]:     for res in socket.getaddrinfo(host, port, family, socket.SOCK_STREAM):
May 29 10:55:30 skinny.msnomer.com insights-client[95670]:   File "/usr/lib64/python3.6/socket.py", line 745, in getaddrinfo
May 29 10:55:30 skinny.msnomer.com insights-client[95670]:     for res in _socket.getaddrinfo(host, port, family, type, proto, flags):
May 29 10:55:30 skinny.msnomer.com insights-client[95670]: socket.gaierror: [Errno -2] Name or service not known
May 29 10:55:30 skinny.msnomer.com insights-client[95670]: During handling of the above exception, another exception occurred:
May 29 10:55:30 skinny.msnomer.com insights-client[95670]: Traceback (most recent call last):
May 29 10:55:30 skinny.msnomer.com insights-client[95670]:   File "/usr/lib/python3.6/site-packages/urllib3/connectionpool.py", line 600, in urlopen
May 29 10:55:30 skinny.msnomer.com insights-client[95670]:     chunked=chunked)
May 29 10:55:30 skinny.msnomer.com insights-client[95670]:   File "/usr/lib/python3.6/site-packages/urllib3/connectionpool.py", line 343, in _make_request
May 29 10:55:30 skinny.msnomer.com insights-client[95670]:     self._validate_conn(conn)
May 29 10:55:30 skinny.msnomer.com insights-client[95670]:   File "/usr/lib/python3.6/site-packages/urllib3/connectionpool.py", line 839, in _validate_conn
May 29 10:55:30 skinny.msnomer.com insights-client[95670]:     conn.connect()
May 29 10:55:30 skinny.msnomer.com insights-client[95670]:   File "/usr/lib/python3.6/site-packages/urllib3/connection.py", line 315, in connect
May 29 10:55:30 skinny.msnomer.com insights-client[95670]:     conn = self._new_conn()
May 29 10:55:30 skinny.msnomer.com insights-client[95670]:   File "/usr/lib/python3.6/site-packages/urllib3/connection.py", line 171, in _new_conn
May 29 10:55:30 skinny.msnomer.com insights-client[95670]:     self, "Failed to establish a new connection: %s" % e)
May 29 10:55:30 skinny.msnomer.com insights-client[95670]: urllib3.exceptions.NewConnectionError: <urllib3.connection.VerifiedHTTPSConnection object at 0x7f94aeca07b8>: Failed to establish a new connection: [Errno -2] Name or service not known
May 29 10:55:30 skinny.msnomer.com insights-client[95670]: During handling of the above exception, another exception occurred:
May 29 10:55:30 skinny.msnomer.com insights-client[95670]: Traceback (most recent call last):
May 29 10:55:30 skinny.msnomer.com insights-client[95670]:   File "/usr/lib/python3.6/site-packages/requests/adapters.py", line 449, in send
May 29 10:55:30 skinny.msnomer.com insights-client[95670]:     timeout=timeout
May 29 10:55:30 skinny.msnomer.com insights-client[95670]:   File "/usr/lib/python3.6/site-packages/urllib3/connectionpool.py", line 638, in urlopen
May 29 10:55:30 skinny.msnomer.com insights-client[95670]:     _stacktrace=sys.exc_info()[2])
May 29 10:55:30 skinny.msnomer.com insights-client[95670]:   File "/usr/lib/python3.6/site-packages/urllib3/util/retry.py", line 399, in increment
May 29 10:55:30 skinny.msnomer.com insights-client[95670]:     raise MaxRetryError(_pool, url, error or ResponseError(cause))
May 29 10:55:30 skinny.msnomer.com insights-client[95670]: urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='cert-api.access.redhat.com', port=443): Max retries exceeded with url: /r/insights/platform/module-update-router/v1/channel?module=insights-core (Caused by NewConnectionError('<urllib3.connection.VerifiedHTTPSConnection object at 0x7f94aeca07b8>: Failed to establish a new connection: [Errno -2] Name or service not known',))
May 29 10:55:30 skinny.msnomer.com insights-client[95670]: During handling of the above exception, another exception occurred:
May 29 10:55:30 skinny.msnomer.com insights-client[95670]: Traceback (most recent call last):
May 29 10:55:30 skinny.msnomer.com insights-client[95670]:   File "/var/lib/insights/last_stable.egg/insights/client/phase/v1.py", line 32, in _f
May 29 10:55:30 skinny.msnomer.com insights-client[95670]:     func(client, config)
May 29 10:55:30 skinny.msnomer.com insights-client[95670]:   File "/var/lib/insights/last_stable.egg/insights/client/phase/v1.py", line 129, in update
May 29 10:55:30 skinny.msnomer.com insights-client[95670]:     client.update()
May 29 10:55:30 skinny.msnomer.com insights-client[95670]:   File "/var/lib/insights/last_stable.egg/insights/client/__init__.py", line 258, in update
May 29 10:55:30 skinny.msnomer.com insights-client[95670]:     egg_paths = self.fetch()
May 29 10:55:30 skinny.msnomer.com insights-client[95670]:   File "/var/lib/insights/last_stable.egg/insights/client/__init__.py", line 130, in fetch
May 29 10:55:30 skinny.msnomer.com insights-client[95670]:     egg_release = self.get_egg_url()
May 29 10:55:30 skinny.msnomer.com insights-client[95670]:   File "/var/lib/insights/last_stable.egg/insights/client/__init__.py", line 72, in _init_connection
May 29 10:55:30 skinny.msnomer.com insights-client[95670]:     return func(self, *args, **kwargs)
May 29 10:55:30 skinny.msnomer.com insights-client[95670]:   File "/var/lib/insights/last_stable.egg/insights/client/__init__.py", line 108, in get_egg_url
May 29 10:55:30 skinny.msnomer.com insights-client[95670]:     response = self.connection.get(url)
May 29 10:55:30 skinny.msnomer.com insights-client[95670]:   File "/var/lib/insights/last_stable.egg/insights/client/connection.py", line 198, in get
May 29 10:55:30 skinny.msnomer.com insights-client[95670]:     return self._http_request(url, 'GET', **kwargs)
May 29 10:55:30 skinny.msnomer.com insights-client[95670]:   File "/var/lib/insights/last_stable.egg/insights/client/connection.py", line 191, in _http_request
May 29 10:55:30 skinny.msnomer.com insights-client[95670]:     res = self.session.request(url=url, method=method, timeout=self.config.http_timeout, **kwargs)
May 29 10:55:30 skinny.msnomer.com insights-client[95670]:   File "/usr/lib/python3.6/site-packages/requests/sessions.py", line 533, in request
May 29 10:55:30 skinny.msnomer.com insights-client[95670]:     resp = self.send(prep, **send_kwargs)
May 29 10:55:30 skinny.msnomer.com insights-client[95670]:   File "/usr/lib/python3.6/site-packages/requests/sessions.py", line 646, in send
May 29 10:55:30 skinny.msnomer.com insights-client[95670]:     r = adapter.send(request, **kwargs)
May 29 10:55:30 skinny.msnomer.com insights-client[95670]:   File "/usr/lib/python3.6/site-packages/requests/adapters.py", line 516, in send
May 29 10:55:30 skinny.msnomer.com insights-client[95670]:     raise ConnectionError(e, request=request)
May 29 10:55:30 skinny.msnomer.com insights-client[95670]: requests.exceptions.ConnectionError: HTTPSConnectionPool(host='cert-api.access.redhat.com', port=443): Max retries exceeded with url: /r/insights/platform/module-update-router/v1/channel?module=insights-core (Caused by NewConnectionError('<urllib3.connection.VerifiedHTTPSConnection object at 0x7f94aeca07b8>: Failed to establish a new connection: [Errno -2] Name or service not known',))
May 29 10:55:34 skinny.msnomer.com insights-client[95690]: Fatal error
May 29 10:55:34 skinny.msnomer.com insights-client[95690]: Traceback (most recent call last):
May 29 10:55:34 skinny.msnomer.com insights-client[95690]:   File "/usr/lib/python3.6/site-packages/urllib3/connection.py", line 162, in _new_conn
May 29 10:55:34 skinny.msnomer.com insights-client[95690]:     (self._dns_host, self.port), self.timeout, **extra_kw)
May 29 10:55:34 skinny.msnomer.com insights-client[95690]:   File "/usr/lib/python3.6/site-packages/urllib3/util/connection.py", line 57, in create_connection
May 29 10:55:34 skinny.msnomer.com insights-client[95690]:     for res in socket.getaddrinfo(host, port, family, socket.SOCK_STREAM):
May 29 10:55:34 skinny.msnomer.com insights-client[95690]:   File "/usr/lib64/python3.6/socket.py", line 745, in getaddrinfo
May 29 10:55:34 skinny.msnomer.com insights-client[95690]:     for res in _socket.getaddrinfo(host, port, family, type, proto, flags):
May 29 10:55:34 skinny.msnomer.com insights-client[95690]: socket.gaierror: [Errno -2] Name or service not known
May 29 10:55:34 skinny.msnomer.com insights-client[95690]: During handling of the above exception, another exception occurred:
May 29 10:55:34 skinny.msnomer.com insights-client[95690]: Traceback (most recent call last):
May 29 10:55:34 skinny.msnomer.com insights-client[95690]:   File "/usr/lib/python3.6/site-packages/urllib3/connectionpool.py", line 600, in urlopen
May 29 10:55:34 skinny.msnomer.com insights-client[95690]:     chunked=chunked)
May 29 10:55:34 skinny.msnomer.com insights-client[95690]:   File "/usr/lib/python3.6/site-packages/urllib3/connectionpool.py", line 343, in _make_request
May 29 10:55:34 skinny.msnomer.com insights-client[95690]:     self._validate_conn(conn)
May 29 10:55:34 skinny.msnomer.com insights-client[95690]:   File "/usr/lib/python3.6/site-packages/urllib3/connectionpool.py", line 839, in _validate_conn
May 29 10:55:34 skinny.msnomer.com insights-client[95690]:     conn.connect()
May 29 10:55:34 skinny.msnomer.com insights-client[95690]:   File "/usr/lib/python3.6/site-packages/urllib3/connection.py", line 315, in connect
May 29 10:55:34 skinny.msnomer.com insights-client[95690]:     conn = self._new_conn()
May 29 10:55:34 skinny.msnomer.com insights-client[95690]:   File "/usr/lib/python3.6/site-packages/urllib3/connection.py", line 171, in _new_conn
May 29 10:55:34 skinny.msnomer.com insights-client[95690]:     self, "Failed to establish a new connection: %s" % e)
May 29 10:55:34 skinny.msnomer.com insights-client[95690]: urllib3.exceptions.NewConnectionError: <urllib3.connection.VerifiedHTTPSConnection object at 0x7fb19be1a6a0>: Failed to establish a new connection: [Errno -2] Name or service not known
May 29 10:55:34 skinny.msnomer.com insights-client[95690]: During handling of the above exception, another exception occurred:
May 29 10:55:34 skinny.msnomer.com insights-client[95690]: Traceback (most recent call last):
May 29 10:55:34 skinny.msnomer.com insights-client[95690]:   File "/usr/lib/python3.6/site-packages/requests/adapters.py", line 449, in send
May 29 10:55:34 skinny.msnomer.com insights-client[95690]:     timeout=timeout
May 29 10:55:34 skinny.msnomer.com insights-client[95690]:   File "/usr/lib/python3.6/site-packages/urllib3/connectionpool.py", line 638, in urlopen
May 29 10:55:34 skinny.msnomer.com insights-client[95690]:     _stacktrace=sys.exc_info()[2])
May 29 10:55:34 skinny.msnomer.com insights-client[95690]:   File "/usr/lib/python3.6/site-packages/urllib3/util/retry.py", line 399, in increment
May 29 10:55:34 skinny.msnomer.com insights-client[95690]:     raise MaxRetryError(_pool, url, error or ResponseError(cause))
May 29 10:55:34 skinny.msnomer.com insights-client[95690]: urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='cert-api.access.redhat.com', port=443): Max retries exceeded with url: /r/insights/platform/module-update-router/v1/channel?module=insights-core (Caused by NewConnectionError('<urllib3.connection.VerifiedHTTPSConnection object at 0x7fb19be1a6a0>: Failed to establish a new connection: [Errno -2] Name or service not known',))
May 29 10:55:34 skinny.msnomer.com insights-client[95690]: During handling of the above exception, another exception occurred:
May 29 10:55:34 skinny.msnomer.com insights-client[95690]: Traceback (most recent call last):
May 29 10:55:34 skinny.msnomer.com insights-client[95690]:   File "/etc/insights-client/rpm.egg/insights/client/phase/v1.py", line 32, in _f
May 29 10:55:34 skinny.msnomer.com insights-client[95690]:     func(client, config)
May 29 10:55:34 skinny.msnomer.com insights-client[95690]:   File "/etc/insights-client/rpm.egg/insights/client/phase/v1.py", line 129, in update
May 29 10:55:34 skinny.msnomer.com insights-client[95690]:     client.update()
May 29 10:55:34 skinny.msnomer.com insights-client[95690]:   File "/etc/insights-client/rpm.egg/insights/client/__init__.py", line 258, in update
May 29 10:55:34 skinny.msnomer.com insights-client[95690]:     egg_paths = self.fetch()
May 29 10:55:34 skinny.msnomer.com insights-client[95690]:   File "/etc/insights-client/rpm.egg/insights/client/__init__.py", line 130, in fetch
May 29 10:55:34 skinny.msnomer.com insights-client[95690]:     egg_release = self.get_egg_url()
May 29 10:55:34 skinny.msnomer.com insights-client[95690]:   File "/etc/insights-client/rpm.egg/insights/client/__init__.py", line 72, in _init_connection
May 29 10:55:34 skinny.msnomer.com insights-client[95690]:     return func(self, *args, **kwargs)
May 29 10:55:34 skinny.msnomer.com insights-client[95690]:   File "/etc/insights-client/rpm.egg/insights/client/__init__.py", line 108, in get_egg_url
May 29 10:55:34 skinny.msnomer.com insights-client[95690]:     response = self.connection.get(url)
May 29 10:55:34 skinny.msnomer.com insights-client[95690]:   File "/etc/insights-client/rpm.egg/insights/client/connection.py", line 195, in get
May 29 10:55:34 skinny.msnomer.com insights-client[95690]:     return self._http_request(url, 'GET', **kwargs)
May 29 10:55:34 skinny.msnomer.com insights-client[95690]:   File "/etc/insights-client/rpm.egg/insights/client/connection.py", line 189, in _http_request
May 29 10:55:34 skinny.msnomer.com insights-client[95690]:     res = self.session.request(url=url, method=method, timeout=self.config.http_timeout, **kwargs)
May 29 10:55:34 skinny.msnomer.com insights-client[95690]:   File "/usr/lib/python3.6/site-packages/requests/sessions.py", line 533, in request
May 29 10:55:34 skinny.msnomer.com insights-client[95690]:     resp = self.send(prep, **send_kwargs)
May 29 10:55:34 skinny.msnomer.com insights-client[95690]:   File "/usr/lib/python3.6/site-packages/requests/sessions.py", line 646, in send
May 29 10:55:34 skinny.msnomer.com insights-client[95690]:     r = adapter.send(request, **kwargs)
May 29 10:55:34 skinny.msnomer.com insights-client[95690]:   File "/usr/lib/python3.6/site-packages/requests/adapters.py", line 516, in send
May 29 10:55:34 skinny.msnomer.com insights-client[95690]:     raise ConnectionError(e, request=request)
May 29 10:55:34 skinny.msnomer.com insights-client[95690]: requests.exceptions.ConnectionError: HTTPSConnectionPool(host='cert-api.access.redhat.com', port=443): Max retries exceeded with url: /r/insights/platform/module-update-router/v1/channel?module=insights-core (Caused by NewConnectionError('<urllib3.connection.VerifiedHTTPSConnection object at 0x7fb19be1a6a0>: Failed to establish a new connection: [Errno -2] Name or service not known',))
May 29 10:55:35 skinny.msnomer.com systemd[1]: insights-client.service: Main process exited, code=exited, status=1/FAILURE
May 29 10:55:35 skinny.msnomer.com systemd[1]: insights-client.service: Failed with result 'exit-code'.


Expected results:
The client starts

Additional info:
Same problem on all of my RHEL 8.6 systems.
Possible regression of https://bugzilla.redhat.com/show_bug.cgi?format=multiple&id=1909866

Comment 3 Bennet Schulz 2022-05-30 11:58:13 UTC
I am facing the same issue described above on RHEL 8.5.

When I change SELinux from enforcing to permissive and reboot, then it's working again.

Comment 4 Tony 2022-05-30 13:22:09 UTC
@prgutier What additional info. do you need?

Comment 5 Priscila Gutierres 2022-05-30 13:39:25 UTC
Can you please give us more info if you are having SE Linux denial messages?

Comment 6 Tony 2022-05-30 14:06:44 UTC
Looks like yes. I attempted to start the client and saw 12 instances of this:
SELinux is preventing /usr/libexec/platform-python3.6 from search access on the directory .local.

*****  Plugin catchall (100. confidence) suggests   **************************

If you believe that platform-python3.6 should be allowed search access on the .local directory by default.
Then you should report this as a bug.
You can generate a local policy module to allow this access.
Do
allow this access for now by executing:
# ausearch -c 'insights-client' --raw | audit2allow -M my-insightsclient
# semodule -X 300 -i my-insightsclient.pp

Additional Information:
Source Context                system_u:system_r:insights_client_t:s0
Target Context                unconfined_u:object_r:gconf_home_t:s0
Target Objects                .local [ dir ]
Source                        insights-client
Source Path                   /usr/libexec/platform-python3.6
Port                          <Unknown>
Host                          skinny.msnomer.com
Source RPM Packages           platform-python-3.6.8-45.el8.x86_64
Target RPM Packages           
SELinux Policy RPM            selinux-policy-targeted-3.14.3-95.el8.noarch
Local Policy RPM              selinux-policy-targeted-3.14.3-95.el8.noarch
Selinux Enabled               True
Policy Type                   targeted
Enforcing Mode                Enforcing
Host Name                     skinny.msnomer.com
Platform                      Linux skinny.msnomer.com 4.18.0-372.9.1.el8.x86_64
                              #1 SMP Fri Apr 15 22:12:19 EDT 2022 x86_64 x86_64
Alert Count                   12
First Seen                    2022-05-30 10:02:36 EDT
Last Seen                     2022-05-30 10:02:42 EDT
Local ID                      45e0dbfb-bb0a-4591-94c3-c4930c351f81

Raw Audit Messages
type=AVC msg=audit(1653919362.989:2039): avc:  denied  { search } for  pid=169455 comm="platform-python" name=".local" dev="dm-0" ino=34860217 scontext=system_u:system_r:insights_client_t:s0 tcontext=unconfined_u:object_r:gconf_home_t:s0 tclass=dir permissive=0


type=SYSCALL msg=audit(1653919362.989:2039): arch=x86_64 syscall=stat success=no exit=EACCES a0=7fc8433b1950 a1=7ffd2f6485d0 a2=7ffd2f6485d0 a3=1 items=0 ppid=169372 pid=169455 auid=4294967295 uid=0 gid=0 euid=0 suid=0 fsuid=0 egid=0 sgid=0 fsgid=0 tty=(none) ses=4294967295 comm=platform-python exe=/usr/libexec/platform-python3.6 subj=system_u:system_r:insights_client_t:s0 key=(null)

Hash: insights-client,insights_client_t,gconf_home_t,dir,search

Comment 7 Zdenek Pytela 2022-05-30 15:13:01 UTC
Is this the only denial you see?

The denial is the same as in https://bugzilla.redhat.com/show_bug.cgi?id=2087069 which is now in the ON_QA state.

Comment 8 Tony 2022-05-30 15:26:23 UTC
This is the only denial, yes.

Comment 9 Zdenek Pytela 2022-05-30 15:28:41 UTC
Thanks for confirming, closing as a dup then.

*** This bug has been marked as a duplicate of bug 2087069 ***

Comment 10 Red Hat Bugzilla 2023-09-15 01:55:17 UTC
The needinfo request[s] on this closed bug have been removed as they have been unresolved for 365 days