Bug 1447722

Summary: rhsmcertd-worker: Error reading networking information: [Errno 13] Permission denied
Product: Red Hat Enterprise Linux 7 Reporter: Kevin Howell <khowell>
Component: subscription-managerAssignee: Jiri Hnidek <jhnidek>
Status: CLOSED ERRATA QA Contact: John Sefler <jsefler>
Severity: high Docs Contact:
Priority: high    
Version: 7.4CC: jhnidek, redakkan, skallesh
Target Milestone: rcKeywords: Triaged
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: subscription-manager-1.19.13-1 Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2017-08-01 19:23:41 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 1519776    

Description Kevin Howell 2017-05-03 14:56:06 UTC 
Description of problem:
rhsmcertd-worker unable to read networking information


Steps to Reproduce:
1.Install build RHEL-7.4-20170421.1
2.remove the system's product cert.
#mv /etc/pki/product-default/*.pem /root/tmp
3.register the system
#subscription-manager register
4.restart the rhsmcertd
#service rhsmcertd restart
5.check rhsm.log in /var/log/rhsm/
[root@dhcp-129-210 rhsm]# cat rhsm.log | grep Error
2017-04-24 13:16:01,216 [WARNING] rhsmcertd-worker:20237:MainThread @hwprobe.py:600 - Error reading networking information: [Errno 13] Permission denied

Actual results:
As step5.

Expected results:
NO error message

Additional info: extracted from bug 1444714
Comment 1 Kevin Howell 2017-05-03 14:57:59 UTC 
Marking same severity/priority as bug 1444714.
Comment 2 Jiri Hnidek 2017-05-04 19:21:30 UTC 
I cannot reproduce this bug at RHEL-7.3. I'm going to try to reproduce it at RHEL-7.4.
Comment 3 Jiri Hnidek 2017-05-05 12:39:28 UTC 
SELinux is responsible for this error:

tail -f /var/log/audit/audit.log
...
type=AVC msg=audit(1493987724.475:304): avc:  denied  { execute } for  pid=20301 comm="rhsmcertd-worke" name="hostname" dev="dm-0" ino=12595301 scontext=system_u:system_r:rhsmcertd_t:s0 tcontext=system_u:object_r:hostname_exec_t:s0 tclass=file
...
Comment 4 Jiri Hnidek 2017-05-05 18:37:09 UTC 
I solved this problem using python module socket and method getfqdn(). No need to change selinux rules.
Comment 5 Kevin Howell 2017-05-05 20:40:15 UTC 
The reason we switched to invoking hostname was to match the way that puppet and katello are gathering FQDN. Unfortunately, we found in bug 1401394 that if the host is configured with differing IPv4 and IPv6 hostnames, then we get inconsistent results from Python and puppet... So I think we should get the selinux policy bug resolved.
Comment 7 Rehana 2017-05-16 10:24:11 UTC 
Reproducing the failure on the build :
=========================================
[root@dhcp35-71 ~]# subscription-manager version
server type: Red Hat Subscription Management
subscription management server: 2.1.0-1
subscription management rules: 5.23
subscription-manager: 1.19.12-1.el7
python-rhsm: 1.19.6-1.el7
[root@dhcp35-71 ~]# subscription-manager list --installed

No installed products to list

root@dhcp35-71 ~]# subscription-manager register
Registering to: F21-candlepin.usersys.redhat.com:8443/candlepin
Username: admin
Password: 
Organization: admin
The system has been registered with ID: b8e3a9dc-d996-4cef-ab38-4686fcfc4a06 

[root@dhcp35-71 ~]# ll /etc/pki/entitlement/
total 0

[root@dhcp35-71 ~]# service rhsmcertd restart
Redirecting to /bin/systemctl restart rhsmcertd.service


[root@dhcp35-71 ~]# tail -f /var/log/rhsm/rhsm.log
2017-05-16 19:13:38,866 [WARNING] rhsmcertd-worker:31836:MainThread @hwprobe.py:600 - Error reading networking information: [Errno 13] Permission denied
2017-05-16 19:13:38,877 [INFO] rhsmcertd-worker:31836:MainThread @dmiinfo.py:75 - Using dmidecode dump file: /dev/mem
2017-05-16 19:13:38,919 [WARNING] rhsmcertd-worker:31836:MainThread @hwprobe.py:600 - Error reading networking information: [Errno 13] Permission denied
2017-05-16 19:13:38,931 [WARNING] rhsmcertd-worker:31836:MainThread @hwprobe.py:600 - Error reading networking information: [Errno 13] Permission denied
2017-05-16 19:13:38,932 [INFO] rhsmcertd-worker:31836:MainThread @dmiinfo.py:75 - Using dmidecode dump file: /dev/mem
2017-05-16 19:13:39,201 [WARNING] rhsmcertd-worker:31836:MainThread @hwprobe.py:600 - Error reading networking information: [Errno 13] Permission denied
2017-05-16 19:13:39,535 [INFO] rhsmcertd-worker:31836:MainThread @connection.py:520 - Response: status=204, requestUuid=12604d1f-9020-4717-bad0-5e380fe97d15, request="PUT /candlepin/consumers/b8e3a9dc-d996-4cef-ab38-4686fcfc4a06"
Total updates: 0
Found (local) serial# []
Expected (UEP) serial# []
Added (new)
  <NONE>
Deleted (rogue):
  <NONE>
2017-05-16 19:11:21,579 [INFO] subscription-manager:31725:MainThread @connection.py:520 - Response: status=200, requestUuid=5dfa79ed-66a5-4d8e-8bc4-2a015e31757c, request="GET /candlepin/consumers/b8e3a9dc-d996-4cef-ab38-4686fcfc4a06/compliance"
2017-05-16 19:11:21,579 [WARNING] subscription-manager:31725:MainThread @connection.py:524 - Clock skew detected, please check your system time
2017-05-16 19:11:21,580 [INFO] subscription-manager:31725:MainThread @cert_sorter.py:205 - Product status: valid_products= partial_products= expired_products= unentitled_producs= future_products= valid_until=None

# cat /var/log/rhsm/rhsm.log | grep Error 
2018-05-16 19:13:38,866 [WARNING] rhsmcertd-worker:31836:MainThread @hwprobe.py:600 - Error reading networking information: [Errno 13] Permission denied
2018-05-16 19:13:38,919 [WARNING] rhsmcertd-worker:31836:MainThread @hwprobe.py:600 - Error reading networking information: [Errno 13] Permission denied
2018-05-16 19:13:38,931 [WARNING] rhsmcertd-worker:31836:MainThread @hwprobe.py:600 - Error reading networking information: [Errno 13] Permission denied
2018-05-16 19:13:39,201 [WARNING] rhsmcertd-worker:31836:MainThread @hwprobe.py:600 - Error reading networking information: [Errno 13] Permission denied

Verifying with latest subscription-managers packages for RHEL74 
========================================================================

# subscription-manager version
server type: This system is currently not registered.
subscription management server: 0.9.51.21-1
subscription management rules: 5.15.1
subscription-manager: 1.19.13-1.el7
python-rhsm: 1.19.6-1.el7

with selinux packages :
[root@inferno home]# rpm -qa selinux*
selinux-policy-3.13.1-148.el7.noarch
selinux-policy-targeted-3.13.1-148.el7.noarch


[root@inferno home]# START_DATE_TIME=`date "+%m/%d/%Y %T"`
[root@inferno home]#  ausearch -m AVC -m USER_AVC -m SELINUX_ERR -i -ts ${START_DATE_TIME}
<no matches>

^ No AVC denials are observed 

[root@inferno home]# subscription-manager config --server.hostname=F21-candlepin.usersys.redhat.com --server.prefix=/candlepin --server.port=8443

[root@inferno home]# mv /etc/pki/product-default/* /home/

[root@inferno home]# subscription-manager list --installed
No installed products to list

[root@inferno home]# subscription-manager register
Registering to: F21-candlepin.usersys.redhat.com:8443/candlepin
Username: admin
Password: 
Organization: admin
The system has been registered with ID: 8510f421-0003-4302-b5a5-50247ef18334
 
[root@inferno home]# service rhsmcertd restart
Redirecting to /bin/systemctl restart rhsmcertd.service


2017-05-16 15:46:22,060 [INFO] rhsmcertd-worker:5364:MainThread @rhsmcertd-worker:61 - X-Correlation-ID: cc05c80f25e34a469638887aed31a7b9
2017-05-16 15:46:22,064 [INFO] rhsmcertd-worker:5364:MainThread @connection.py:780 - Connection built: host=F21-candlepin.usersys.redhat.com port=8443 handler=/candlepin auth=identity_cert ca_dir=/etc/rhsm/ca/ insecure=False
2017-05-16 15:46:22,817 [INFO] rhsmcertd-worker:5364:MainThread @connection.py:520 - Response: status=200, requestUuid=8d9a82b8-d3ea-4d58-9610-4dbb21b145c0, request="GET /candlepin/"
2017-05-16 15:46:23,636 [INFO] rhsmcertd-worker:5364:MainThread @connection.py:520 - Response: status=200, requestUuid=1cadc50c-bdbd-4649-8602-b32a5c225cf0, request="GET /candlepin/consumers/ee765b94-0e63-4213-8324-3ebe2d69f9d7/certificates/serials"
2017-05-16 15:46:23,638 [INFO] rhsmcertd-worker:5364:MainThread @entcertlib.py:130 - certs updated:
Total updates: 0
Found (local) serial# []
Expected (UEP) serial# []
Added (new)
  <NONE>
Deleted (rogue):
  <NONE>
2017-05-16 15:46:23,836 [INFO] rhsmcertd-worker:5364:MainThread @connection.py:520 - Response: status=200, requestUuid=9022121c-678e-4367-831d-37bfdbed89e5, request="GET /candlepin/status"
2017-05-16 15:46:24,247 [INFO] rhsmcertd-worker:5364:MainThread @connection.py:520 - Response: status=200, requestUuid=490bfd7b-94a9-44e8-8952-e84ea1ad17e9, request="GET /candlepin/consumers/ee765b94-0e63-4213-8324-3ebe2d69f9d7"
2017-05-16 15:46:24,469 [INFO] rhsmcertd-worker:5364:MainThread @connection.py:520 - Response: status=200, requestUuid=7548526e-a7d9-4002-b6c6-3afd230e1305, request="GET /candlepin/consumers/ee765b94-0e63-4213-8324-3ebe2d69f9d7/content_overrides"
2017-05-16 15:46:24,473 [INFO] rhsmcertd-worker:5364:MainThread @repolib.py:329 - repos updated: Repo updates

Total repo updates: 0
Updated
    <NONE>
Added (new)
    <NONE>
Deleted
    <NONE>
2017-05-16 15:46:24,850 [INFO] rhsmcertd-worker:5364:MainThread @dmiinfo.py:75 - Using dmidecode dump file: /dev/mem
2017-05-16 15:46:24,916 [INFO] rhsmcertd-worker:5364:MainThread @cache.py:401 - Server does not support packages, skipping profile upload.
2017-05-16 15:46:25,111 [INFO] rhsmcertd-worker:5407:MainThread @rhsmcertd-worker:61 - X-Correlation-ID: b2282a0ac7854f13b7debfedf6ab3c56
2017-05-16 15:46:25,114 [INFO] rhsmcertd-worker:5407:MainThread @connection.py:780 - Connection built: host=F21-candlepin.usersys.redhat.com port=8443 handler=/candlepin auth=identity_cert ca_dir=/etc/rhsm/ca/ insecure=False
2017-05-16 15:46:25,290 [INFO] rhsmcertd-worker:5407:MainThread @connection.py:520 - Response: status=200, requestUuid=3a0adb88-4816-4976-8906-2d525025a144, request="GET /candlepin/"
2017-05-16 15:46:25,585 [INFO] rhsmcertd-worker:5407:MainThread @connection.py:520 - Response: status=200, requestUuid=c9e8ecca-2d39-404d-a3ee-6e05051b51fa, request="GET /candlepin/consumers/ee765b94-0e63-4213-8324-3ebe2d69f9d7"
2017-05-16 15:46:26,060 [INFO] rhsmcertd-worker:5407:MainThread @connection.py:520 - Response: status=200, requestUuid=2e2a3a5c-c62c-4401-8b0b-7c0371ab684d, request="GET /candlepin/consumers/ee765b94-0e63-4213-8324-3ebe2d69f9d7/compliance"
2017-05-16 15:46:26,061 [INFO] rhsmcertd-worker:5407:MainThread @cert_sorter.py:205 - Product status: valid_products= partial_products= expired_products= unentitled_producs= future_products= valid_until=None
2017-05-16 15:46:26,063 [WARNING] rhsmcertd-worker:5407:MainThread @healinglib.py:114 - Got valid status from server but no valid until date.
2017-05-16 15:46:26,063 [INFO] rhsmcertd-worker:5407:MainThread @healinglib.py:131 - Entitlement auto healing was checked and entitlements are valid today 2017-05-16 10:16:25.586888+00:00
2017-05-16 15:46:26,064 [INFO] rhsmcertd-worker:5407:MainThread @connection.py:780 - Connection built: host=F21-candlepin.usersys.redhat.com port=8443 handler=/candlepin auth=identity_cert ca_dir=/etc/rhsm/ca/ insecure=False
2017-05-16 15:46:26,354 [INFO] rhsmcertd-worker:5407:MainThread @connection.py:520 - Response: status=200, requestUuid=353951cc-c8e8-4e27-967e-bd7c0fa668cb, request="GET /candlepin/consumers/ee765b94-0e63-4213-8324-3ebe2d69f9d7/certificates/serials"
2017-05-16 15:46:26,354 [INFO] rhsmcertd-worker:5407:MainThread @entcertlib.py:130 - certs updated:
Total updates: 0
Found (local) serial# []
Expected (UEP) serial# []
Added (new)
  <NONE>
Deleted (rogue):
  <NONE>
2017-05-16 15:46:26,490 [INFO] rhsmcertd-worker:5407:MainThread @connection.py:520 - Response: status=200, requestUuid=c8cf6b5e-0531-499e-875d-04b1f57e3ee9, request="GET /candlepin/status"

[root@inferno home]# cat /var/log/rhsm/rhsm.log | grep Error
[root@inferno home]#  ausearch -m AVC -m USER_AVC -m SELINUX_ERR -i -ts ${START_DATE_TIME}
<no matches>

No errors are observed in the rhsm.log after rhsmcertd was restarted. Also AVC denials no longer appear . 

Based on the above test steps , marking the bug as verified!!
Comment 8 errata-xmlrpc 2017-08-01 19:23:41 UTC 
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2017:2083