RHEL Engineering is moving the tracking of its product development work on RHEL 6 through RHEL 9 to Red Hat Jira (issues.redhat.com). If you're a Red Hat customer, please continue to file support cases via the Red Hat customer portal. If you're not, please head to the "RHEL project" in Red Hat Jira and file new tickets here. Individual Bugzilla bugs in the statuses "NEW", "ASSIGNED", and "POST" are being migrated throughout September 2023. Bugs of Red Hat partners with an assigned Engineering Partner Manager (EPM) are migrated in late September as per pre-agreed dates. Bugs against components "kernel", "kernel-rt", and "kpatch" are only migrated if still in "NEW" or "ASSIGNED". If you cannot log in to RH Jira, please consult article #7032570. That failing, please send an e-mail to the RH Jira admins at rh-issues@redhat.com to troubleshoot your issue as a user management inquiry. The email creates a ServiceNow ticket with Red Hat. Individual Bugzilla bugs that are migrated will be moved to status "CLOSED", resolution "MIGRATED", and set with "MigratedToJIRA" in "Keywords". The link to the successor Jira issue will be found under "Links", have a little "two-footprint" icon next to it, and direct you to the "RHEL project" in Red Hat Jira (issue links are of type "https://issues.redhat.com/browse/RHEL-XXXX", where "X" is a digit). This same link will be available in a blue banner at the top of the page informing you that that bug has been migrated.
Bug 1167463 - Cronjob /etc/cron.daily/rhsmd fails sometimes with python traceback
Summary: Cronjob /etc/cron.daily/rhsmd fails sometimes with python traceback
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat Enterprise Linux 7
Classification: Red Hat
Component: subscription-manager
Version: 7.0
Hardware: All
OS: Linux
high
high
Target Milestone: rc
: ---
Assignee: candlepin-bugs
QA Contact: John Sefler
URL:
Whiteboard:
: 1142470 (view as bug list)
Depends On: 1103824 1167464 1170768 1171385
Blocks: rhsm-rhel71 rhsm-rhel67 1128951 1168402 1170769 1171386
TreeView+ depends on / blocked
 
Reported: 2014-11-24 20:39 UTC by Devan Goodwin
Modified: 2019-07-11 08:23 UTC (History)
14 users (show)

Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Clone Of: 1103824
: 1170769 1171386 (view as bug list)
Environment:
Last Closed: 2015-03-05 13:16:15 UTC
Target Upstream Version:
Embargoed:

Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Red Hat Product Errata RHBA-2015:0502 0 normal SHIPPED_LIVE subscription-manager bug fix and enhancement update 2015-03-05 16:20:39 UTC

Comment 2 Devan Goodwin 2014-11-26 13:27:14 UTC 
Built as subscription-manager-1.13.9-1.el7. For verification steps please see: https://bugzilla.redhat.com/show_bug.cgi?id=1103824#c24
Comment 3 Shwetha Kallesh 2014-11-28 14:11:50 UTC 
Verified!!

[root@dhcp35-236 ~]# subscription-manager version
server type: Red Hat Subscription Management
subscription management server: Unknown
subscription management rules: Unknown
subscription-manager: 1.13.9-1.el7
python-rhsm: 1.13.8-1.el7


[root@dhcp35-236 ~]# /etc/cron.daily/rhsmd
     
[root@dhcp35-236 ~]# echo $?
0
  
[root@dhcp35-236 ~]# tail -f /var/log/messages
   Nov 28 19:23:59 dhcp35-236 gnome-session: (gnome-settings-daemon:17555): power-plugin-WARNING **: failed to turn the panel off: Display is not DPMS capable
Nov 28 19:24:00 dhcp35-236 gnome-session: (gnome-settings-daemon:17555): power-plugin-WARNING **: failed to turn the panel on: Display is not DPMS capable
Nov 28 19:24:15 dhcp35-236 gnome-session: (gnome-settings-daemon:17555): power-plugin-WARNING **: failed to turn the panel off: Display is not DPMS capable
Nov 28 19:30:01 dhcp35-236 systemd: Starting Session 231 of user root.
Nov 28 19:30:01 dhcp35-236 systemd: Started Session 231 of user root.
Nov 28 19:37:26 dhcp35-236 rhsmd: This system is missing one or more subscriptions. Please run subscription-manager for more information.
Nov 28 19:40:01 dhcp35-236 systemd: Starting Session 232 of user root.
Nov 28 19:40:01 dhcp35-236 systemd: Started Session 232 of user root.

     
   
 [root@dhcp35-236 ~]# tail -f /var/log/rhsm/rhsm.log
    2014-11-28 19:37:25,960 [INFO] rhsmd @rhsmd:273 - rhsmd started
2014-11-28 19:37:25,961 [INFO] rhsmd @rhsmd:304 - logging subscription status to syslog
2014-11-28 19:37:25,996 [DEBUG] rhsmd @identity.py:131 - Loading consumer info from identity certificates.
2014-11-28 19:37:26,000 [DEBUG] rhsmd @profile.py:97 - Loading current RPM profile.
2014-11-28 19:37:26,059 [INFO] rhsmd @connection.py:679 - Using certificate authentication: key = /etc/pki/consumer/key.pem, cert = /etc/pki/consumer/cert.pem, ca = /etc/rhsm/ca/, insecure = True
2014-11-28 19:37:26,059 [INFO] rhsmd @connection.py:690 - Connection Built: host: mock-server-hostname/ip, port: 8444, handler: /subscription
2014-11-28 19:37:26,059 [INFO] rhsmd @cache.py:138 - Checking current system info against cache: /var/lib/rhsm/cache/installed_products.json
2014-11-28 19:37:26,060 [INFO] rhsmd @cache.py:155 - No changes.
2014-11-28 19:37:26,060 [DEBUG] rhsmd @certdirectory.py:216 - Installed product IDs: ['69', '32060']
2014-11-28 19:37:26,061 [DEBUG] rhsmd @connection.py:466 - Making request: GET /subscription/consumers/5fd2cb84-d6e2-4474-9452-603c9a8fee75/compliance
2014-11-28 19:37:26,066 [ERROR] rhsmd @cache.py:204 - [Errno -2] Name or service not known
2014-11-28 19:37:26,066 [WARNING] rhsmd @cache.py:210 - Unable to reach server, using cached status.
2014-11-28 19:37:26,066 [DEBUG] rhsmd @cert_sorter.py:193 - valid entitled products: []
2014-11-28 19:37:26,066 [DEBUG] rhsmd @cert_sorter.py:194 - expired entitled products: []
2014-11-28 19:37:26,066 [DEBUG] rhsmd @cert_sorter.py:195 - partially entitled products: []
2014-11-28 19:37:26,066 [DEBUG] rhsmd @cert_sorter.py:196 - unentitled products: [u'69', u'32060']
2014-11-28 19:37:26,066 [DEBUG] rhsmd @cert_sorter.py:197 - future products: []
2014-11-28 19:37:26,067 [DEBUG] rhsmd @cert_sorter.py:198 - partial stacks: []
2014-11-28 19:37:26,067 [DEBUG] rhsmd @cert_sorter.py:199 - entitlements valid until: None
2014-11-28 19:37:26,068 [INFO] rhsmd @rhsmd:266 - rhsmd: This system is missing one or more subscriptions. Please run subscription-manager for more information.
Comment 5 Shwetha Kallesh 2014-12-04 12:54:46 UTC 
Verified!!

[root@dhcp35-236 ~]# subscription-manager version
server type: Red Hat Subscription Management
subscription management server: Unknown
subscription management rules: Unknown
subscription-manager: 1.13.10-1.el7
python-rhsm: 1.13.8-1.el7
[root@dhcp35-236 ~]# 

[root@dhcp35-236 ~]# /etc/cron.daily/rhsmd
[root@dhcp35-236 ~]# echo $?
0
 
[root@dhcp35-236 ~]# tail -f /var/log/messages
Dec  3 22:03:44 dhcp35-236 subscription-manager: Removed subscription for product 'Awesome OS Instance Server Bits'
Dec  3 22:03:44 dhcp35-236 python: detected unhandled Python exception in '/usr/share/rhsm/subscription_manager/plugin/ostree/gi_wrapper.py'
Dec  3 22:03:45 dhcp35-236 abrt-server: Package 'subscription-manager-plugin-ostree' isn't signed with proper key
Dec  3 22:03:45 dhcp35-236 abrt-server: 'post-create' on '/var/tmp/abrt/Python-2014-12-03-22:03:44-24709' exited with 1
Dec  3 22:03:45 dhcp35-236 abrt-server: Deleting problem directory '/var/tmp/abrt/Python-2014-12-03-22:03:44-24709'
Dec  3 22:03:45 dhcp35-236 dbus-daemon: dbus[697]: [system] Activating service name='com.redhat.SubscriptionManager' (using servicehelper)
Dec  3 22:03:45 dhcp35-236 dbus[697]: [system] Activating service name='com.redhat.SubscriptionManager' (using servicehelper)
Dec  3 22:03:45 dhcp35-236 dbus-daemon: dbus[697]: [system] Successfully activated service 'com.redhat.SubscriptionManager'
Dec  3 22:03:45 dhcp35-236 dbus[697]: [system] Successfully activated service 'com.redhat.SubscriptionManager'
Dec  3 22:05:18 dhcp35-236 rhsmd: This system is missing one or more subscriptions. Please run subscription-manager for more information.
 
 
[root@dhcp35-236 ~]# tail -f /var/log/rhsm/rhsm.log
2014-12-03 22:05:18,564 [ERROR] rhsmd @cache.py:204 - Server error attempting a GET to /candlepin/consumers/b6f22fd5-b5ae-4456-a600-e42947d95d04/compliance returned status 404
2014-12-03 22:05:18,564 [WARNING] rhsmd @cache.py:210 - Unable to reach server, using cached status.
2014-12-03 22:05:18,565 [DEBUG] rhsmd @cert_sorter.py:193 - valid entitled products: []
2014-12-03 22:05:18,565 [DEBUG] rhsmd @cert_sorter.py:194 - expired entitled products: []
2014-12-03 22:05:18,565 [DEBUG] rhsmd @cert_sorter.py:195 - partially entitled products: []
2014-12-03 22:05:18,606 [DEBUG] rhsmd @cert_sorter.py:196 - unentitled products: [u'32060']
2014-12-03 22:05:18,606 [DEBUG] rhsmd @cert_sorter.py:197 - future products: []
2014-12-03 22:05:18,606 [DEBUG] rhsmd @cert_sorter.py:198 - partial stacks: []
2014-12-03 22:05:18,607 [DEBUG] rhsmd @cert_sorter.py:199 - entitlements valid until: None
2014-12-03 22:05:18,607 [INFO] rhsmd @rhsmd:295 - rhsmd: This system is missing one or more subscriptions. Please run subscription-manager for more information.
 

Index error:

[root@dhcp35-236 ~]# /etc/cron.daily/rhsmd
[root@dhcp35-236 ~]# echo $?
0

[root@dhcp35-236 ~]# tail -f /var/log/rhsm/rhsm.log
  File "/usr/lib64/python2.7/site-packages/rhsm/connection.py", line 889, in getCompliance
    return self.conn.request_get(method)
  File "/usr/lib64/python2.7/site-packages/rhsm/connection.py", line 570, in request_get
    return self._request("GET", method)
  File "/usr/lib64/python2.7/site-packages/rhsm/connection.py", line 473, in _request
    conn.request(request_type, handler, body=body, headers=headers)
  File "/usr/lib64/python2.7/httplib.py", line 973, in request
    raise IndexError()
IndexError


[root@dhcp35-236 ~]# tail -f /var/log/messages
Dec  4 18:01:01 dhcp35-236 systemd: Started Session 1265 of user root.
Dec  4 18:10:01 dhcp35-236 systemd: Starting Session 1266 of user root.
Dec  4 18:10:01 dhcp35-236 systemd: Started Session 1266 of user root.
Dec  4 18:11:02 dhcp35-236 systemd-logind: Removed session 1222.
Dec  4 18:12:25 dhcp35-236 systemd-logind: Removed session 1231.
Dec  4 18:18:33 dhcp35-236 systemd-logind: New session 1267 of user root.
Dec  4 18:18:33 dhcp35-236 systemd: Starting Session 1267 of user root.
Dec  4 18:18:33 dhcp35-236 systemd: Started Session 1267 of user root.
Dec  4 18:20:01 dhcp35-236 systemd: Starting Session 1268 of user root.
Dec  4 18:20:01 dhcp35-236 systemd: Started Session 1268 of user root.
Comment 11 William Poteat 2015-01-16 16:05:18 UTC 
*** Bug 1142470 has been marked as a duplicate of this bug. ***
Comment 12 Shwetha Kallesh 2015-01-19 19:01:34 UTC 
For the reproducer:
--------------------------------------------------------------------------------
[root@dell-p690-01 ~]# subscription-manager version
server type: Red Hat Subscription Management
subscription management server: Unknown
subscription management rules: Unknown
subscription-manager: 1.13.8-1.el7
python-rhsm: 1.13.7-1.el7

[root@dell-p690-01 ~]# subscription-manager  register --serverurl subscription.rhn.stage.redhat.com
Username: qa
Password:
The system has been registered with ID: 465bfca6-8a70-45dc-9865-419cd08c13e7


edit the file /usr/lib64/python2.7/httplib.py to raise IndexError;

[root@dell-p690-01 ~]# mail
No mail for root

[root@dell-p690-01 ~]# crontab -e
no crontab for root - using an empty one
crontab: installing new crontab

[root@dell-p690-01 ~]# crontab -l
* * * * * /etc/cron.daily/rhsmd
[root@dell-p690-01 ~]# sleep 60
You have new mail in /var/spool/mail/root
[root@dell-p690-01 ~]# mail
Heirloom Mail version 12.5 7/5/10.  Type ? for help.
"/var/spool/mail/root": 2 messages 2 new
>N  1 (Cron Daemon)         Mon Jan 19 13:54  43/2066  "Cron <root@dell-p690-01> /etc/cron.daily/rhsmd"
 N  2 (Cron Daemon)         Mon Jan 19 13:55  43/2066  "Cron <root@dell-p690-01> /etc/cron.daily/rhsmd"
& 1
Message  1:
From root.eng.bos.redhat.com  Mon Jan 19 13:54:01 2015
Return-Path: <root.eng.bos.redhat.com>
Date: Mon, 19 Jan 2015 13:54:01 -0500
From: "(Cron Daemon)" <root.eng.bos.redhat.com>
To: root.eng.bos.redhat.com
Subject: Cron <root@dell-p690-01> /etc/cron.daily/rhsmd
Content-Type: text/plain; charset=UTF-8
Auto-Submitted: auto-generated
Precedence: bulk
X-Cron-Env: <XDG_SESSION_ID=52>
X-Cron-Env: <XDG_RUNTIME_DIR=/run/user/0>
X-Cron-Env: <LANG=en_US.UTF-8>
X-Cron-Env: <SHELL=/bin/sh>
X-Cron-Env: <HOME=/root>
X-Cron-Env: <PATH=/usr/bin:/bin>
X-Cron-Env: <LOGNAME=root>
X-Cron-Env: <USER=root>
Status: R

Traceback (most recent call last):
  File "/usr/libexec/rhsmd", line 34, in <module>
    from subscription_manager.injectioninit import init_dep_injection
  File "/usr/share/rhsm/subscription_manager/injectioninit.py", line 18, in <module>
    from subscription_manager.cache import ProductStatusCache, EntitlementStatusCache, OverrideStatusCache, \
  File "/usr/share/rhsm/subscription_manager/cache.py", line 27, in <module>
    from M2Crypto import SSL
  File "/usr/lib64/python2.7/site-packages/M2Crypto/__init__.py", line 42, in <module>
    import m2urllib
  File "/usr/lib64/python2.7/site-packages/M2Crypto/m2urllib.py", line 10, in <module>
    import httpslib
  File "/usr/lib64/python2.7/site-packages/M2Crypto/httpslib.py", line 10, in <module>
    from httplib import *
  File "/usr/lib64/python2.7/httplib.py", line 974
    self._send_request(method, url, body, headers)
    ^
IndentationError: unexpected indent



verification:

--------------------------------------------------------------------------------


[root@dell-p690-01 ~]# subscription-manager  version
server type: Red Hat Subscription Management
subscription management server: Unknown
subscription management rules: Unknown
subscription-manager: 1.13.12-1.el7
python-rhsm: 1.13.8-1.el7

[root@dell-p690-01 ~]# crontab -e
no crontab for root - using an empty one
crontab: installing new crontab
[root@dell-p690-01 ~]# crontab -l
* * * * * /etc/cron.daily/rhsmd
[root@dell-p690-01 ~]# sleep 60
[root@dell-p690-01 ~]# mail
No mail for root
[root@dell-p690-01 ~]# vi /usr/lib64/python2.7/httplib.py
[root@dell-p690-01 ~]# sleep 60
[root@dell-p690-01 ~]# mail
No mail for root
[root@dell-p690-01 ~]# tail -f /var/log/rhsm/rhsm.log
    import m2urllib
  File "/usr/lib64/python2.7/site-packages/M2Crypto/m2urllib.py", line 10, in <module>
    import httpslib
  File "/usr/lib64/python2.7/site-packages/M2Crypto/httpslib.py", line 10, in <module>
    from httplib import *
  File "/usr/lib64/python2.7/httplib.py", line 974
    self._send_request(method, url, body, headers)
   ^
IndentationError: unexpected indent
Comment 13 Shwetha Kallesh 2015-01-19 19:02:09 UTC 
With mock-server,

For the reproducer:
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
[root@dell-p690-01 ~]# subscription-manager  register --serverurl subscription.rhn.stage.redhat.com
Username: qa
Password:
The system has been registered with ID: 465bfca6-8a70-45dc-9865-419cd08c13e7


[root@dell-p690-01 ~]# subscription-manager config --server.hostname=10.70.35.236 --server.port=8445 --server.insecure=1
[root@dell-p690-01 ~]# crontab -r
[root@dell-p690-01 ~]# mail
No mail for root
[root@dell-p690-01 ~]# crontab -e
no crontab for root - using an empty one
crontab: installing new crontab
[root@dell-p690-01 ~]# crontab -l
* * * * * /etc/cron.daily/rhsmd


[root@dell-p690-01 ~]# sleep 60
[root@dell-p690-01 ~]# mail
Heirloom Mail version 12.5 7/5/10.  Type ? for help.
"/var/spool/mail/root": 2 messages 2 new
>N  1 (Cron Daemon)         Mon Jan 19 13:33  27/1197  "Cron <root@dell-p690-01> /etc/cron.daily/rhsmd"
 N  2 (Cron Daemon)         Mon Jan 19 13:34  27/1197  "Cron <root@dell-p690-01> /etc/cron.daily/rhsmd"
& 1
Message  1:
From root.eng.bos.redhat.com  Mon Jan 19 13:33:01 2015
Return-Path: <root.eng.bos.redhat.com>
Date: Mon, 19 Jan 2015 13:33:01 -0500
From: "(Cron Daemon)" <root.eng.bos.redhat.com>
To: root.eng.bos.redhat.com
Subject: Cron <root@dell-p690-01> /etc/cron.daily/rhsmd
Content-Type: text/plain; charset=UTF-8
Auto-Submitted: auto-generated
Precedence: bulk
X-Cron-Env: <XDG_SESSION_ID=33>
X-Cron-Env: <XDG_RUNTIME_DIR=/run/user/0>
X-Cron-Env: <LANG=en_US.UTF-8>
X-Cron-Env: <SHELL=/bin/sh>
X-Cron-Env: <HOME=/root>
X-Cron-Env: <PATH=/usr/bin:/bin>
X-Cron-Env: <LOGNAME=root>
X-Cron-Env: <USER=root>
Status: R

/bin/sh: /etc/cron.daily/rhsmd: No such file or directory

&


Verification Part:
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

[root@dell-p690-01 ~]# subscription-manager  version
server type: Red Hat Subscription Management
subscription management server: Unknown
subscription management rules: Unknown
subscription-manager: 1.13.12-1.el7
python-rhsm: 1.13.8-1.el7

[root@dell-p690-01 ~]# subscription-manager config --server.hostname=10.70.35.236 --server.port=8445 --server.insecure=1
[root@dell-p690-01 ~]# crontab -r
[root@dell-p690-01 ~]# mail
No mail for root
[root@dell-p690-01 ~]# crontab -e
no crontab for root - using an empty one
crontab: installing new crontab
[root@dell-p690-01 ~]# crontab -l
* * * * * /etc/cron.daily/rhsmd
[root@dell-p690-01 ~]# sleep 60
[root@dell-p690-01 ~]# mail
No mail for root


Verified!!
Comment 15 errata-xmlrpc 2015-03-05 13:16:15 UTC 
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://rhn.redhat.com/errata/RHBA-2015-0502.html
Note You need to log in before you can comment on or make changes to this bug.