RHEL Engineering is moving the tracking of its product development work on RHEL 6 through RHEL 9 to Red Hat Jira (issues.redhat.com). If you're a Red Hat customer, please continue to file support cases via the Red Hat customer portal. If you're not, please head to the "RHEL project" in Red Hat Jira and file new tickets here. Individual Bugzilla bugs in the statuses "NEW", "ASSIGNED", and "POST" are being migrated throughout September 2023. Bugs of Red Hat partners with an assigned Engineering Partner Manager (EPM) are migrated in late September as per pre-agreed dates. Bugs against components "kernel", "kernel-rt", and "kpatch" are only migrated if still in "NEW" or "ASSIGNED". If you cannot log in to RH Jira, please consult article #7032570. That failing, please send an e-mail to the RH Jira admins at rh-issues@redhat.com to troubleshoot your issue as a user management inquiry. The email creates a ServiceNow ticket with Red Hat. Individual Bugzilla bugs that are migrated will be moved to status "CLOSED", resolution "MIGRATED", and set with "MigratedToJIRA" in "Keywords". The link to the successor Jira issue will be found under "Links", have a little "two-footprint" icon next to it, and direct you to the "RHEL project" in Red Hat Jira (issue links are of type "https://issues.redhat.com/browse/RHEL-XXXX", where "X" is a digit). This same link will be available in a blue banner at the top of the page informing you that that bug has been migrated.
Bug 1803794 - /etc/cron.daily/rhsmd gives error sleep: missing operand
Summary: /etc/cron.daily/rhsmd gives error sleep: missing operand
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat Enterprise Linux 7
Classification: Red Hat
Component: subscription-manager
Version: 7.9
Hardware: Unspecified
OS: Unspecified
high
medium
Target Milestone: rc
: ---
Assignee: candlepin-bugs
QA Contact: Red Hat subscription-manager QE Team
URL:
Whiteboard:
: 1819613 1820635 1821325 (view as bug list)
Depends On: 1794653 1823523
Blocks: 1823780
TreeView+ depends on / blocked
 
Reported: 2020-02-17 13:33 UTC by Rehana
Modified: 2024-10-01 16:29 UTC (History)
50 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of: 1794653
: 1823780 (view as bug list)
Environment:
Last Closed: 2020-09-29 19:23:08 UTC
Target Upstream Version:
Embargoed:

Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Red Hat Knowledge Base (Solution) 4963101 0 None None None 2020-04-06 16:13:42 UTC
Red Hat Product Errata RHBA-2020:3866 0 None None None 2020-09-29 19:23:45 UTC

Description Rehana 2020-02-17 13:33:25 UTC 
+++ This bug was initially created as a clone of Bug #1794653 +++

I am getting an error from the /etc/cron.daily/rhsmd cron job in subscription-manager-1.26.9-1.el8.x86_64 on RHEL 8.2 beta. The error message is

/etc/cron.daily/rhsmd:

sleep: missing operand
Try 'sleep --help' for more information.
/etc/cron.daily/rhsmd: line 19:  8215 Terminated
/usr/libexec/rhsmd -s

I believe this is because I don't have a processTimeout line in /etc/rhsm/rhsm.conf so $config from line 5 is the empty string
However the if clause starting at line 6 is 
if [ -n $config ]; then
  rhsmd_timeout=$config
else
  rhsmd_timeout=300
fi

so line 6 test is now
if [ -n ]; then
which is true so rhsmd_timeout is set to the empty string
I think line 6 should be
if [ -n "$config" ]; then
so the line 6 would be
if [ -n "" ]; then
which is false so rhsmd_timeout gets set to 300 as intended
Comment 2 Chris Snyder 2020-02-17 13:44:42 UTC 
Upstream commit: a71d800985e10de71717b0aaa032ac7a8cf4caac
Comment 5 daryl herzmann 2020-04-02 10:45:48 UTC 
Is this really not going to be fixed until 7.9 or is an async errata coming for 7.8 hosts? I updated to RHEL 7.8 and am seeing this error now.  Thank you.
Comment 6 Vadym Chepkov 2020-04-03 08:12:53 UTC 
Affects all our newly upgraded RHEL7.8 systems
Comment 9 Rehana 2020-04-06 16:12:27 UTC 
*** Bug 1820635 has been marked as a duplicate of this bug. ***
Comment 10 Rehana 2020-04-06 16:13:42 UTC 
*** Bug 1819613 has been marked as a duplicate of this bug. ***
Comment 11 Rehana 2020-04-06 16:14:35 UTC 
*** Bug 1821325 has been marked as a duplicate of this bug. ***
Comment 12 Preslav 2020-04-07 07:53:19 UTC 
Hi,

I confirm what Vadym Chepkov says in comment #6. We see this on all RHEL7.8 systems that do not have a "processTimeout" setting in /etc/rhsm/rhsm.conf. The systems have subscription-manager-1.24.26-1.el7.x86_64 RPM installed.

Sorry, I don't know how to see the Upstream commit: a71d800985e10de71717b0aaa032ac7a8cf4caac mentioned in comment #2 and the fix. In which got repository is it?

I think the code snippet from /etc/cron.daily/rhsmd:

if [ -n $config ]; then
  rhsmd_timeout=$config
else
  rhsmd_timeout=300
fi

is missing double quotes. It should be:

if [ -n "$config" ]; then
  rhsmd_timeout=$config
else
  rhsmd_timeout=300
fi

In order for it to work correctly in the case where "processTimeout" is not defined.

I would say that an update to the subscription-manager RPM is needed.
Comment 13 Rehana 2020-04-07 08:49:48 UTC 
(In reply to Preslav from comment #12)
> Hi,
> 
> I confirm what Vadym Chepkov says in comment #6. We see this on all RHEL7.8
> systems that do not have a "processTimeout" setting in /etc/rhsm/rhsm.conf.
> The systems have subscription-manager-1.24.26-1.el7.x86_64 RPM installed.
> 
> Sorry, I don't know how to see the Upstream commit:
> a71d800985e10de71717b0aaa032ac7a8cf4caac mentioned in comment #2 and the
> fix. In which got repository is it?
> 
> I think the code snippet from /etc/cron.daily/rhsmd:
> 
> if [ -n $config ]; then
>   rhsmd_timeout=$config
> else
>   rhsmd_timeout=300
> fi
> 
> is missing double quotes. It should be:
> 
> if [ -n "$config" ]; then
>   rhsmd_timeout=$config
> else
>   rhsmd_timeout=300
> fi
> 
> In order for it to work correctly in the case where "processTimeout" is not
> defined.
> 
> I would say that an update to the subscription-manager RPM is needed.

Hi Preslav, 

Thanks for reaching out to us, link to referred commit is here https://github.com/candlepin/subscription-manager/pull/2218/commits/a71d800985e10de71717b0aaa032ac7a8cf4caac . We are looking in to options for making this fix available in 7.8 . 

Sorry for the inconvenience and thanks for all your valuable feed back.

thanks,
Rehana
Comment 23 John Sefler 2020-04-23 17:17:59 UTC 
Ahhh!  I totally agree with Sayan's concern in comment 18 and comment 22.
There is a second coding fix that must be made to /etc/cron.daily/rhsmd.

Please add an "--ignore-case" option to the first line of code in /etc/cron.daily/rhsmd
Change this line of code...

> config=$(grep -E "^processTimeout" /etc/rhsm/rhsm.conf | grep -Po "[0-9]+")

to this...

> config=$(grep -iE "^processTimeout" /etc/rhsm/rhsm.conf | grep -Po "[0-9]+")

Moving back to NEW/FailedQA
Comment 24 Sayan Das 2020-04-23 17:26:03 UTC 
(In reply to John Sefler from comment #23)
> Ahhh!  I totally agree with Sayan's concern in comment 18 and comment 22.
> There is a second coding fix that must be made to /etc/cron.daily/rhsmd.
> 
> Please add an "--ignore-case" option to the first line of code in
> /etc/cron.daily/rhsmd
> Change this line of code...
> 
> > config=$(grep -E "^processTimeout" /etc/rhsm/rhsm.conf | grep -Po "[0-9]+")
> 
> to this...
> 
> > config=$(grep -iE "^processTimeout" /etc/rhsm/rhsm.conf | grep -Po "[0-9]+")
> 
> Moving back to NEW/FailedQA


Thanks, John. 

I should have pointed out in my first comment only but I am glad that we are on the same page now.
Comment 26 John Sefler 2020-04-24 17:06:01 UTC 
Let's handle the fix for comment 18 thru comment 24 in NEW Bug 1827708 and clones thereof.
Comment 27 John Sefler 2020-05-12 20:58:50 UTC 
Verifying Version...

[root@intel-s3e8132-01 ~]# cat /etc/redhat-release
Red Hat Enterprise Linux Server release 7.9 Beta (Maipo)
[root@intel-s3e8132-01 ~]# 
[root@intel-s3e8132-01 ~]# rpm -q --whatprovides /etc/cron.daily/rhsmd
subscription-manager-1.24.37-1.el7.x86_64
[root@intel-s3e8132-01 ~]# 
[root@intel-s3e8132-01 ~]# rpm --verify subscription-manager
[root@intel-s3e8132-01 ~]# 
[root@intel-s3e8132-01 ~]# rpm -q subscription-manager --changelog | egrep "1803794|1794653|1823523"
- 1823523: Detect rhsm-icon running without psutil (csnyder)
- 1794653: corrected missing quotes for config check; ENT-2010
[root@intel-s3e8132-01 ~]# 
[root@intel-s3e8132-01 ~]# head /etc/cron.daily/rhsmd
#!/bin/sh
# nightly run of rhsmd to log entitlement expiration/validity errors to syslog
# this is a cron job because it doesn't need to 'phone home'. should that
# change, look into calling the dbus interface from rhsmcertd instead.
config=$(grep -iE "^processTimeout" /etc/rhsm/rhsm.conf | grep -Po "[0-9]+")
if [ -n "$config" ]; then               <================ VERIFIED: $config is now double quoted
  rhsmd_timeout=$config
else
  rhsmd_timeout=300
fi
[root@intel-s3e8132-01 ~]# 

VERIFIED: the logic error in the original if clause will now evaluate as intended with the double quoted "$config" when $config is an empty string.

Note: an empty string is possible in the following malicious config setting...

[root@intel-s3e8132-01 ~]# subscription-manager config --rhsmd.processtimeout=""
[root@intel-s3e8132-01 ~]# 
[root@intel-s3e8132-01 ~]# grep -i processtimeout /etc/rhsm/rhsm.conf 
processTimeout = 
[root@intel-s3e8132-01 ~]# 
[root@intel-s3e8132-01 ~]# time /etc/cron.daily/rhsmd

real	5m0.036s
user	0m0.329s
sys	0m0.139s
[root@intel-s3e8132-01 ~]# 

VERIFIED: even when there is no processTimeout value configured, the default (300 second) value of 5 minutes will be used.

Note: when a customer upgrades from an older version of RHEL-7.y to RHEL-7.9, it is also possible that the /etc/rhsm/rhsm.conf has no configuration setting at all for processTimeout, as simulated here...

[root@intel-s3e8132-01 ~]# sed -i s/^\\[rhsmd\\]//g /etc/rhsm/rhsm.conf
[root@intel-s3e8132-01 ~]# sed -i s/^process.*//g /etc/rhsm/rhsm.conf
[root@intel-s3e8132-01 ~]# 
[root@intel-s3e8132-01 ~]# grep -i "\[rhsmd\]" /etc/rhsm/rhsm.conf 
[root@intel-s3e8132-01 ~]# grep -i processtimeout /etc/rhsm/rhsm.conf 
[root@intel-s3e8132-01 ~]# 
[root@intel-s3e8132-01 ~]# time /etc/cron.daily/rhsmd

real	5m0.026s
user	0m0.338s
sys	0m0.121s
[root@intel-s3e8132-01 ~]# 

VERIFIED: and when there is no processTimeout configuration at all after an upgrade, the default (300 second) value of 5 minutes will be used.
VERIFIED: best of all... no more "sleep: missing operand" when /etc/cron.daily/rhsmd is run
Comment 28 Kurt Bechstein 2020-06-05 19:04:06 UTC 
There is still a problem with the /etc/cron.daily/rhsmd that needs to be corrected.  The processTimeout issue is correct however the script is now killing off an rhsmd process it should not kill.  The script kicks off the rhsmd process and then sleeps the timeout value.  When the script begins to execute again it blindly checks for a /usr/libexec/rhsmd process and kills it when it should be looking specifically for the rhsmd process that it kicked off.  

When this script runs we are now seeing the following every morning in the logs because the rhsmd script is killing the rhsmd process that was started by dbus rather than checking if what it is killing is the proper process:

Jun  5 03:26:02 testhost1 rhsmd: rhsmd process exceeded runtime and was killed.
Jun  5 03:26:05 testhost1 dbus[778]: [system] Activating service name='com.redhat.SubscriptionManager' (using servicehelper)
Jun  5 03:26:05 testhost1 dbus[778]: [system] Successfully activated service 'com.redhat.SubscriptionManager'
Comment 29 Rehana 2020-06-09 14:25:17 UTC 
(In reply to Kurt Bechstein from comment #28)
> There is still a problem with the /etc/cron.daily/rhsmd that needs to be
> corrected.  The processTimeout issue is correct however the script is now
> killing off an rhsmd process it should not kill.  The script kicks off the
> rhsmd process and then sleeps the timeout value.  When the script begins to
> execute again it blindly checks for a /usr/libexec/rhsmd process and kills
> it when it should be looking specifically for the rhsmd process that it
> kicked off.  
> 
> When this script runs we are now seeing the following every morning in the
> logs because the rhsmd script is killing the rhsmd process that was started
> by dbus rather than checking if what it is killing is the proper process:
> 
> Jun  5 03:26:02 testhost1 rhsmd: rhsmd process exceeded runtime and was
> killed.
> Jun  5 03:26:05 testhost1 dbus[778]: [system] Activating service
> name='com.redhat.SubscriptionManager' (using servicehelper)
> Jun  5 03:26:05 testhost1 dbus[778]: [system] Successfully activated service
> 'com.redhat.SubscriptionManager'

Hi Kurt, 

Thanks for the observation, Can you please open a separate bug for this particular scenario ?. Also Can you please help us understand if this is only a log issue or Is there any functional impact ?   

thanks,
Rehana
Comment 30 Kurt Bechstein 2020-06-12 13:48:44 UTC 
Rehana, I will go ahead and open a new bug on this one as requested.  We haven't noticed any actual functional impact from this bug, but I'm assuming dbus is restarting this process for a reason.  I don't necessarily know what that exact reason is, but it would seem reasonable that a little more care can be taken when killing off system processes as root.
Comment 31 Kurt Bechstein 2020-06-12 14:02:40 UTC 
Bug 1846665 has been created.
Comment 34 Ronald 2020-08-11 10:05:38 UTC 
Kurt has created Bug 1846665 and I am adding to that one.

But is seams no one is working on this one.
Comment 40 errata-xmlrpc 2020-09-29 19:23:08 UTC 
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (subscription-manager bug fix update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2020:3866
Note You need to log in before you can comment on or make changes to this bug.