1827103 – [RHEL/Bug] log rotation of rhsm.log not correctly handled [rhel-7.8.z]

RHEL Engineering is moving the tracking of its product development work on RHEL 6 through RHEL 9 to Red Hat Jira (issues.redhat.com). If you're a Red Hat customer, please continue to file support cases via the Red Hat customer portal. If you're not, please head to the "RHEL project" in Red Hat Jira and file new tickets here. Individual Bugzilla bugs in the statuses "NEW", "ASSIGNED", and "POST" are being migrated throughout September 2023. Bugs of Red Hat partners with an assigned Engineering Partner Manager (EPM) are migrated in late September as per pre-agreed dates. Bugs against components "kernel", "kernel-rt", and "kpatch" are only migrated if still in "NEW" or "ASSIGNED". If you cannot log in to RH Jira, please consult article #7032570. That failing, please send an e-mail to the RH Jira admins at rh-issues@redhat.com to troubleshoot your issue as a user management inquiry. The email creates a ServiceNow ticket with Red Hat. Individual Bugzilla bugs that are migrated will be moved to status "CLOSED", resolution "MIGRATED", and set with "MigratedToJIRA" in "Keywords". The link to the successor Jira issue will be found under "Links", have a little "two-footprint" icon next to it, and direct you to the "RHEL project" in Red Hat Jira (issue links are of type "https://issues.redhat.com/browse/RHEL-XXXX", where "X" is a digit). This same link will be available in a blue banner at the top of the page informing you that that bug has been migrated.

Bug 1827103 - [RHEL/Bug] log rotation of rhsm.log not correctly handled [rhel-7.8.z]

Summary: [RHEL/Bug] log rotation of rhsm.log not correctly handled [rhel-7.8.z]

Keywords:
Status:	CLOSED ERRATA
Alias:	None
Product:	Red Hat Enterprise Linux 7
Classification:	Red Hat
Component:	subscription-manager
Sub Component:
Version:	7.6
Hardware:	All
OS:	Linux
Priority:	high
Severity:	medium
Target Milestone:	rc
Target Release:	---
Assignee:	candlepin-bugs
QA Contact:	Red Hat subscription-manager QE Team
Docs Contact:
URL:
Whiteboard:
Depends On:	1803783
Blocks:
TreeView+	depends on / blocked

Reported:	2020-04-23 09:45 UTC by RAD team bot copy to z-stream
Modified:	2023-09-07 22:55 UTC (History)
CC List:	8 users (show)
Fixed In Version:
Doc Type:	If docs needed, set a value
Doc Text:
Clone Of:	1803783
Environment:
Last Closed:	2020-05-12 18:40:21 UTC
Target Upstream Version:
Embargoed:

Attachments	(Terms of Use)

Links
System	ID	Priority	Status	Summary	Last Updated
Github	candlepin subscription-manager pull 2225	None	closed	1803783: Added copytruncate option to logrotate conf file; ENT-2114	2020-07-08 05:14:17 UTC
Red Hat Knowledge Base (Solution)	5088661	None	None	None	2020-05-19 13:14:55 UTC
Red Hat Product Errata	RHBA-2020:2101	None	None	None	2020-05-12 18:40:25 UTC

Description RAD team bot copy to z-stream 2020-04-23 09:45:04 UTC

This bug has been copied from bug #1803783 and has been proposed to be backported to 7.8 z-stream (EUS).

Comment 4 Rehana 2020-04-28 07:11:08 UTC

Reproducing the issue : 
========================
Register and subscribe the system 
with virt-who /rhsmd running 

Steps 1: Make virt-who /rhsmd running ;
============================
#service virt-who restart and
# /usr/libexec/rhsmd -k in terminal

Step 2 : execute some sub-man commands to create log entries
=============================================
[root@kvm-06-guest10 rhsm]# subscription-manager remove --all
1 local certificate has been deleted.
1 subscription removed at the server.
 
Step 3 : force logrotate to run
======================
[root@kvm-06-guest10 rhsm]# logrotate -f /etc/logrotate.conf
 
[root@kvm-06-guest10 rhsm]# ls
rhsmcertd.log  rhsmcertd.log-20200422  rhsm.log  rhsm.log-20200422
 
^^ notice "rhsm.log-20200422" is now created
 
Step 4: Grep for open files
===================
 
[root@kvm-06-guest10 rhsm]# sudo lsof | grep deleted | grep rhsm
rhsmd     21406          root    6u      REG              253,0      4096  100709773 /tmp/ffib6dPFX (deleted)
[root@kvm-06-guest10 rhsm]# rm -rf rhsm.log-20200422

^^ Delete the file 

[root@kvm-06-guest10 rhsm]# sudo lsof | grep deleted | grep rhsm
rhsmd     21406          root    4w      REG              253,0     20908   34568341 /var/log/rhsm/rhsm.log-20200422 (deleted)
rhsmd     21406          root    6u      REG              253,0      4096  100709773 /tmp/ffib6dPFX (deleted)
gmain     21406 21417    root    4w      REG              253,0     20908   34568341 /var/log/rhsm/rhsm.log-20200422 (deleted)
virt-who  21871          root    5w      REG              253,0     20908   34568341 /var/log/rhsm/rhsm.log-20200422 (deleted)
virt-who  21871          root    6w      REG              253,0     20908   34568341 /var/log/rhsm/rhsm.log-20200422 (deleted)
virt-who  21871          root    7w      REG              253,0     20908   34568341 /var/log/rhsm/rhsm.log-20200422 (deleted)
virt-who  21871 21876    root    5w      REG              253,0     20908   34568341 /var/log/rhsm/rhsm.log-20200422 (deleted)
virt-who  21871 21876    root    6w      REG              253,0     20908   34568341 /var/log/rhsm/rhsm.log-20200422 (deleted)
virt-who  21871 21876    root    7w      REG              253,0     20908   34568341 /var/log/rhsm/rhsm.log-20200422 (deleted)
virt-who  21871 21877    root    5w      REG              253,0     20908   34568341 /var/log/rhsm/rhsm.log-20200422 (deleted)
virt-who  21871 21877    root    6w      REG              253,0     20908   34568341 /var/log/rhsm/rhsm.log-20200422 (deleted)
virt-who  21871 21877    root    7w      REG              253,0     20908   34568341 /var/log/rhsm/rhsm.log-20200422 (deleted)
virt-who  21871 21878    root    5w      REG              253,0     20908   34568341 /var/log/rhsm/rhsm.log-20200422 (deleted)
virt-who  21871 21878    root    6w      REG              253,0     20908   34568341 /var/log/rhsm/rhsm.log-20200422 (deleted)
virt-who  21871 21878    root    7w      REG              253,0     20908   34568341 /var/log/rhsm/rhsm.log-20200422 (deleted)
virt-who  21871 21879    root    5w      REG              253,0     20908   34568341 /var/log/rhsm/rhsm.log-20200422 (deleted)
virt-who  21871 21879    root    6w      REG              253,0     20908   34568341 /var/log/rhsm/rhsm.log-20200422 (deleted)
virt-who  21871 21879    root    7w      REG              253,0     20908   34568341 /var/log/rhsm/rhsm.log-20200422 (deleted)
 
[root@kvm-06-guest10 rhsm]# cat /etc/logrotate.d/subscription-manager
/var/log/rhsm/*.log {
    missingok
    notifempty
    sharedscripts
}


Verifying on RHEL 78 with updating build from internal 7.8z brew build ( not shown ) 
==================================================================================
subscription management server: 2.9.21-1
subscription management rules: 5.37
subscription-manager: 1.24.26-3.el7_8

[root@kvm-01-guest27 rhsm]# cat /etc/logrotate.d/subscription-manager 
/var/log/rhsm/*.log {
    missingok
    notifempty
    sharedscripts
    copytruncate  ---> Notice the new "copytruncate" option added to the logrotate subscription manager file 
}

Register the system 

[root@kvm-01-guest27 rhsm]# subscription-manager register --username=**** --password=****
Registering to: subscription.rhsm.stage.redhat.com:443/subscription
The system has been registered with ID: d8d53e95-4c8a-43fc-ab13-af28b5e88399
The registered system name is: kvm-01-guest27.lab.eng.rdu2.redhat.com

Make sure the services are running 

[root@kvm-01-guest27 rhsm]# ps -aux | grep virt-who
root     20829  0.0  0.7 965288 30292 ?        Ssl  02:52   0:00 /usr/bin/python /usr/bin/virt-who
root     21313  0.0  0.0 112808   968 pts/0    R+   03:05   0:00 grep --color=auto virt-who
[root@kvm-01-guest27 rhsm]# ps -aux | grep rhsmd
root     20801  0.0  0.8 564560 32672 pts/1    Sl+  02:52   0:00 /usr/bin/python /usr/libexec/rhsmd -k
root     21315  0.0  0.0 112808   968 pts/0    R+   03:05   0:00 grep --color=auto rhsmd
[root@kvm-01-guest27 rhsm]# 

Execute logroate -f /etc/logrotate.conf

[root@kvm-01-guest27 rhsm]# logrotate -f /etc/logrotate.conf
[root@kvm-01-guest27 rhsm]# ll
total 8
-rw-r--r--. 1 root root    0 Apr 28 02:53 rhsmcertd.log
-rw-r--r--. 1 root root    0 Apr 28 03:06 rhsm.log
-rw-r--r--. 1 root root 4622 Apr 28 03:06 rhsm.log-20200428


Delete the file rhsm.log-20200428


[root@kvm-01-guest27 rhsm]# rm -rf rhsm.log-20200428 

Notice that the service are still running 

[root@kvm-01-guest27 rhsm]# ps -aux | grep rhsmd
root     20801  0.0  0.8 564560 32672 pts/1    Sl+  02:52   0:00 /usr/bin/python /usr/libexec/rhsmd -k
root     21320  0.0  0.0 112808   968 pts/0    R+   03:07   0:00 grep --color=auto rhsmd
[root@kvm-01-guest27 rhsm]# ps -aux | grep virt-who
root     20829  0.0  0.8 965288 32316 ?        Ssl  02:52   0:00 /usr/bin/python /usr/bin/virt-who
root     21322  0.0  0.0 112808   968 pts/0    R+   03:07   0:00 grep --color=auto virt-who


Lets the lsof results for rhsm file 

[root@kvm-01-guest27 rhsm]# lsof | grep deleted | grep rhsm*
[root@kvm-01-guest27 rhsm]# lsof | grep deleted | grep rhsm
rhsmd     20801          root    6u      REG              253,0      4096  100664085 /tmp/#100664085 (deleted)
[root@kvm-01-guest27 rhsm]# ll
total 0
-rw-r--r--. 1 root root 0 Apr 28 02:53 rhsmcertd.log
-rw-r--r--. 1 root root 0 Apr 28 03:06 rhsm.log


^^ Notice there are no reference to the deleted rhsm log file. 

Based on the above observation moving the bug to verified

Comment 6 errata-xmlrpc 2020-05-12 18:40:21 UTC

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2020:2101

Note You need to log in before you can comment on or make changes to this bug.