1940267 – insights-client change log management to logrotate standard

RHEL Engineering is moving the tracking of its product development work on RHEL 6 through RHEL 9 to Red Hat Jira (issues.redhat.com). If you're a Red Hat customer, please continue to file support cases via the Red Hat customer portal. If you're not, please head to the "RHEL project" in Red Hat Jira and file new tickets here. Individual Bugzilla bugs in the statuses "NEW", "ASSIGNED", and "POST" are being migrated throughout September 2023. Bugs of Red Hat partners with an assigned Engineering Partner Manager (EPM) are migrated in late September as per pre-agreed dates. Bugs against components "kernel", "kernel-rt", and "kpatch" are only migrated if still in "NEW" or "ASSIGNED". If you cannot log in to RH Jira, please consult article #7032570. That failing, please send an e-mail to the RH Jira admins at rh-issues@redhat.com to troubleshoot your issue as a user management inquiry. The email creates a ServiceNow ticket with Red Hat. Individual Bugzilla bugs that are migrated will be moved to status "CLOSED", resolution "MIGRATED", and set with "MigratedToJIRA" in "Keywords". The link to the successor Jira issue will be found under "Links", have a little "two-footprint" icon next to it, and direct you to the "RHEL project" in Red Hat Jira (issue links are of type "https://issues.redhat.com/browse/RHEL-XXXX", where "X" is a digit). This same link will be available in a blue banner at the top of the page informing you that that bug has been migrated.

Bug 1940267 - insights-client change log management to logrotate standard

Summary: insights-client change log management to logrotate standard

Keywords:
Status:	CLOSED ERRATA
Alias:	None
Product:	Red Hat Enterprise Linux 8
Classification:	Red Hat
Component:	insights-client
Sub Component:
Version:	8.9
Hardware:	All
OS:	Linux
Priority:	medium
Severity:	medium
Target Milestone:	rc
Target Release:	---
Assignee:	Jason Jerome
QA Contact:	Pavol Kotvan
Docs Contact:
URL:
Whiteboard:
Depends On:
Blocks:	2189281
TreeView+	depends on / blocked

Reported:	2021-03-18 00:56 UTC by Bryan Kinney
Modified:	2023-11-14 17:02 UTC (History)
CC List:	11 users (show)
Fixed In Version:	insights-client-3.2.0-2.el8
Doc Type:	If docs needed, set a value
Doc Text:
Clone Of:
Clones:	2189281 (view as bug list)
Environment:
Last Closed:	2023-11-14 15:35:31 UTC
Type:	Bug
Target Upstream Version:
Embargoed:
Dependent Products:

Attachments	(Terms of Use)

Links
System	ID	Private	Priority	Status	Summary	Last Updated
Red Hat Issue Tracker	RHELPLAN-105917	0	None	None	None	2021-12-15 15:20:36 UTC
Red Hat Product Errata	RHBA-2023:7023	0	None	None	None	2023-11-14 15:35:49 UTC

Description Bryan Kinney 2021-03-18 00:56:00 UTC

Description of problem:

The current logging method used by the insights-client presents several problems.
  - When any change to logging is required a bug needs to be filed (Example: bug 1734283)
  - Changes or new requirements create new issues (Example: bug 1923835)
  - Use of certain parameters force changes to be made (Example: bug 1734283)

It is believed the use of a mature system like logrotate will alleviate most of those problems.
There are many solutions for logrotate and it provides a flowing history that can be kept as the user sees fit.

Version-Release number of selected component (if applicable):


How reproducible:
N/A

Steps to Reproduce:
1.
2.
3.

Actual results:
Log features and rotation require internal changes to insights-client

Expected results:
Use the logrotate system to handle all logging in the insights-client

Additional info:

Comment 8 Archana Pandey 2023-06-06 12:17:05 UTC

pre-verification -
>> Case 1: latest core and  scratch build of insights-client

using core https://github.com/RedHatInsights/insights-core/pull/3765/

[root@iqe-vm-arpandey insights-core]# insights-client --version
Client: 3.2.0
Core: 3.0.8-dev

[root@iqe-vm-arpandey insights-core]# rpm -qa | grep insights-client
insights-client-3.2.0.git.424.5dfefcf7-0.el8.noarch
[root@iqe-vm-arpandey insights-core]# 

[root@iqe-vm-arpandey insights-core]# cat /etc/logrotate.d/insights-client
/var/log/insights-client/*.log {
    rotate 4
    weekly
    missingok
    notifempty
    sharedscripts
    copytruncate
    create 0644 root root
}                                         ^^^^^------------ conf file for insights-client logrotate feature installed
[root@iqe-vm-arpandey insights-core]#

running insights-client to test if logs are getting rotated 'weekly'

[root@iqe-vm-arpandey insights-core]# insights-client
Starting to collect Insights data for iqe-vm-arpandey
Uploading Insights data.
Successfully uploaded report for iqe-vm-arpandey.
[root@iqe-vm-arpandey insights-core]# ll /var/log/insights-client/
total 448
-rw-r--r--. 1 root root 258746 Jun  6 08:00 insights-client.log <<< logs not rotated
[root@iqe-vm-arpandey insights-core]# insights-client
Starting to collect Insights data for iqe-vm-arpandey
Uploading Insights data.
Successfully uploaded report for iqe-vm-arpandey.
[root@iqe-vm-arpandey insights-core]# ll /var/log/insights-client/
total 960
-rw-r--r--. 1 root root 517492 Jun  6 08:01 insights-client.log << logs not rotated
[root@iqe-vm-arpandey insights-core]#

lets rotate the logs forcefully - 

root@iqe-vm-arpandey insights-core]# logrotate -vf /etc/logrotate.d/insights-client 
reading config file /etc/logrotate.d/insights-client
Reading state from file: /var/lib/logrotate/logrotate.status
Allocating hash table for state file, size 64 entries
Creating new state
Creating new state
Creating new state
Creating new state
Creating new state
Creating new state
Creating new state
Creating new state
Creating new state
Creating new state
Creating new state
Creating new state
Creating new state
Creating new state
Creating new state

Handling 1 logs

rotating pattern: /var/log/insights-client/*.log  forced from command line (4 rotations)
empty log files are not rotated, old logs are removed
considering log /var/log/insights-client/insights-client.log
  Now: 2023-06-06 08:05
  Last rotated at 2023-06-04 03:11
  log needs rotating
rotating log /var/log/insights-client/insights-client.log, log->rotateCount is 4
dateext suffix '-20230606'
glob pattern '-[0-9][0-9][0-9][0-9][0-9][0-9][0-9][0-9]'
set default create context to unconfined_u:object_r:insights_client_var_log_t:s0
renaming /var/log/insights-client/insights-client.log.4 to /var/log/insights-client/insights-client.log.5 (rotatecount 4, logstart 1, i 4), 
old log /var/log/insights-client/insights-client.log.4 does not exist
renaming /var/log/insights-client/insights-client.log.3 to /var/log/insights-client/insights-client.log.4 (rotatecount 4, logstart 1, i 3), 
old log /var/log/insights-client/insights-client.log.3 does not exist
renaming /var/log/insights-client/insights-client.log.2 to /var/log/insights-client/insights-client.log.3 (rotatecount 4, logstart 1, i 2), 
old log /var/log/insights-client/insights-client.log.2 does not exist
renaming /var/log/insights-client/insights-client.log.1 to /var/log/insights-client/insights-client.log.2 (rotatecount 4, logstart 1, i 1), 
old log /var/log/insights-client/insights-client.log.1 does not exist
renaming /var/log/insights-client/insights-client.log.0 to /var/log/insights-client/insights-client.log.1 (rotatecount 4, logstart 1, i 0), 
old log /var/log/insights-client/insights-client.log.0 does not exist
log /var/log/insights-client/insights-client.log.5 doesn't exist -- won't try to dispose of it
copying /var/log/insights-client/insights-client.log to /var/log/insights-client/insights-client.log.1
set default create context to unconfined_u:object_r:insights_client_var_log_t:s0
truncating /var/log/insights-client/insights-client.log
set default create context to system_u:object_r:logrotate_var_lib_t:s0
[root@iqe-vm-arpandey insights-core]# ll /var/log/insights-client/
total 508
-rw-r--r--. 1 root root      0 Jun  6 08:05 insights-client.log
-rw-r--r--. 1 root root 517492 Jun  6 08:05 insights-client.log.1   <<<< logs rotated
[root@iqe-vm-arpandey insights-core]# 

Hence verified that log rotation feature is working as expected

--------------------------------------------------------------------------------------------

>>Case 2 -  older version of  insights-client and latest core
core : https://github.com/RedHatInsights/insights-core/pull/3765/
[root@iqe-vm-arpandey insights-core]# insights-client --version
Client: 3.1.7
Core: 3.0.8-dev


[root@iqe-vm-arpandey insights-core]# ll /etc/logrotate.d/insights-client
ls: cannot access '/etc/logrotate.d/insights-client': No such file or directory
[root@iqe-vm-arpandey insights-core]#

[root@iqe-vm-arpandey insights-core]# insights-client 
Starting to collect Insights data for iqe-vm-arpandey
Uploading Insights data.
Successfully uploaded report for iqe-vm-arpandey.
[root@iqe-vm-arpandey insights-core]# ll /var/log/insights-client/
total 460
-rw-r--r--. 1 root root 257985 Jun  6 07:52 insights-client.log
-rw-r--r--. 1 root root    189 Jun  6 07:52 insights-client.log.1
-rw-r--r--. 1 root root   5382 Jun  6 07:50 insights-client.log.2
[root@iqe-vm-arpandey insights-core]# insights-client 
Starting to collect Insights data for iqe-vm-arpandey
Uploading Insights data.
Successfully uploaded report for iqe-vm-arpandey.
[root@iqe-vm-arpandey insights-core]# ll /var/log/insights-client/
total 908
-rw-r--r--. 1 root root 258373 Jun  6 07:54 insights-client.log
-rw-r--r--. 1 root root 257985 Jun  6 07:52 insights-client.log.1
-rw-r--r--. 1 root root    189 Jun  6 07:52 insights-client.log.2
-rw-r--r--. 1 root root   5382 Jun  6 07:50 insights-client.log.3

Here verified that older logging method is being used because the insights client does not have logrotate implementation.
Setting verified tested.

Comment 12 Archana Pandey 2023-07-21 13:01:24 UTC

Final Verification on build insights-client-3.2.0-2.el8

Bug has been verified by running automated tests - https://main-jenkins-csb-insights-qe.apps.ocp-c1.prod.psi.redhat.com/job/rhc_and_insights_clients/job/main_rhc_insights/9019/testReport/csbprodrhel89.tests/test_client/

Few logs from jenkins job console: 
+ rpm -qa
+ grep insights-client
insights-client-3.2.0-2.el8.noarch

tests/test_client.py::test_verify_logrotate_is_configured <- ../../../../../../iqe_venv/lib64/python3.8/site-packages/iqe_insights_client/tests/test_client.py Content of log file is /var/log/insights-client/*.log {
    rotate 4
    weekly
    missingok
    notifempty
    copytruncate
}


-------------------------------- live log call ---------------------------------
2023-07-21 06:55:49 INFO Content of log file is /var/log/insights-client/*.log {
    rotate 4
    weekly
    missingok
    notifempty
    copytruncate
}

PASSED                                                                   [ 13%]
tests/test_client.py::test_verify_logrotate_when_forced_conf <- ../../../../../../iqe_venv/lib64/python3.8/site-packages/iqe_insights_client/tests/test_client.py 
-------------------------------- live log call ---------------------------------
2023-07-21 06:55:49 INFO log file size before test is 258606
2023-07-21 06:55:49 INFO number of log files before test is : 1
2023-07-21 06:55:49 INFO Starting to collect Insights data for rhiqe.2dc6297a-8188-4b24-988b-d5c43118ab4f.iqe-insights-client-plugin
2023-07-21 06:56:08 INFO Uploading Insights data.
2023-07-21 06:56:18 INFO Successfully uploaded report for rhiqe.2dc6297a-8188-4b24-988b-d5c43118ab4f.iqe-insights-client-plugin.
2023-07-21 06:56:18 INFO View the Red Hat Insights console at https://console.redhat.com/insights/
PASSED                                                                   [ 14%]
tests/test_client_options.py::test_group <- ../../../../../../iqe_venv/lib64/python3.8/site-packages/iqe_insights_client/tests/test_client_options.py Starting to collect Insights data for rhiqe.ddcce076-ebd3-49bc-8cdc-5d9b52b926ba.iqe-insights-client-plugin

-------------------------------- live log call ---------------------------------
2023-07-21 06:56:18 INFO Starting to collect Insights data for rhiqe.ddcce076-ebd3-49bc-8cdc-5d9b52b926ba.iqe-insights-client-plugin
PASSED                                                

Based on automated test results - Verification: Passed

Comment 14 errata-xmlrpc 2023-11-14 15:35:31 UTC

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (insights-client bug fix and enhancement update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2023:7023

Note You need to log in before you can comment on or make changes to this bug.