Bug 1940267

Summary: insights-client change log management to logrotate standard
Product: Red Hat Enterprise Linux 8 Reporter: Bryan Kinney <bkinney>
Component: insights-clientAssignee: Jason Jerome <jajerome>
Status: CLOSED ERRATA QA Contact: Pavol Kotvan <pakotvan>
Severity: medium Docs Contact:
Priority: medium    
Version: 8.9CC: ahitacat, arpandey, cmarinea, fjansen, klape, link, lphiri, pakotvan, peter.vreman, ptoscano, robwilli
Target Milestone: rcKeywords: Triaged
Target Release: ---Flags: pm-rhel: mirror+
Hardware: All   
OS: Linux   
Whiteboard:
Fixed In Version: insights-client-3.2.0-2.el8 Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of:
: 2189281 (view as bug list) Environment:
Last Closed: 2023-11-14 15:35:31 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 2189281    

Description Bryan Kinney 2021-03-18 00:56:00 UTC 
Description of problem:

The current logging method used by the insights-client presents several problems.
  - When any change to logging is required a bug needs to be filed (Example: bug 1734283)
  - Changes or new requirements create new issues (Example: bug 1923835)
  - Use of certain parameters force changes to be made (Example: bug 1734283)

It is believed the use of a mature system like logrotate will alleviate most of those problems.
There are many solutions for logrotate and it provides a flowing history that can be kept as the user sees fit.

Version-Release number of selected component (if applicable):


How reproducible:
N/A

Steps to Reproduce:
1.
2.
3.

Actual results:
Log features and rotation require internal changes to insights-client

Expected results:
Use the logrotate system to handle all logging in the insights-client

Additional info:
Comment 8 Archana Pandey 2023-06-06 12:17:05 UTC 
pre-verification -
>> Case 1: latest core and  scratch build of insights-client

using core https://github.com/RedHatInsights/insights-core/pull/3765/

[root@iqe-vm-arpandey insights-core]# insights-client --version
Client: 3.2.0
Core: 3.0.8-dev

[root@iqe-vm-arpandey insights-core]# rpm -qa | grep insights-client
insights-client-3.2.0.git.424.5dfefcf7-0.el8.noarch
[root@iqe-vm-arpandey insights-core]# 

[root@iqe-vm-arpandey insights-core]# cat /etc/logrotate.d/insights-client
/var/log/insights-client/*.log {
    rotate 4
    weekly
    missingok
    notifempty
    sharedscripts
    copytruncate
    create 0644 root root
}                                         ^^^^^------------ conf file for insights-client logrotate feature installed
[root@iqe-vm-arpandey insights-core]#

running insights-client to test if logs are getting rotated 'weekly'

[root@iqe-vm-arpandey insights-core]# insights-client
Starting to collect Insights data for iqe-vm-arpandey
Uploading Insights data.
Successfully uploaded report for iqe-vm-arpandey.
[root@iqe-vm-arpandey insights-core]# ll /var/log/insights-client/
total 448
-rw-r--r--. 1 root root 258746 Jun  6 08:00 insights-client.log <<< logs not rotated
[root@iqe-vm-arpandey insights-core]# insights-client
Starting to collect Insights data for iqe-vm-arpandey
Uploading Insights data.
Successfully uploaded report for iqe-vm-arpandey.
[root@iqe-vm-arpandey insights-core]# ll /var/log/insights-client/
total 960
-rw-r--r--. 1 root root 517492 Jun  6 08:01 insights-client.log << logs not rotated
[root@iqe-vm-arpandey insights-core]#

lets rotate the logs forcefully - 

root@iqe-vm-arpandey insights-core]# logrotate -vf /etc/logrotate.d/insights-client 
reading config file /etc/logrotate.d/insights-client
Reading state from file: /var/lib/logrotate/logrotate.status
Allocating hash table for state file, size 64 entries
Creating new state
Creating new state
Creating new state
Creating new state
Creating new state
Creating new state
Creating new state
Creating new state
Creating new state
Creating new state
Creating new state
Creating new state
Creating new state
Creating new state
Creating new state

Handling 1 logs

rotating pattern: /var/log/insights-client/*.log  forced from command line (4 rotations)
empty log files are not rotated, old logs are removed
considering log /var/log/insights-client/insights-client.log
  Now: 2023-06-06 08:05
  Last rotated at 2023-06-04 03:11
  log needs rotating
rotating log /var/log/insights-client/insights-client.log, log->rotateCount is 4
dateext suffix '-20230606'
glob pattern '-[0-9][0-9][0-9][0-9][0-9][0-9][0-9][0-9]'
set default create context to unconfined_u:object_r:insights_client_var_log_t:s0
renaming /var/log/insights-client/insights-client.log.4 to /var/log/insights-client/insights-client.log.5 (rotatecount 4, logstart 1, i 4), 
old log /var/log/insights-client/insights-client.log.4 does not exist
renaming /var/log/insights-client/insights-client.log.3 to /var/log/insights-client/insights-client.log.4 (rotatecount 4, logstart 1, i 3), 
old log /var/log/insights-client/insights-client.log.3 does not exist
renaming /var/log/insights-client/insights-client.log.2 to /var/log/insights-client/insights-client.log.3 (rotatecount 4, logstart 1, i 2), 
old log /var/log/insights-client/insights-client.log.2 does not exist
renaming /var/log/insights-client/insights-client.log.1 to /var/log/insights-client/insights-client.log.2 (rotatecount 4, logstart 1, i 1), 
old log /var/log/insights-client/insights-client.log.1 does not exist
renaming /var/log/insights-client/insights-client.log.0 to /var/log/insights-client/insights-client.log.1 (rotatecount 4, logstart 1, i 0), 
old log /var/log/insights-client/insights-client.log.0 does not exist
log /var/log/insights-client/insights-client.log.5 doesn't exist -- won't try to dispose of it
copying /var/log/insights-client/insights-client.log to /var/log/insights-client/insights-client.log.1
set default create context to unconfined_u:object_r:insights_client_var_log_t:s0
truncating /var/log/insights-client/insights-client.log
set default create context to system_u:object_r:logrotate_var_lib_t:s0
[root@iqe-vm-arpandey insights-core]# ll /var/log/insights-client/
total 508
-rw-r--r--. 1 root root      0 Jun  6 08:05 insights-client.log
-rw-r--r--. 1 root root 517492 Jun  6 08:05 insights-client.log.1   <<<< logs rotated
[root@iqe-vm-arpandey insights-core]# 

Hence verified that log rotation feature is working as expected

--------------------------------------------------------------------------------------------

>>Case 2 -  older version of  insights-client and latest core
core : https://github.com/RedHatInsights/insights-core/pull/3765/
[root@iqe-vm-arpandey insights-core]# insights-client --version
Client: 3.1.7
Core: 3.0.8-dev


[root@iqe-vm-arpandey insights-core]# ll /etc/logrotate.d/insights-client
ls: cannot access '/etc/logrotate.d/insights-client': No such file or directory
[root@iqe-vm-arpandey insights-core]#

[root@iqe-vm-arpandey insights-core]# insights-client 
Starting to collect Insights data for iqe-vm-arpandey
Uploading Insights data.
Successfully uploaded report for iqe-vm-arpandey.
[root@iqe-vm-arpandey insights-core]# ll /var/log/insights-client/
total 460
-rw-r--r--. 1 root root 257985 Jun  6 07:52 insights-client.log
-rw-r--r--. 1 root root    189 Jun  6 07:52 insights-client.log.1
-rw-r--r--. 1 root root   5382 Jun  6 07:50 insights-client.log.2
[root@iqe-vm-arpandey insights-core]# insights-client 
Starting to collect Insights data for iqe-vm-arpandey
Uploading Insights data.
Successfully uploaded report for iqe-vm-arpandey.
[root@iqe-vm-arpandey insights-core]# ll /var/log/insights-client/
total 908
-rw-r--r--. 1 root root 258373 Jun  6 07:54 insights-client.log
-rw-r--r--. 1 root root 257985 Jun  6 07:52 insights-client.log.1
-rw-r--r--. 1 root root    189 Jun  6 07:52 insights-client.log.2
-rw-r--r--. 1 root root   5382 Jun  6 07:50 insights-client.log.3

Here verified that older logging method is being used because the insights client does not have logrotate implementation.
Setting verified tested.
Comment 12 Archana Pandey 2023-07-21 13:01:24 UTC 
Final Verification on build insights-client-3.2.0-2.el8

Bug has been verified by running automated tests - https://main-jenkins-csb-insights-qe.apps.ocp-c1.prod.psi.redhat.com/job/rhc_and_insights_clients/job/main_rhc_insights/9019/testReport/csbprodrhel89.tests/test_client/

Few logs from jenkins job console: 
+ rpm -qa
+ grep insights-client
insights-client-3.2.0-2.el8.noarch

tests/test_client.py::test_verify_logrotate_is_configured <- ../../../../../../iqe_venv/lib64/python3.8/site-packages/iqe_insights_client/tests/test_client.py Content of log file is /var/log/insights-client/*.log {
    rotate 4
    weekly
    missingok
    notifempty
    copytruncate
}


-------------------------------- live log call ---------------------------------
2023-07-21 06:55:49 INFO Content of log file is /var/log/insights-client/*.log {
    rotate 4
    weekly
    missingok
    notifempty
    copytruncate
}

PASSED                                                                   [ 13%]
tests/test_client.py::test_verify_logrotate_when_forced_conf <- ../../../../../../iqe_venv/lib64/python3.8/site-packages/iqe_insights_client/tests/test_client.py 
-------------------------------- live log call ---------------------------------
2023-07-21 06:55:49 INFO log file size before test is 258606
2023-07-21 06:55:49 INFO number of log files before test is : 1
2023-07-21 06:55:49 INFO Starting to collect Insights data for rhiqe.2dc6297a-8188-4b24-988b-d5c43118ab4f.iqe-insights-client-plugin
2023-07-21 06:56:08 INFO Uploading Insights data.
2023-07-21 06:56:18 INFO Successfully uploaded report for rhiqe.2dc6297a-8188-4b24-988b-d5c43118ab4f.iqe-insights-client-plugin.
2023-07-21 06:56:18 INFO View the Red Hat Insights console at https://console.redhat.com/insights/
PASSED                                                                   [ 14%]
tests/test_client_options.py::test_group <- ../../../../../../iqe_venv/lib64/python3.8/site-packages/iqe_insights_client/tests/test_client_options.py Starting to collect Insights data for rhiqe.ddcce076-ebd3-49bc-8cdc-5d9b52b926ba.iqe-insights-client-plugin

-------------------------------- live log call ---------------------------------
2023-07-21 06:56:18 INFO Starting to collect Insights data for rhiqe.ddcce076-ebd3-49bc-8cdc-5d9b52b926ba.iqe-insights-client-plugin
PASSED                                                

Based on automated test results - Verification: Passed
Comment 14 errata-xmlrpc 2023-11-14 15:35:31 UTC 
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (insights-client bug fix and enhancement update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2023:7023