Bug 1851998
| Summary: | Prevent SELinux errors when rotating ansible-runner-service.log to allow logging of ARS and not exhaust space on RHV Manager | ||||||
|---|---|---|---|---|---|---|---|
| Product: | Red Hat Enterprise Virtualization Manager | Reporter: | David Vaanunu <dvaanunu> | ||||
| Component: | ansible-runner-service | Assignee: | Martin Necas <mnecas> | ||||
| Status: | CLOSED ERRATA | QA Contact: | Petr Matyáš <pmatyas> | ||||
| Severity: | high | Docs Contact: | |||||
| Priority: | high | ||||||
| Version: | 4.4.0 | CC: | aoconnor, bugs, dagur, dfodor, lleistne, mburman, michal.skrivanek, mlehrer, mnecas, mperina, mtessun, pmatyas | ||||
| Target Milestone: | ovirt-4.4.1-1 | Keywords: | Performance, Reopened | ||||
| Target Release: | --- | ||||||
| Hardware: | Unspecified | ||||||
| OS: | Unspecified | ||||||
| Whiteboard: | |||||||
| Fixed In Version: | ansible-runner-service-1.0.4 | Doc Type: | No Doc Update | ||||
| Doc Text: | Story Points: | --- | |||||
| Clone Of: | Environment: | ||||||
| Last Closed: | 2020-08-04 16:23:10 UTC | Type: | Bug | ||||
| Regression: | --- | Mount Type: | --- | ||||
| Documentation: | --- | CRM: | |||||
| Verified Versions: | Category: | --- | |||||
| oVirt Team: | Infra | RHEL 7.3 requirements from Atomic Host: | |||||
| Cloudforms Team: | --- | Target Upstream Version: | |||||
| Embargoed: | |||||||
| Bug Depends On: | |||||||
| Bug Blocks: | 1855959 | ||||||
| Attachments: |
|
||||||
The problem is with error_log file (/var/log/httpd) In scale lab the size was 9GB [root@rhev-red-01 ~]# ls -lh /var/log/httpd/ total 1.9G -rw-r--r--. 1 root root 16M Jun 30 03:49 access_log -rw-r--r--. 1 root root 1.8G Jun 29 13:17 error_log -rw-r--r--. 1 root root 36M Jun 30 03:49 ovirt-requests-log -rw-r--r--. 1 root root 24M Jun 30 03:49 ssl_access_log -rw-r--r--. 1 root root 21K Jun 28 11:54 ssl_error_log -rw-r--r--. 1 root root 28M Jun 30 03:49 ssl_request_log [root@rhev-red-01 ~]# (In reply to David Vaanunu from comment #2) > The problem is with error_log file (/var/log/httpd) > In scale lab the size was 9GB > > [root@rhev-red-01 ~]# ls -lh /var/log/httpd/ > total 1.9G > -rw-r--r--. 1 root root 16M Jun 30 03:49 access_log > -rw-r--r--. 1 root root 1.8G Jun 29 13:17 error_log > -rw-r--r--. 1 root root 36M Jun 30 03:49 ovirt-requests-log > -rw-r--r--. 1 root root 24M Jun 30 03:49 ssl_access_log > -rw-r--r--. 1 root root 21K Jun 28 11:54 ssl_error_log > -rw-r--r--. 1 root root 28M Jun 30 03:49 ssl_request_log > [root@rhev-red-01 ~]# error_log is not part of engine, but part of httpd system package. The logrotate configuration for httpd is stored in /etc/logrotate.d/httpd So if default configuration doesn't suite your needs, then you need to adapt this configuration manually. This is not part of engine, but part of standard OS administrator role. ansible-runner-service-1.0.2-1.el8ev.noarch I was not able to reproduce this error but I did more investigation and noticed that the log file was removed after the rotate (but no error :/ ). And another issue was that the logging was not configured properly for logrotate. Both fixes I have added to the patch hopefully it will solve your issue. I also removed the logging to the httpd logs so all ansible stuff will go to /var/log/ovirt-engine/ansible-runner-service.log and if there will be some issue when setting up the WSGI or apache we would still get the error in /var/log/httpd/error_log Does the /var/log/ovirt-engine/ansible-runner-service.log exist? (it is possible it was removed previously by the logrotate) Have you restarted the service? it's not that hard to answer that yourself.... [root@she-test-03 ~]# ls -lZ /var/log/ovirt-engine/ansible-runner-service.log -rw-r--r--. 1 ovirt ovirt system_u:object_r:var_log_t:s0 683008 Jul 13 11:07 /var/log/ovirt-engine/ansible-runner-service.log [root@she-test-03 ~]# rpm -q ansible-runner-service ansible-runner-service-1.0.4-1.el8ev.noarch Exactly as Michal said, I had the correct version. It does happen after restart of httpd, the file was created with wrong context in the past, which needs to be solved by package update. Can be workarounded by deleting the log file. As Petr has done the validation steps here already, no need for scale teams involvement. Moving back to Petr with agreement from Luci and Petr. To make the state clear, moving back as per comment 12 and discussion with Martin. This is not a blocker for GA anymore, ansible-runner-service-1.0.4 fixed permissions issue for both flows: new RHV 4.4 installation and upgrade from RHV 4.3. We will fix the permission issue happeding when upgrading ARS < 1.0.4 to >= 1.0.4 in the new ARS 1.0.5, but it will be available only in 4.4.2, but this is a flow which cannot happen now in a real flow Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory (RHV Manager (ovirt-engine) 4.4 (0-day)), and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHBA-2020:3317 |
Created attachment 1699185 [details] httpd error_log Description of problem: /var/log/httpd/error_log file have error: PermissionError: [Errno 13] Permission denied: '/var/log/ovirt-engine/ansible-runner-service.log' -> '/var /log/ovirt-engine/ansible-runner-service.log.1' The log file is increased and cause to partition becomes full (/var/log). When the partition is full no log rotation. [root@rhev-red-03 httpd]# ls -lh /var/log/ovirt-engine total 84M drwx------. 2 ovirt ovirt 4.0K Jun 21 14:37 ansible -rw-r--r--. 1 ovirt ovirt 10M Jun 26 14:50 ansible-runner-service.log -rw-r--r--. 1 ovirt ovirt 5.4K Jun 21 14:34 boot.log drwx------. 2 ovirt ovirt 6 Jun 3 11:49 brick-setup drwx------. 2 ovirt ovirt 6 Jun 3 11:49 cinderlib -rw-r--r--. 1 ovirt ovirt 1.7K Jun 21 14:34 console.log Version-Release number of selected component (if applicable): rhv-release-4.4.1-3-001.noarch redhat-release-eula-8.2-1.0.el8.x86_64 How reproducible: Steps to Reproduce: 1. Check if a partition is full (df -h) 2. verify httpd error_log 3. Actual results: Have permission error to file have no log rotation Expected results: Have permission to file No Errors and have a rotation Additional info: