Hello, logratating does not work for candlepin in Satellite 6.3. Although it might look like it's doing something, it is not working correctly. Log rotation must happen everyday on midnight, instead logs are only created when application is restarted. What's the problem is inability of reopening files when they are rotated - Candlepin carries on writing logs into deleted/renamed file until it is restarted. Then the correct file is appended again. This is a big problem when debugging customer calls as we end up with files of big sizes in our debug report or sos report. Currently (Satellite 6.3) candlepin uses log4j/logstash with simple FileAppender and there is no way of telling it to reopen logs. TRIAGE NOTES: Please do backport for us, create some simple way of telling the log4j stack to reopen logs (SIGHUP or HTTP endpoint) and use that in /etc/logrotate.d/candlepin file. Make sure it works under SELinux (logrotate runs confined and often cause problems). SIDENOTE: Long term I recommend to consider using syslog/journad for all logs. This way, rotation is solved problem as it is provided by logging daemon "for free" (plus it enables sending logs over network - planned feature for future Satellite version). If you choose to use syslog for
If you choose to use syslog for the future, this enables Satellite 6 to continue the "common logging" initiative - all logs in one place with correlation ids for better investigation of errors. (Finished incomplete sentence, thanks for looking into this.)
Lzap, Just for testing sake, what version of logrotate is your Satellite instance using? We have some conditional code depending on the logrotate version. I'm not sure why this is happening. I thought that the `copytruncate` option in logrotate was meant to allow for a process to continue writing to the same file. Would you mind posting your Candlepin logrotate configuration from /etc/logrotate.d?
Alex, [root@satellite1 ~]# rpm -q logrotate logrotate-3.8.6-14.el7.x86_64 [root@satellite1 ~]# cat /etc/logrotate.d/candlepin #1343753: cant use * here cause not all logs in this folder will be owned by tomcat /var/log/candlepin/access.log /var/log/candlepin/audit.log /var/log/candlepin/candlepin.log /var/log/candlepin/error.log { # logrotate 3.8 requires the su directive, # where as prior versions do not recognize it. su tomcat tomcat copytruncate daily rotate 52 compress missingok create 0644 tomcat tomcat } This is our older 6.2 instance that hasn't been restarted some time. I see candlepin.log not rotated, but this is system that is idling. I've seen the same from customer sos report when this was very same situation, but the file was 2 GB. -rw-r--r--. 1 tomcat tomcat 12013536 Jan 12 09:00 candlepin.log -rw-r--r--. 1 tomcat tomcat 6665 Jun 21 2017 candlepin.log-20170621.gz -rw-r--r--. 1 tomcat tomcat 9388 Jun 22 2017 candlepin.log-20170622.gz -rw-r--r--. 1 tomcat tomcat 5148 Jun 23 2017 candlepin.log-20170623.gz -rw-r--r--. 1 tomcat tomcat 10186 Jun 24 2017 candlepin.log-20170624.gz -rw-r--r--. 1 tomcat tomcat 4780 Jun 25 2017 candlepin.log-20170625.gz -rw-r--r--. 1 tomcat tomcat 4914 Jun 26 2017 candlepin.log-20170626.gz -rw-r--r--. 1 tomcat tomcat 4886 Jun 27 2017 candlepin.log-20170627.gz -rw-r--r--. 1 tomcat tomcat 5454 Jun 28 2017 candlepin.log-20170628.gz -rw-r--r--. 1 tomcat tomcat 5032 Jun 29 2017 candlepin.log-20170629.gz -rw-r--r--. 1 tomcat tomcat 6782 Jun 30 2017 candlepin.log-20170630.gz -rw-r--r--. 1 tomcat tomcat 5042 Jul 1 2017 candlepin.log-20170701.gz -rw-r--r--. 1 tomcat tomcat 5144 Jul 2 2017 candlepin.log-20170702.gz -rw-r--r--. 1 tomcat tomcat 4777 Jul 3 2017 candlepin.log-20170703.gz -rw-r--r--. 1 tomcat tomcat 4858 Jul 4 2017 candlepin.log-20170704.gz -rw-r--r--. 1 tomcat tomcat 4917 Jul 5 2017 candlepin.log-20170705.gz -rw-r--r--. 1 tomcat tomcat 7904 Jul 6 2017 candlepin.log-20170706.gz -rw-r--r--. 1 tomcat tomcat 6391 Jul 7 2017 candlepin.log-20170707.gz -rw-r--r--. 1 tomcat tomcat 5920 Jul 8 2017 candlepin.log-20170708.gz -rw-r--r--. 1 tomcat tomcat 4885 Jul 9 2017 candlepin.log-20170709.gz -rw-r--r--. 1 tomcat tomcat 5214 Jul 10 2017 candlepin.log-20170710.gz -rw-r--r--. 1 tomcat tomcat 4703 Jul 11 2017 candlepin.log-20170711.gz -rw-r--r--. 1 tomcat tomcat 5190 Jul 12 2017 candlepin.log-20170712.gz -rw-r--r--. 1 tomcat tomcat 40687 Jul 13 2017 candlepin.log-20170713.gz -rw-r--r--. 1 tomcat tomcat 5039 Jul 14 03:11 candlepin.log-20170714.gz -rw-r--r--. 1 tomcat tomcat 5109 Jul 15 03:11 candlepin.log-20170715.gz -rw-r--r--. 1 tomcat tomcat 4855 Jul 16 03:11 candlepin.log-20170716.gz -rw-r--r--. 1 tomcat tomcat 4890 Jul 17 03:11 candlepin.log-20170717.gz -rw-r--r--. 1 tomcat tomcat 4904 Jul 18 03:11 candlepin.log-20170718.gz -rw-r--r--. 1 tomcat tomcat 4969 Jul 19 03:41 candlepin.log-20170719.gz -rw-r--r--. 1 tomcat tomcat 8719 Jul 20 03:41 candlepin.log-20170720.gz -rw-r--r--. 1 tomcat tomcat 4792 Jul 21 03:11 candlepin.log-20170721.gz -rw-r--r--. 1 tomcat tomcat 5105 Jul 22 03:11 candlepin.log-20170722.gz -rw-r--r--. 1 tomcat tomcat 4974 Jul 23 03:11 candlepin.log-20170723.gz -rw-r--r--. 1 tomcat tomcat 4835 Jul 24 03:11 candlepin.log-20170724.gz -rw-r--r--. 1 tomcat tomcat 4914 Jul 25 03:11 candlepin.log-20170725.gz -rw-r--r--. 1 tomcat tomcat 4905 Jul 26 03:00 candlepin.log-20170726.gz -rw-r--r--. 1 tomcat tomcat 4968 Jul 27 03:11 candlepin.log-20170727.gz -rw-r--r--. 1 tomcat tomcat 4948 Jul 28 03:17 candlepin.log-20170728.gz -rw-r--r--. 1 tomcat tomcat 5181 Jul 29 03:41 candlepin.log-20170729.gz -rw-r--r--. 1 tomcat tomcat 4853 Jul 30 03:24 candlepin.log-20170730.gz -rw-r--r--. 1 tomcat tomcat 4868 Jul 31 03:11 candlepin.log-20170731.gz -rw-r--r--. 1 tomcat tomcat 6838 Aug 1 03:13 candlepin.log-20170801.gz -rw-r--r--. 1 tomcat tomcat 5535 Aug 2 03:11 candlepin.log-20170802.gz -rw-r--r--. 1 tomcat tomcat 6704 Aug 3 03:11 candlepin.log-20170803.gz -rw-r--r--. 1 tomcat tomcat 4969 Aug 4 03:12 candlepin.log-20170804.gz -rw-r--r--. 1 tomcat tomcat 5176 Aug 5 03:22 candlepin.log-20170805.gz -rw-r--r--. 1 tomcat tomcat 4898 Aug 6 03:11 candlepin.log-20170806.gz -rw-r--r--. 1 tomcat tomcat 4929 Aug 7 03:12 candlepin.log-20170807.gz -rw-r--r--. 1 tomcat tomcat 4885 Aug 8 03:11 candlepin.log-20170808.gz -rw-r--r--. 1 tomcat tomcat 4883 Aug 9 03:00 candlepin.log-20170809.gz -rw-r--r--. 1 tomcat tomcat 5005 Aug 10 03:11 candlepin.log-20170810.gz -rw-r--r--. 1 tomcat tomcat 4974 Aug 11 03:19 candlepin.log-20170811.gz
If you log the output of logrotate, does it have any errors or info that may shed some light on this? I've tried doing a few test setups to rotate some logs using a similar configuration to what has been outlined above, but it worked as expected for me. The only time I got similar behavior to what you're seeing is when the copy operation wasn't able to be completed. In that case, it just left the existing log data where it was, didn't truncate and continued forward. I'd be interested to see if there's a similar issue happening in your environment.
Peter, I struggle to reproduce the misbehavior on both Satellite 6.2.13+ and 6.3. Can you confirm this is no longer the case? I only saw your paste of candlepin.log not being rotated, but according to all configuration of candlepin and logrotate, it should now work correctly.
Hoi Lukas, On 6.2.13 i still have the issue because of SELinux, see https://bugzilla.redhat.com/show_bug.cgi?id=1473349 [crash/LI] root@li-lc-1578:~# ls -l /var/log/candlepin/ total 468224 -rw-r--r--. 1 tomcat tomcat 420323 Jan 18 22:27 audit.log -rw-r--r--. 1 tomcat tomcat 327 Jun 23 2017 audit.log-20170624.gz -rw-r--r--. 1 tomcat tomcat 20 Jun 24 2017 audit.log-20170625.gz -rw-r--r--. 1 tomcat tomcat 168 Jun 25 2017 audit.log-20170626.gz -rw-r--r--. 1 tomcat tomcat 492 Jun 26 2017 audit.log-20170627.gz -rw-r--r--. 1 tomcat tomcat 377 Jun 27 2017 audit.log-20170628.gz -rw-r--r--. 1 tomcat tomcat 172 Jun 28 2017 audit.log-20170629.gz -rw-r--r--. 1 tomcat tomcat 20 Jun 29 2017 audit.log-20170630.gz -rw-r--r--. 1 tomcat tomcat 20 Jun 30 2017 audit.log-20170701.gz -rw-r--r--. 1 tomcat tomcat 20 Jul 1 2017 audit.log-20170702.gz -rw-r--r--. 1 tomcat tomcat 171 Jul 2 2017 audit.log-20170703.gz -rw-r--r--. 1 tomcat tomcat 20 Jul 3 2017 audit.log-20170704.gz -rw-r--r--. 1 tomcat tomcat 20 Jul 4 2017 audit.log-20170705.gz -rw-r--r--. 1 tomcat tomcat 20 Jul 5 2017 audit.log-20170706.gz -rw-r--r--. 1 tomcat tomcat 1632 Jul 6 2017 audit.log-20170707.gz -rw-r--r--. 1 tomcat tomcat 259 Jul 7 2017 audit.log-20170708.gz -rw-r--r--. 1 tomcat tomcat 20 Jul 8 2017 audit.log-20170709.gz -rw-r--r--. 1 tomcat tomcat 189 Jul 9 2017 audit.log-20170710.gz -rw-r--r--. 1 tomcat tomcat 167 Jul 10 2017 audit.log-20170711.gz -rw-r--r--. 1 tomcat tomcat 20 Jul 11 2017 audit.log-20170712.gz -rw-r--r--. 1 tomcat tomcat 172 Jul 12 2017 audit.log-20170713.gz -rw-r--r--. 1 tomcat tomcat 172 Jul 13 2017 audit.log-20170714.gz -rw-r--r--. 1 tomcat tomcat 220 Jul 14 2017 audit.log-20170715.gz -rw-r--r--. 1 tomcat tomcat 20 Jul 15 2017 audit.log-20170716.gz -rw-r--r--. 1 tomcat tomcat 189 Jul 16 2017 audit.log-20170717.gz -rw-r--r--. 1 tomcat tomcat 20 Jul 17 2017 audit.log-20170718.gz -rw-r--r--. 1 tomcat tomcat 344 Jul 18 2017 audit.log-20170719.gz -rw-r--r--. 1 tomcat tomcat 311 Jul 19 2017 audit.log-20170720.gz -rw-r--r--. 1 tomcat tomcat 206 Jul 20 2017 audit.log-20170721.gz -rw-r--r--. 1 tomcat tomcat 290 Jul 21 2017 audit.log-20170722.gz -rw-r--r--. 1 tomcat tomcat 20 Jul 22 2017 audit.log-20170723.gz -rw-r--r--. 1 tomcat tomcat 189 Jul 23 2017 audit.log-20170724.gz -rw-r--r--. 1 tomcat tomcat 241 Jul 24 2017 audit.log-20170725.gz -rw-r--r--. 1 tomcat tomcat 205 Jul 25 2017 audit.log-20170726.gz -rw-r--r--. 1 tomcat tomcat 210 Jul 26 2017 audit.log-20170727.gz -rw-r--r--. 1 tomcat tomcat 265 Jul 27 2017 audit.log-20170728.gz -rw-r--r--. 1 tomcat tomcat 234 Jul 28 2017 audit.log-20170729.gz -rw-r--r--. 1 tomcat tomcat 20 Jul 29 2017 audit.log-20170730.gz -rw-r--r--. 1 tomcat tomcat 189 Jul 30 2017 audit.log-20170731.gz -rw-r--r--. 1 tomcat tomcat 20 Jul 31 03:10 audit.log-20170801.gz -rw-r--r--. 1 tomcat tomcat 213 Aug 1 15:04 audit.log-20170802.gz -rw-r--r--. 1 tomcat tomcat 20 Aug 2 03:38 audit.log-20170803.gz -rw-r--r--. 1 tomcat tomcat 247 Aug 3 19:07 audit.log-20170804.gz -rw-r--r--. 1 tomcat tomcat 206 Aug 4 12:04 audit.log-20170805.gz -rw-r--r--. 1 tomcat tomcat 20 Aug 5 03:44 audit.log-20170806.gz -rw-r--r--. 1 tomcat tomcat 189 Aug 6 21:03 audit.log-20170807.gz -rw-r--r--. 1 tomcat tomcat 370 Aug 7 09:34 audit.log-20170808.gz -rw-r--r--. 1 tomcat tomcat 220 Aug 8 07:13 audit.log-20170809.gz -rw-r--r--. 1 tomcat tomcat 20 Aug 9 03:50 audit.log-20170810.gz -rw-r--r--. 1 tomcat tomcat 20 Aug 10 03:41 audit.log-20170811.gz -rw-r--r--. 1 tomcat tomcat 20 Aug 11 03:21 audit.log-20170812.gz -rw-r--r--. 1 tomcat tomcat 20 Aug 12 03:23 audit.log-20170813.gz -rw-r--r--. 1 tomcat tomcat 210 Aug 13 21:34 audit.log-20170814.gz -rw-r--r--. 1 tomcat tomcat 459454167 Jan 29 16:14 candlepin.log -rw-r--r--. 1 tomcat tomcat 282749 Jun 24 2017 candlepin.log-20170624.gz -rw-r--r--. 1 tomcat tomcat 248289 Jun 25 2017 candlepin.log-20170625.gz -rw-r--r--. 1 tomcat tomcat 261195 Jun 26 2017 candlepin.log-20170626.gz -rw-r--r--. 1 tomcat tomcat 304978 Jun 27 2017 candlepin.log-20170627.gz -rw-r--r--. 1 tomcat tomcat 255087 Jun 28 2017 candlepin.log-20170628.gz -rw-r--r--. 1 tomcat tomcat 253585 Jun 29 2017 candlepin.log-20170629.gz -rw-r--r--. 1 tomcat tomcat 259813 Jun 30 2017 candlepin.log-20170630.gz -rw-r--r--. 1 tomcat tomcat 262405 Jul 1 2017 candlepin.log-20170701.gz -rw-r--r--. 1 tomcat tomcat 242611 Jul 2 2017 candlepin.log-20170702.gz -rw-r--r--. 1 tomcat tomcat 260931 Jul 3 2017 candlepin.log-20170703.gz -rw-r--r--. 1 tomcat tomcat 245882 Jul 4 2017 candlepin.log-20170704.gz -rw-r--r--. 1 tomcat tomcat 244210 Jul 5 2017 candlepin.log-20170705.gz -rw-r--r--. 1 tomcat tomcat 246794 Jul 6 2017 candlepin.log-20170706.gz -rw-r--r--. 1 tomcat tomcat 401013 Jul 7 2017 candlepin.log-20170707.gz -rw-r--r--. 1 tomcat tomcat 388577 Jul 8 2017 candlepin.log-20170708.gz -rw-r--r--. 1 tomcat tomcat 330269 Jul 9 2017 candlepin.log-20170709.gz -rw-r--r--. 1 tomcat tomcat 352786 Jul 10 2017 candlepin.log-20170710.gz -rw-r--r--. 1 tomcat tomcat 355448 Jul 11 2017 candlepin.log-20170711.gz -rw-r--r--. 1 tomcat tomcat 328828 Jul 12 2017 candlepin.log-20170712.gz -rw-r--r--. 1 tomcat tomcat 344562 Jul 13 2017 candlepin.log-20170713.gz -rw-r--r--. 1 tomcat tomcat 370603 Jul 14 2017 candlepin.log-20170714.gz -rw-r--r--. 1 tomcat tomcat 360731 Jul 15 2017 candlepin.log-20170715.gz -rw-r--r--. 1 tomcat tomcat 326711 Jul 16 2017 candlepin.log-20170716.gz -rw-r--r--. 1 tomcat tomcat 348382 Jul 17 2017 candlepin.log-20170717.gz -rw-r--r--. 1 tomcat tomcat 329749 Jul 18 2017 candlepin.log-20170718.gz -rw-r--r--. 1 tomcat tomcat 465965 Jul 19 2017 candlepin.log-20170719.gz -rw-r--r--. 1 tomcat tomcat 425030 Jul 20 2017 candlepin.log-20170720.gz -rw-r--r--. 1 tomcat tomcat 353814 Jul 21 2017 candlepin.log-20170721.gz -rw-r--r--. 1 tomcat tomcat 429077 Jul 22 2017 candlepin.log-20170722.gz -rw-r--r--. 1 tomcat tomcat 315601 Jul 23 2017 candlepin.log-20170723.gz -rw-r--r--. 1 tomcat tomcat 332148 Jul 24 2017 candlepin.log-20170724.gz -rw-r--r--. 1 tomcat tomcat 367593 Jul 25 2017 candlepin.log-20170725.gz -rw-r--r--. 1 tomcat tomcat 351381 Jul 26 2017 candlepin.log-20170726.gz -rw-r--r--. 1 tomcat tomcat 157019 Jul 27 2017 candlepin.log-20170727.gz -rw-r--r--. 1 tomcat tomcat 587342 Jul 28 2017 candlepin.log-20170728.gz -rw-r--r--. 1 tomcat tomcat 375769 Jul 29 2017 candlepin.log-20170729.gz -rw-r--r--. 1 tomcat tomcat 315214 Jul 30 2017 candlepin.log-20170730.gz -rw-r--r--. 1 tomcat tomcat 331053 Jul 31 03:10 candlepin.log-20170731.gz -rw-r--r--. 1 tomcat tomcat 321764 Aug 1 03:22 candlepin.log-20170801.gz -rw-r--r--. 1 tomcat tomcat 320743 Aug 2 03:37 candlepin.log-20170802.gz -rw-r--r--. 1 tomcat tomcat 320166 Aug 3 03:25 candlepin.log-20170803.gz -rw-r--r--. 1 tomcat tomcat 394208 Aug 4 03:13 candlepin.log-20170804.gz -rw-r--r--. 1 tomcat tomcat 171339 Aug 5 03:37 candlepin.log-20170805.gz -rw-r--r--. 1 tomcat tomcat 133370 Aug 6 03:37 candlepin.log-20170806.gz -rw-r--r--. 1 tomcat tomcat 145686 Aug 7 03:13 candlepin.log-20170807.gz -rw-r--r--. 1 tomcat tomcat 401255 Aug 8 03:16 candlepin.log-20170808.gz -rw-r--r--. 1 tomcat tomcat 355885 Aug 9 03:42 candlepin.log-20170809.gz -rw-r--r--. 1 tomcat tomcat 307807 Aug 10 03:28 candlepin.log-20170810.gz -rw-r--r--. 1 tomcat tomcat 318463 Aug 11 03:14 candlepin.log-20170811.gz -rw-r--r--. 1 tomcat tomcat 318277 Aug 12 03:13 candlepin.log-20170812.gz -rw-r--r--. 1 tomcat tomcat 306145 Aug 13 03:07 candlepin.log-20170813.gz -rw-r--r--. 1 tomcat tomcat 329404 Aug 14 03:36 candlepin.log-20170814.gz -rw-r--r--. 1 root root 28 Oct 14 2016 cpdb.log -rw-r--r--. 1 root root 20 Oct 14 2016 cpinit.log -rw-r--r--. 1 tomcat tomcat 1526961 Jan 19 12:49 error.log -rw-r--r--. 1 tomcat tomcat 20 Jun 23 2017 error.log-20170624.gz -rw-r--r--. 1 tomcat tomcat 20 Jun 24 2017 error.log-20170625.gz -rw-r--r--. 1 tomcat tomcat 20 Jun 25 2017 error.log-20170626.gz -rw-r--r--. 1 tomcat tomcat 20 Jun 26 2017 error.log-20170627.gz -rw-r--r--. 1 tomcat tomcat 20 Jun 27 2017 error.log-20170628.gz -rw-r--r--. 1 tomcat tomcat 20 Jun 28 2017 error.log-20170629.gz -rw-r--r--. 1 tomcat tomcat 20 Jun 29 2017 error.log-20170630.gz -rw-r--r--. 1 tomcat tomcat 20 Jun 30 2017 error.log-20170701.gz -rw-r--r--. 1 tomcat tomcat 20 Jul 1 2017 error.log-20170702.gz -rw-r--r--. 1 tomcat tomcat 20 Jul 2 2017 error.log-20170703.gz -rw-r--r--. 1 tomcat tomcat 20 Jul 3 2017 error.log-20170704.gz -rw-r--r--. 1 tomcat tomcat 20 Jul 4 2017 error.log-20170705.gz -rw-r--r--. 1 tomcat tomcat 20 Jul 5 2017 error.log-20170706.gz -rw-r--r--. 1 tomcat tomcat 513 Jul 6 2017 error.log-20170707.gz -rw-r--r--. 1 tomcat tomcat 20 Jul 7 2017 error.log-20170708.gz -rw-r--r--. 1 tomcat tomcat 20 Jul 8 2017 error.log-20170709.gz -rw-r--r--. 1 tomcat tomcat 20 Jul 9 2017 error.log-20170710.gz -rw-r--r--. 1 tomcat tomcat 20 Jul 10 2017 error.log-20170711.gz -rw-r--r--. 1 tomcat tomcat 20 Jul 11 2017 error.log-20170712.gz -rw-r--r--. 1 tomcat tomcat 20 Jul 12 2017 error.log-20170713.gz -rw-r--r--. 1 tomcat tomcat 20 Jul 13 2017 error.log-20170714.gz -rw-r--r--. 1 tomcat tomcat 20 Jul 14 2017 error.log-20170715.gz -rw-r--r--. 1 tomcat tomcat 20 Jul 15 2017 error.log-20170716.gz -rw-r--r--. 1 tomcat tomcat 20 Jul 16 2017 error.log-20170717.gz -rw-r--r--. 1 tomcat tomcat 20 Jul 17 2017 error.log-20170718.gz -rw-r--r--. 1 tomcat tomcat 20 Jul 18 2017 error.log-20170719.gz -rw-r--r--. 1 tomcat tomcat 20 Jul 19 2017 error.log-20170720.gz -rw-r--r--. 1 tomcat tomcat 20 Jul 20 2017 error.log-20170721.gz -rw-r--r--. 1 tomcat tomcat 20 Jul 21 2017 error.log-20170722.gz -rw-r--r--. 1 tomcat tomcat 20 Jul 22 2017 error.log-20170723.gz -rw-r--r--. 1 tomcat tomcat 20 Jul 23 2017 error.log-20170724.gz -rw-r--r--. 1 tomcat tomcat 2132 Jul 24 2017 error.log-20170725.gz -rw-r--r--. 1 tomcat tomcat 20 Jul 25 2017 error.log-20170726.gz -rw-r--r--. 1 tomcat tomcat 20 Jul 26 2017 error.log-20170727.gz -rw-r--r--. 1 tomcat tomcat 20 Jul 27 2017 error.log-20170728.gz -rw-r--r--. 1 tomcat tomcat 20 Jul 28 2017 error.log-20170729.gz -rw-r--r--. 1 tomcat tomcat 20 Jul 29 2017 error.log-20170730.gz -rw-r--r--. 1 tomcat tomcat 20 Jul 30 2017 error.log-20170731.gz -rw-r--r--. 1 tomcat tomcat 20 Jul 31 03:10 error.log-20170801.gz -rw-r--r--. 1 tomcat tomcat 316 Aug 1 15:04 error.log-20170802.gz -rw-r--r--. 1 tomcat tomcat 20 Aug 2 03:38 error.log-20170803.gz -rw-r--r--. 1 tomcat tomcat 20 Aug 3 03:29 error.log-20170804.gz -rw-r--r--. 1 tomcat tomcat 20 Aug 4 03:26 error.log-20170805.gz -rw-r--r--. 1 tomcat tomcat 20 Aug 5 03:44 error.log-20170806.gz -rw-r--r--. 1 tomcat tomcat 20 Aug 6 03:46 error.log-20170807.gz -rw-r--r--. 1 tomcat tomcat 4994 Aug 7 09:02 error.log-20170808.gz -rw-r--r--. 1 tomcat tomcat 20 Aug 8 03:16 error.log-20170809.gz -rw-r--r--. 1 tomcat tomcat 20 Aug 9 03:50 error.log-20170810.gz -rw-r--r--. 1 tomcat tomcat 20 Aug 10 03:41 error.log-20170811.gz -rw-r--r--. 1 tomcat tomcat 20 Aug 11 03:21 error.log-20170812.gz -rw-r--r--. 1 tomcat tomcat 20 Aug 12 03:23 error.log-20170813.gz -rw-r--r--. 1 tomcat tomcat 20 Aug 13 03:07 error.log-20170814.gz From timing you can see the the issue also started in Aug-2017, similar to the listing above in https://bugzilla.redhat.com/show_bug.cgi?id=1520357#c5 I currently do not have running 6.3Beta installation to verify it there. Peter
Peter, Would you mind looking at the output of `ls -alZ /var/log/candlepin`? If you see any SELinux context besides "system_u:object_r:var_log_t:s0" please post it. Any context besides that indicates that you're probably running into Bug #1473349. The bug has been fixed in recent versions of Candlepin, but if you don't want to upgrade, the process below should fix the problem in the Candlepin policy module. # yum install selinux-policy-devel (This may not be the exact directory name depending on your specific version) # cd /usr/share/doc/candlepin-selinux-0.9.54.23/ (If candlepin.pp isn't present, that's okay too) # cp -p candlepin.pp candlepin.pp.orig # sed -ie 's/^files_type\((candlepin_var_log_t)\)$/logging_log_file\1/' candlepin.te (No results should be returned from this grep command) # grep '(candlepin_var_log_t)' candlepin.te # make NAME=targeted -f /usr/share/selinux/devel/Makefile # semodule -i candlepin.pp
The candlepin logs have the more restrictive candlepin_var_log vrempet@li-lc-1578 ~ $ ls -alZ /var/log/candlepin drwxrwxr-x. tomcat root system_u:object_r:candlepin_var_log_t:s0 . drwxr-xr-x. root root system_u:object_r:var_log_t:s0 .. -rw-r--r--. tomcat tomcat system_u:object_r:candlepin_var_log_t:s0 audit.log -rw-r--r--. tomcat tomcat system_u:object_r:candlepin_var_log_t:s0 audit.log-20170624.gz -rw-r--r--. tomcat tomcat system_u:object_r:candlepin_var_log_t:s0 audit.log-20170625.gz -rw-r--r--. tomcat tomcat system_u:object_r:candlepin_var_log_t:s0 audit.log-20170626.gz -rw-r--r--. tomcat tomcat system_u:object_r:candlepin_var_log_t:s0 audit.log-20170627.gz -rw-r--r--. tomcat tomcat system_u:object_r:candlepin_var_log_t:s0 audit.log-20170628.gz -rw-r--r--. tomcat tomcat system_u:object_r:candlepin_var_log_t:s0 audit.log-20170629.gz -rw-r--r--. tomcat tomcat system_u:object_r:candlepin_var_log_t:s0 audit.log-20170630.gz -rw-r--r--. tomcat tomcat system_u:object_r:candlepin_var_log_t:s0 audit.log-20170701.gz -rw-r--r--. tomcat tomcat system_u:object_r:candlepin_var_log_t:s0 audit.log-20170702.gz -rw-r--r--. tomcat tomcat system_u:object_r:candlepin_var_log_t:s0 audit.log-20170703.gz -rw-r--r--. tomcat tomcat system_u:object_r:candlepin_var_log_t:s0 audit.log-20170704.gz -rw-r--r--. tomcat tomcat system_u:object_r:candlepin_var_log_t:s0 audit.log-20170705.gz -rw-r--r--. tomcat tomcat system_u:object_r:candlepin_var_log_t:s0 audit.log-20170706.gz -rw-r--r--. tomcat tomcat system_u:object_r:candlepin_var_log_t:s0 audit.log-20170707.gz -rw-r--r--. tomcat tomcat system_u:object_r:candlepin_var_log_t:s0 audit.log-20170708.gz -rw-r--r--. tomcat tomcat system_u:object_r:candlepin_var_log_t:s0 audit.log-20170709.gz -rw-r--r--. tomcat tomcat system_u:object_r:candlepin_var_log_t:s0 audit.log-20170710.gz -rw-r--r--. tomcat tomcat system_u:object_r:candlepin_var_log_t:s0 audit.log-20170711.gz -rw-r--r--. tomcat tomcat system_u:object_r:candlepin_var_log_t:s0 audit.log-20170712.gz -rw-r--r--. tomcat tomcat system_u:object_r:candlepin_var_log_t:s0 audit.log-20170713.gz -rw-r--r--. tomcat tomcat system_u:object_r:candlepin_var_log_t:s0 audit.log-20170714.gz -rw-r--r--. tomcat tomcat system_u:object_r:candlepin_var_log_t:s0 audit.log-20170715.gz -rw-r--r--. tomcat tomcat system_u:object_r:candlepin_var_log_t:s0 audit.log-20170716.gz -rw-r--r--. tomcat tomcat system_u:object_r:candlepin_var_log_t:s0 audit.log-20170717.gz -rw-r--r--. tomcat tomcat system_u:object_r:candlepin_var_log_t:s0 audit.log-20170718.gz -rw-r--r--. tomcat tomcat system_u:object_r:candlepin_var_log_t:s0 audit.log-20170719.gz -rw-r--r--. tomcat tomcat system_u:object_r:candlepin_var_log_t:s0 audit.log-20170720.gz -rw-r--r--. tomcat tomcat system_u:object_r:candlepin_var_log_t:s0 audit.log-20170721.gz -rw-r--r--. tomcat tomcat system_u:object_r:candlepin_var_log_t:s0 audit.log-20170722.gz -rw-r--r--. tomcat tomcat system_u:object_r:candlepin_var_log_t:s0 audit.log-20170723.gz -rw-r--r--. tomcat tomcat system_u:object_r:candlepin_var_log_t:s0 audit.log-20170724.gz -rw-r--r--. tomcat tomcat system_u:object_r:candlepin_var_log_t:s0 audit.log-20170725.gz -rw-r--r--. tomcat tomcat system_u:object_r:candlepin_var_log_t:s0 audit.log-20170726.gz -rw-r--r--. tomcat tomcat system_u:object_r:candlepin_var_log_t:s0 audit.log-20170727.gz -rw-r--r--. tomcat tomcat system_u:object_r:candlepin_var_log_t:s0 audit.log-20170728.gz -rw-r--r--. tomcat tomcat system_u:object_r:candlepin_var_log_t:s0 audit.log-20170729.gz -rw-r--r--. tomcat tomcat system_u:object_r:candlepin_var_log_t:s0 audit.log-20170730.gz -rw-r--r--. tomcat tomcat system_u:object_r:candlepin_var_log_t:s0 audit.log-20170731.gz -rw-r--r--. tomcat tomcat system_u:object_r:candlepin_var_log_t:s0 audit.log-20170801.gz -rw-r--r--. tomcat tomcat system_u:object_r:candlepin_var_log_t:s0 audit.log-20170802.gz -rw-r--r--. tomcat tomcat system_u:object_r:candlepin_var_log_t:s0 audit.log-20170803.gz -rw-r--r--. tomcat tomcat system_u:object_r:candlepin_var_log_t:s0 audit.log-20170804.gz -rw-r--r--. tomcat tomcat system_u:object_r:candlepin_var_log_t:s0 audit.log-20170805.gz -rw-r--r--. tomcat tomcat system_u:object_r:candlepin_var_log_t:s0 audit.log-20170806.gz -rw-r--r--. tomcat tomcat system_u:object_r:candlepin_var_log_t:s0 audit.log-20170807.gz -rw-r--r--. tomcat tomcat system_u:object_r:candlepin_var_log_t:s0 audit.log-20170808.gz -rw-r--r--. tomcat tomcat system_u:object_r:candlepin_var_log_t:s0 audit.log-20170809.gz -rw-r--r--. tomcat tomcat system_u:object_r:candlepin_var_log_t:s0 audit.log-20170810.gz -rw-r--r--. tomcat tomcat system_u:object_r:candlepin_var_log_t:s0 audit.log-20170811.gz -rw-r--r--. tomcat tomcat system_u:object_r:candlepin_var_log_t:s0 audit.log-20170812.gz -rw-r--r--. tomcat tomcat system_u:object_r:candlepin_var_log_t:s0 audit.log-20170813.gz -rw-r--r--. tomcat tomcat system_u:object_r:candlepin_var_log_t:s0 audit.log-20170814.gz -rw-r--r--. tomcat tomcat system_u:object_r:candlepin_var_log_t:s0 candlepin.log -rw-r--r--. tomcat tomcat system_u:object_r:candlepin_var_log_t:s0 candlepin.log-20170624.gz -rw-r--r--. tomcat tomcat system_u:object_r:candlepin_var_log_t:s0 candlepin.log-20170625.gz -rw-r--r--. tomcat tomcat system_u:object_r:candlepin_var_log_t:s0 candlepin.log-20170626.gz -rw-r--r--. tomcat tomcat system_u:object_r:candlepin_var_log_t:s0 candlepin.log-20170627.gz -rw-r--r--. tomcat tomcat system_u:object_r:candlepin_var_log_t:s0 candlepin.log-20170628.gz -rw-r--r--. tomcat tomcat system_u:object_r:candlepin_var_log_t:s0 candlepin.log-20170629.gz -rw-r--r--. tomcat tomcat system_u:object_r:candlepin_var_log_t:s0 candlepin.log-20170630.gz -rw-r--r--. tomcat tomcat system_u:object_r:candlepin_var_log_t:s0 candlepin.log-20170701.gz -rw-r--r--. tomcat tomcat system_u:object_r:candlepin_var_log_t:s0 candlepin.log-20170702.gz -rw-r--r--. tomcat tomcat system_u:object_r:candlepin_var_log_t:s0 candlepin.log-20170703.gz -rw-r--r--. tomcat tomcat system_u:object_r:candlepin_var_log_t:s0 candlepin.log-20170704.gz -rw-r--r--. tomcat tomcat system_u:object_r:candlepin_var_log_t:s0 candlepin.log-20170705.gz -rw-r--r--. tomcat tomcat system_u:object_r:candlepin_var_log_t:s0 candlepin.log-20170706.gz -rw-r--r--. tomcat tomcat system_u:object_r:candlepin_var_log_t:s0 candlepin.log-20170707.gz -rw-r--r--. tomcat tomcat system_u:object_r:candlepin_var_log_t:s0 candlepin.log-20170708.gz -rw-r--r--. tomcat tomcat system_u:object_r:candlepin_var_log_t:s0 candlepin.log-20170709.gz -rw-r--r--. tomcat tomcat system_u:object_r:candlepin_var_log_t:s0 candlepin.log-20170710.gz -rw-r--r--. tomcat tomcat system_u:object_r:candlepin_var_log_t:s0 candlepin.log-20170711.gz -rw-r--r--. tomcat tomcat system_u:object_r:candlepin_var_log_t:s0 candlepin.log-20170712.gz -rw-r--r--. tomcat tomcat system_u:object_r:candlepin_var_log_t:s0 candlepin.log-20170713.gz -rw-r--r--. tomcat tomcat system_u:object_r:candlepin_var_log_t:s0 candlepin.log-20170714.gz -rw-r--r--. tomcat tomcat system_u:object_r:candlepin_var_log_t:s0 candlepin.log-20170715.gz -rw-r--r--. tomcat tomcat system_u:object_r:candlepin_var_log_t:s0 candlepin.log-20170716.gz -rw-r--r--. tomcat tomcat system_u:object_r:candlepin_var_log_t:s0 candlepin.log-20170717.gz -rw-r--r--. tomcat tomcat system_u:object_r:candlepin_var_log_t:s0 candlepin.log-20170718.gz -rw-r--r--. tomcat tomcat system_u:object_r:candlepin_var_log_t:s0 candlepin.log-20170719.gz -rw-r--r--. tomcat tomcat system_u:object_r:candlepin_var_log_t:s0 candlepin.log-20170720.gz -rw-r--r--. tomcat tomcat system_u:object_r:candlepin_var_log_t:s0 candlepin.log-20170721.gz -rw-r--r--. tomcat tomcat system_u:object_r:candlepin_var_log_t:s0 candlepin.log-20170722.gz -rw-r--r--. tomcat tomcat system_u:object_r:candlepin_var_log_t:s0 candlepin.log-20170723.gz -rw-r--r--. tomcat tomcat system_u:object_r:candlepin_var_log_t:s0 candlepin.log-20170724.gz -rw-r--r--. tomcat tomcat system_u:object_r:candlepin_var_log_t:s0 candlepin.log-20170725.gz -rw-r--r--. tomcat tomcat system_u:object_r:candlepin_var_log_t:s0 candlepin.log-20170726.gz -rw-r--r--. tomcat tomcat system_u:object_r:candlepin_var_log_t:s0 candlepin.log-20170727.gz -rw-r--r--. tomcat tomcat system_u:object_r:candlepin_var_log_t:s0 candlepin.log-20170728.gz -rw-r--r--. tomcat tomcat system_u:object_r:candlepin_var_log_t:s0 candlepin.log-20170729.gz -rw-r--r--. tomcat tomcat system_u:object_r:candlepin_var_log_t:s0 candlepin.log-20170730.gz -rw-r--r--. tomcat tomcat system_u:object_r:candlepin_var_log_t:s0 candlepin.log-20170731.gz -rw-r--r--. tomcat tomcat system_u:object_r:candlepin_var_log_t:s0 candlepin.log-20170801.gz -rw-r--r--. tomcat tomcat system_u:object_r:candlepin_var_log_t:s0 candlepin.log-20170802.gz -rw-r--r--. tomcat tomcat system_u:object_r:candlepin_var_log_t:s0 candlepin.log-20170803.gz -rw-r--r--. tomcat tomcat system_u:object_r:candlepin_var_log_t:s0 candlepin.log-20170804.gz -rw-r--r--. tomcat tomcat system_u:object_r:candlepin_var_log_t:s0 candlepin.log-20170805.gz -rw-r--r--. tomcat tomcat system_u:object_r:candlepin_var_log_t:s0 candlepin.log-20170806.gz -rw-r--r--. tomcat tomcat system_u:object_r:candlepin_var_log_t:s0 candlepin.log-20170807.gz -rw-r--r--. tomcat tomcat system_u:object_r:candlepin_var_log_t:s0 candlepin.log-20170808.gz -rw-r--r--. tomcat tomcat system_u:object_r:candlepin_var_log_t:s0 candlepin.log-20170809.gz -rw-r--r--. tomcat tomcat system_u:object_r:candlepin_var_log_t:s0 candlepin.log-20170810.gz -rw-r--r--. tomcat tomcat system_u:object_r:candlepin_var_log_t:s0 candlepin.log-20170811.gz -rw-r--r--. tomcat tomcat system_u:object_r:candlepin_var_log_t:s0 candlepin.log-20170812.gz -rw-r--r--. tomcat tomcat system_u:object_r:candlepin_var_log_t:s0 candlepin.log-20170813.gz -rw-r--r--. tomcat tomcat system_u:object_r:candlepin_var_log_t:s0 candlepin.log-20170814.gz -rw-r--r--. root root unconfined_u:object_r:candlepin_var_log_t:s0 cpdb.log -rw-r--r--. root root unconfined_u:object_r:candlepin_var_log_t:s0 cpinit.log -rw-r--r--. tomcat tomcat system_u:object_r:candlepin_var_log_t:s0 error.log -rw-r--r--. tomcat tomcat system_u:object_r:candlepin_var_log_t:s0 error.log-20170624.gz -rw-r--r--. tomcat tomcat system_u:object_r:candlepin_var_log_t:s0 error.log-20170625.gz -rw-r--r--. tomcat tomcat system_u:object_r:candlepin_var_log_t:s0 error.log-20170626.gz -rw-r--r--. tomcat tomcat system_u:object_r:candlepin_var_log_t:s0 error.log-20170627.gz -rw-r--r--. tomcat tomcat system_u:object_r:candlepin_var_log_t:s0 error.log-20170628.gz -rw-r--r--. tomcat tomcat system_u:object_r:candlepin_var_log_t:s0 error.log-20170629.gz -rw-r--r--. tomcat tomcat system_u:object_r:candlepin_var_log_t:s0 error.log-20170630.gz -rw-r--r--. tomcat tomcat system_u:object_r:candlepin_var_log_t:s0 error.log-20170701.gz -rw-r--r--. tomcat tomcat system_u:object_r:candlepin_var_log_t:s0 error.log-20170702.gz -rw-r--r--. tomcat tomcat system_u:object_r:candlepin_var_log_t:s0 error.log-20170703.gz -rw-r--r--. tomcat tomcat system_u:object_r:candlepin_var_log_t:s0 error.log-20170704.gz -rw-r--r--. tomcat tomcat system_u:object_r:candlepin_var_log_t:s0 error.log-20170705.gz -rw-r--r--. tomcat tomcat system_u:object_r:candlepin_var_log_t:s0 error.log-20170706.gz -rw-r--r--. tomcat tomcat system_u:object_r:candlepin_var_log_t:s0 error.log-20170707.gz -rw-r--r--. tomcat tomcat system_u:object_r:candlepin_var_log_t:s0 error.log-20170708.gz -rw-r--r--. tomcat tomcat system_u:object_r:candlepin_var_log_t:s0 error.log-20170709.gz -rw-r--r--. tomcat tomcat system_u:object_r:candlepin_var_log_t:s0 error.log-20170710.gz -rw-r--r--. tomcat tomcat system_u:object_r:candlepin_var_log_t:s0 error.log-20170711.gz -rw-r--r--. tomcat tomcat system_u:object_r:candlepin_var_log_t:s0 error.log-20170712.gz -rw-r--r--. tomcat tomcat system_u:object_r:candlepin_var_log_t:s0 error.log-20170713.gz -rw-r--r--. tomcat tomcat system_u:object_r:candlepin_var_log_t:s0 error.log-20170714.gz -rw-r--r--. tomcat tomcat system_u:object_r:candlepin_var_log_t:s0 error.log-20170715.gz -rw-r--r--. tomcat tomcat system_u:object_r:candlepin_var_log_t:s0 error.log-20170716.gz -rw-r--r--. tomcat tomcat system_u:object_r:candlepin_var_log_t:s0 error.log-20170717.gz -rw-r--r--. tomcat tomcat system_u:object_r:candlepin_var_log_t:s0 error.log-20170718.gz -rw-r--r--. tomcat tomcat system_u:object_r:candlepin_var_log_t:s0 error.log-20170719.gz -rw-r--r--. tomcat tomcat system_u:object_r:candlepin_var_log_t:s0 error.log-20170720.gz -rw-r--r--. tomcat tomcat system_u:object_r:candlepin_var_log_t:s0 error.log-20170721.gz -rw-r--r--. tomcat tomcat system_u:object_r:candlepin_var_log_t:s0 error.log-20170722.gz -rw-r--r--. tomcat tomcat system_u:object_r:candlepin_var_log_t:s0 error.log-20170723.gz -rw-r--r--. tomcat tomcat system_u:object_r:candlepin_var_log_t:s0 error.log-20170724.gz -rw-r--r--. tomcat tomcat system_u:object_r:candlepin_var_log_t:s0 error.log-20170725.gz -rw-r--r--. tomcat tomcat system_u:object_r:candlepin_var_log_t:s0 error.log-20170726.gz -rw-r--r--. tomcat tomcat system_u:object_r:candlepin_var_log_t:s0 error.log-20170727.gz -rw-r--r--. tomcat tomcat system_u:object_r:candlepin_var_log_t:s0 error.log-20170728.gz -rw-r--r--. tomcat tomcat system_u:object_r:candlepin_var_log_t:s0 error.log-20170729.gz -rw-r--r--. tomcat tomcat system_u:object_r:candlepin_var_log_t:s0 error.log-20170730.gz -rw-r--r--. tomcat tomcat system_u:object_r:candlepin_var_log_t:s0 error.log-20170731.gz -rw-r--r--. tomcat tomcat system_u:object_r:candlepin_var_log_t:s0 error.log-20170801.gz -rw-r--r--. tomcat tomcat system_u:object_r:candlepin_var_log_t:s0 error.log-20170802.gz -rw-r--r--. tomcat tomcat system_u:object_r:candlepin_var_log_t:s0 error.log-20170803.gz -rw-r--r--. tomcat tomcat system_u:object_r:candlepin_var_log_t:s0 error.log-20170804.gz -rw-r--r--. tomcat tomcat system_u:object_r:candlepin_var_log_t:s0 error.log-20170805.gz -rw-r--r--. tomcat tomcat system_u:object_r:candlepin_var_log_t:s0 error.log-20170806.gz -rw-r--r--. tomcat tomcat system_u:object_r:candlepin_var_log_t:s0 error.log-20170807.gz -rw-r--r--. tomcat tomcat system_u:object_r:candlepin_var_log_t:s0 error.log-20170808.gz -rw-r--r--. tomcat tomcat system_u:object_r:candlepin_var_log_t:s0 error.log-20170809.gz -rw-r--r--. tomcat tomcat system_u:object_r:candlepin_var_log_t:s0 error.log-20170810.gz -rw-r--r--. tomcat tomcat system_u:object_r:candlepin_var_log_t:s0 error.log-20170811.gz -rw-r--r--. tomcat tomcat system_u:object_r:candlepin_var_log_t:s0 error.log-20170812.gz -rw-r--r--. tomcat tomcat system_u:object_r:candlepin_var_log_t:s0 error.log-20170813.gz -rw-r--r--. tomcat tomcat system_u:object_r:candlepin_var_log_t:s0 error.log-20170814.gz
I think we drifting of here from the initial description of case that was about 6.3 candlepin logs not being correct. As Sat6.3 uses a newer candlepin using different logging libraries. Sadly i have no 6.3 to test, so i cannot confirm the behaviour of Sat6.3 (upgraded from 6.2, neither fresh installed)
Peter, Version mismatch aside, it looks like your logs have the wrong context; they should be "..:var_log_t" rather than "..:candlepin_var_log_t". The steps outlined in comment #9 should resolve the issue you're seeing. *** This bug has been marked as a duplicate of bug 1473349 ***