Bug 1339224
| Summary: | RabbitMQ logs are not rotated properly | |||
|---|---|---|---|---|
| Product: | Red Hat OpenStack | Reporter: | Robin Cernin <rcernin> | |
| Component: | rabbitmq-server | Assignee: | Peter Lemenkov <plemenko> | |
| Status: | CLOSED ERRATA | QA Contact: | Asaf Hirshberg <ahirshbe> | |
| Severity: | medium | Docs Contact: | ||
| Priority: | medium | |||
| Version: | 7.0 (Kilo) | CC: | apevec, djuran, ipetrova, jbuchta, jeckersb, lhh, pbarta, plemenko, srevivo | |
| Target Milestone: | --- | Keywords: | ZStream | |
| Target Release: | 8.0 (Liberty) | Flags: | ipetrova:
needinfo+
|
|
| Hardware: | Unspecified | |||
| OS: | Unspecified | |||
| Whiteboard: | ||||
| Fixed In Version: | rabbitmq-server-3.3.5-27.el7ost | Doc Type: | If docs needed, set a value | |
| Doc Text: | Story Points: | --- | ||
| Clone Of: | ||||
| : | 1371521 1405211 (view as bug list) | Environment: | ||
| Last Closed: | 2016-11-14 19:57:20 UTC | Type: | Bug | |
| Regression: | --- | Mount Type: | --- | |
| Documentation: | --- | CRM: | ||
| Verified Versions: | Category: | --- | ||
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | ||
| Cloudforms Team: | --- | Target Upstream Version: | ||
| Embargoed: | ||||
| Bug Depends On: | ||||
| Bug Blocks: | 1371521, 1405211, 1462144, 1673827 | |||
The same problem can also be observed in OSP 8, with rabbitmq-server-3.3.5-22.el7ost. To reproduce, just deploy an OpenStack using OSP-d and then leave to matute for a week.... Cannot confirm this. II've just tried to rotate logs with /usr/sbin/logrotate /etc/logrotate.conf And everything was rotated just fine (rabbitmq continues with a new log and no PID is holding older log). I've also had prblems reproducing this when I manually try to trigger the log rotation. If I however deploy an OverCloud from OSP-d 8 and then let it sit for a week, this problem does occur every time. Hello All! Please try this build ( rabbitmq-server-3.3.5-26.el7ost ) - it fixes related issues in logging subsystem and very likely fixes this issue as well. (In reply to Peter Lemenkov from comment #11) > Hello All! > > Please try this build ( rabbitmq-server-3.3.5-26.el7ost ) - it fixes related > issues in logging subsystem and very likely fixes this issue as well. Hey Peter!! :) I don't think that's just available yet. At least I don't see it in our Portal: https://access.redhat.com/downloads/content/rabbitmq-server/3.3.5-22.el7ost/noarch/fd431d51/package Do you have any info about that? Best, Irina PS Many thanks for taking a look. Verified, OSPD-8 using rabbitmq-server-3.3.5-27.el7ost.noarch
[root@controller-2 ~]# fuser -v /var/log/rabbitmq/*
USER PID ACCESS COMMAND
/var/log/rabbitmq/rabbit:
rabbitmq 20819 F.... beam.smp
/var/log/rabbitmq/rabbit:
rabbitmq 20819 F.... beam.smp
/var/log/rabbitmq/startup_err:
root 20794 F.... rabbitmq-server
root 20811 F.... su
rabbitmq 20819 F.... beam.smp
rabbitmq 20954 F.... inet_gethost
rabbitmq 20955 F.... inet_gethost
/var/log/rabbitmq/startup_log:
root 20794 F.... rabbitmq-server
root 20811 F.... su
rabbitmq 20819 F.... beam.smp
[root@controller-2 rabbitmq]# rabbitmqctl rotate_logs .1
Rotating logs to files with suffix ".1" ...
...done.
[root@controller-2 rabbitmq]# ll
total 236
-rw-r-----. 1 rabbitmq rabbitmq 0 Oct 9 04:11 rabbit
-rw-r-----. 1 rabbitmq rabbitmq 43304 Oct 9 04:11 rabbit.1
-rw-r-----. 1 rabbitmq rabbitmq 189340 Oct 6 16:41 rabbit
-rw-r-----. 1 rabbitmq rabbitmq 0 Oct 9 04:11 rabbit
-rw-r-----. 1 rabbitmq rabbitmq 0 Oct 9 04:11 rabbit.1
-rw-r-----. 1 root root 0 Oct 9 03:36 startup_err
-rw-r-----. 1 root root 698 Oct 9 03:36 startup_log
[root@controller-2 rabbitmq]# fuser -v /var/log/rabbitmq/*
USER PID ACCESS COMMAND
/var/log/rabbitmq/rabbit:
rabbitmq 20819 F.... beam.smp
/var/log/rabbitmq/rabbit:
rabbitmq 20819 F.... beam.smp
/var/log/rabbitmq/startup_err:
root 20794 F.... rabbitmq-server
root 20811 F.... su
rabbitmq 20819 F.... beam.smp
rabbitmq 20954 F.... inet_gethost
rabbitmq 20955 F.... inet_gethost
/var/log/rabbitmq/startup_log:
root 20794 F.... rabbitmq-server
root 20811 F.... su
rabbitmq 20819 F.... beam.smp
[root@controller-2 rabbitmq]# rabbitmqctl rotate_logs .2
Rotating logs to files with suffix ".2" ...
...done.
[root@controller-2 rabbitmq]# fuser -v /var/log/rabbitmq/*
USER PID ACCESS COMMAND
/var/log/rabbitmq/rabbit:
rabbitmq 20819 F.... beam.smp
/var/log/rabbitmq/rabbit:
rabbitmq 20819 F.... beam.smp
/var/log/rabbitmq/startup_err:
root 20794 F.... rabbitmq-server
root 20811 F.... su
rabbitmq 20819 F.... beam.smp
rabbitmq 20954 F.... inet_gethost
rabbitmq 20955 F.... inet_gethost
/var/log/rabbitmq/startup_log:
root 20794 F.... rabbitmq-server
root 20811 F.... su
rabbitmq 20819 F.... beam.smp
[root@controller-2 rabbitmq]# ll
total 240
-rw-r-----. 1 rabbitmq rabbitmq 0 Oct 9 04:13 rabbit
-rw-r-----. 1 rabbitmq rabbitmq 43304 Oct 9 04:11 rabbit.1
-rw-r-----. 1 rabbitmq rabbitmq 74 Oct 9 04:13 rabbit.2
-rw-r-----. 1 rabbitmq rabbitmq 189340 Oct 6 16:41 rabbit
-rw-r-----. 1 rabbitmq rabbitmq 0 Oct 9 04:13 rabbit
-rw-r-----. 1 rabbitmq rabbitmq 0 Oct 9 04:11 rabbit.1
-rw-r-----. 1 rabbitmq rabbitmq 0 Oct 9 04:13 rabbit.2
-rw-r-----. 1 root root 0 Oct 9 03:36 startup_err
-rw-r-----. 1 root root 698 Oct 9 03:36 startup_log
[root@controller-2 rabbitmq]#
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://rhn.redhat.com/errata/RHBA-2016-2712.html |
The rabbitmq-server logs are not rotated properly /var/log/rabbitmq/*.log { weekly missingok rotate 20 compress delaycompress notifempty sharedscripts postrotate /usr/sbin/rabbitmqctl -q rotate_logs endscript } After the rotation using fuser shows the rabbitmq is still using the rabbit [root@overcloud-controller-0 ~]# fuser -v /var/log/rabbitmq/* USER PID ACCESS COMMAND /var/log/rabbitmq/rabbit: rabbitmq 22869 F.... beam.smp /var/log/rabbitmq/rabbit: rabbitmq 22869 F.... beam.smp /var/log/rabbitmq/startup_err: root 22842 F.... rabbitmq-server root 22867 F.... su rabbitmq 22869 F.... beam.smp rabbitmq 23245 F.... inet_gethost rabbitmq 23246 F.... inet_gethost /var/log/rabbitmq/startup_log: root 22842 F.... rabbitmq-server root 22867 F.... su rabbitmq 22869 F.... beam.smp Version-Release number of selected component (if applicable): rabbitmq-server-3.3.5-16.el7ost.noarch How reproducible: always Steps to Reproduce: 1. wait until logs are rotated 2. 3. Actual results: Wrong file is use /var/log/rabbitmq/rabbit Expected results: Correct file in use /var/log/rabbitmq/rabbit