Bug 1462144 - RabbitMQ logs are not rotated properly
RabbitMQ logs are not rotated properly
Status: ASSIGNED
Product: Red Hat OpenStack
Classification: Red Hat
Component: rabbitmq-server (Show other bugs)
7.0 (Kilo)
Unspecified Unspecified
medium Severity medium
: ---
: 7.0 (Kilo)
Assigned To: Peter Lemenkov
Udi Shkalim
: Reopened, TestOnly, Triaged, ZStream
Depends On: 1339224 1405211
Blocks: 1371521
  Show dependency treegraph
 
Reported: 2017-06-16 06:09 EDT by Pablo Iranzo Gómez
Modified: 2018-01-13 22:03 EST (History)
15 users (show)

See Also:
Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: 1405211
Environment:
Last Closed:
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)


External Trackers
Tracker ID Priority Status Summary Last Updated
Launchpad 1473405 None None None 2017-06-16 06:09 EDT
Github fedora-selinux/selinux-policy-contrib/pull/15 None None None 2017-10-02 09:48 EDT
Github rabbitmq/rabbitmq-server/pull/543 None None None 2017-10-02 09:49 EDT

  None (edit)
Description Pablo Iranzo Gómez 2017-06-16 06:09:54 EDT
Original bug:


The rabbitmq-server logs are not rotated properly

/var/log/rabbitmq/*.log {
        weekly
        missingok
        rotate 20
        compress
        delaycompress
        notifempty
        sharedscripts
        postrotate
            /usr/sbin/rabbitmqctl -q rotate_logs
        endscript
}

After the rotation using fuser shows the rabbitmq is still using the rabbit@overcloud-controller-0.log-20160522 

[root@overcloud-controller-0 ~]# fuser -v /var/log/rabbitmq/*
                     USER        PID ACCESS COMMAND
/var/log/rabbitmq/rabbit@overcloud-controller-0.log-20160522:
                     rabbitmq  22869 F.... beam.smp
/var/log/rabbitmq/rabbit@overcloud-controller-0-sasl.log:
                     rabbitmq  22869 F.... beam.smp
/var/log/rabbitmq/startup_err:
                     root      22842 F.... rabbitmq-server
                     root      22867 F.... su
                     rabbitmq  22869 F.... beam.smp
                     rabbitmq  23245 F.... inet_gethost
                     rabbitmq  23246 F.... inet_gethost
/var/log/rabbitmq/startup_log:
                     root      22842 F.... rabbitmq-server
                     root      22867 F.... su
                     rabbitmq  22869 F.... beam.smp

After the errata on masterbug (1405211) there is still issues:


Rabbit MQ version: 3.3.5-31

Status:
Logrorate hasn't changed and still process using the rotated files instead of new ones:


~~~~
rabbitmq  9516  9510  4 May13 ?        1-01:09:08 /usr/lib64/erlang/erts-5.10.4/bin/beam.smp -W w -K true -A30 -P 1048576 -- -root /usr/lib64/erlang -progname erl -- -home /var/lib/rabbitmq -- -pa /usr/lib/rabbitmq/lib/rabbitmq_server-3.3.5/sbin/../ebin -noshell -noinput -s rabbit boot -sname rabbit@esah-ost1-cc01p -boot start_sasl -config /etc/rabbitmq/rabbitmq -kernel inet_default_connect_options [{nodelay,true},{raw,6,18,<<5000:64/native>>}] -kernel inet_default_listen_options [{raw,6,18,<<5000:64/native>>}] -rabbit tcp_listeners [{"172.24.13.16",5672}] -sasl errlog_type error -sasl sasl_error_logger false -rabbit error_logger {file,"/var/log/rabbitmq/rabbit@esah-ost1-cc01p.log"} -rabbit sasl_error_logger {file,"/var/log/rabbitmq/rabbit@esah-ost1-cc01p-sasl.log"} -rabbit enabled_plugins_file "/etc/rabbitmq/enabled_plugins" -rabbit plugins_dir "/usr/lib/rabbitmq/lib/rabbitmq_server-3.3.5/sbin/../plugins" -rabbit plugins_expand_dir "/var/lib/rabbitmq/mnesia/rabbit@esah-ost1-cc01p-plugins-expand" -os_mon start_cpu_sup false -os_mon start_disksup false -os_mon start_memsup false -mnesia dir "/var/lib/rabbitmq/mnesia/rabbit@esah-ost1-cc01p"
root     11870 29338  0 12:07 pts/0    00:00:00 grep --color=auto rabbit

[root@esah-ost1-cc01p rabbitmq]# lsof rabbit@esah-ost1-cc01p.log
[root@esah-ost1-cc01p rabbitmq]# lsof rabbit@esah-ost1-cc01p-sasl.log
[root@esah-ost1-cc01p rabbitmq]# lsof rabbit@esah-ost1-cc01p.log-20170514
COMMAND   PID     USER   FD   TYPE DEVICE SIZE/OFF     NODE NAME
beam.smp 9516 rabbitmq   10w   REG    8,2  2445512 17689604 rabbit@esah-ost1-cc01p.log-20170514
[root@esah-ost1-cc01p rabbitmq]# lsof rabbit@esah-ost1-cc01p-sasl.log-20170514
COMMAND   PID     USER   FD   TYPE DEVICE SIZE/OFF     NODE NAME
beam.smp 9516 rabbitmq   11w   REG    8,2    13766 17698260 rabbit@esah-ost1-cc01p-sasl.log-20170514
Comment 1 Red Hat Bugzilla Rules Engine 2017-06-16 06:10:13 EDT
This bugzilla has been removed from the release and needs to be reviewed and Triaged for another Target Release.
Comment 2 Red Hat Bugzilla Rules Engine 2017-06-16 06:13:54 EDT
This bugzilla has been removed from the release and needs to be reviewed and Triaged for another Target Release.
Comment 8 Peter Lemenkov 2017-10-02 09:45:37 EDT
As far as I know the only remaining issue related to logrotation is one related to selinux. See bug 1426600 (Fedora EPEL) or bug 1413775 (RHOS9, RHOS10). Could yoyu please check that openstack-selinux is installed? I also believe that switching SELinux to permissive work will help but this might not be exactly what you wanted.

I'm working on this SElinux issue.

Note You need to log in before you can comment on or make changes to this bug.