Bug 657025
| Summary: | some /var/log/dcloud-agg logs missing log-rotate, causing filesystem full | ||
|---|---|---|---|
| Product: | [Retired] CloudForms Cloud Engine | Reporter: | Dave Johnson <dajohnso> |
| Component: | aeolus-conductor | Assignee: | Mo Morsi <mmorsi> |
| Status: | CLOSED CURRENTRELEASE | QA Contact: | Dave Johnson <dajohnso> |
| Severity: | high | Docs Contact: | |
| Priority: | high | ||
| Version: | 0.3.1 | CC: | cpelland, deltacloud-maint, morazi, whayutin |
| Target Milestone: | rc | Keywords: | Reopened |
| Target Release: | --- | Flags: | mmorsi:
needinfo+
|
| Hardware: | Unspecified | ||
| OS: | Unspecified | ||
| Whiteboard: | |||
| Fixed In Version: | Doc Type: | Bug Fix | |
| Doc Text: | Story Points: | --- | |
| Clone Of: | Environment: | ||
| Last Closed: | 2011-03-31 20:46:04 UTC | Type: | --- |
| Regression: | --- | Mount Type: | --- |
| Documentation: | --- | CRM: | |
| Verified Versions: | Category: | --- | |
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |
| Cloudforms Team: | --- | Target Upstream Version: | |
| Embargoed: | |||
/etc/logrotate.d /etc/cron.daily/logrotate /etc/logrotate.d/deltacloud-aggregator /etc/logrotate.d/httpd /etc/logrotate.d/mongodb /etc/logrotate.d/ppp /etc/logrotate.d/psacct /etc/logrotate.d/puppet /etc/logrotate.d/sssd /etc/logrotate.d/syslog /etc/logrotate.d/wpa_supplicant /etc/logrotate.d/yum Looks like we need a log rotate script for condor deltacloud-core iwhd.log Patch submitted and pushed http://git.fedorahosted.org/git/?p=aeolus/conductor.git;a=commitdiff;h=2a8a5d5b82902197e2b9fcf00a3ab8ad882832d2 I dont see that the patch has been applied to an rpm.. What rpm would that patch go into... [root@ip-10-80-242-101 config]# cd /etc/logrotate.d/ [root@ip-10-80-242-101 logrotate.d]# ls aeolus-conductor dracut httpd libvirtd.lxc libvirtd.qemu mongodb puppet syslog up2date yum [root@ip-10-80-242-101 logrotate.d]# [root@ip-10-80-242-101 logrotate.d]# rpm -qa | grep aeolus aeolus-conductor-doc-0.0.3-6.el6.x86_64 aeolus-conductor-0.0.3-6.el6.x86_64 aeolus-conductor-daemons-0.0.3-6.el6.x86_64 aeolus-configure-2.0.0-8.el6.noarch [root@ip-10-80-242-101 logrotate.d]# The file exists in the /etc/logrotate.d directory, as output in your ls command. Inside you should see the dbomatic, connector, and warehouse_sync logrotations. Chris removed the condor_refreshd one when he removed that component from the project alltogether [root@dell-pem710-01 wes]# ls /etc/logrotate.d/
aeolus-conductor dracut libvirtd.lxc mongodb puppet syslog yum
cups httpd libvirtd.qemu psacct subscription-manager up2date
[root@dell-pem710-01 wes]# cat /etc/logrotate.d/aeolus-conductor
weekly
rotate 52
compress
missingok
notifempty
/var/log/aeolus-conductor/rails.log /var/log/aeolus-conductor/thin.log {
sharedscripts
postrotate
/etc/init.d/aeolus-conductor restart
endscript
}
/var/log/aeolus-conductor/aeolus-connector.log {
sharedscripts
postrotate
/etc/init.d/aeolus-connector restart
endscript
}
/var/log/aeolus-conductor/dbomatic.log {
sharedscripts
postrotate
/etc/init.d/conductor-dbomatic restart
endscript
}
/var/log/aeolus-conductor/warehouse_sync.log {
sharedscripts
postrotate
/etc/init.d/conductor-warehouse_sync restart
endscript
}
release pending... release pending... release pending.. 2 perm close closing out old bugs |
I noticed that after I run a deltacloud instance for a week or two, images would always fail to build. I took a look at the filesystem and the / partition was at 99% utilization. [root@localhost deltacloud-aggregator]# df -k Filesystem 1K-blocks Used Available Use% Mounted on /dev/mapper/VolGroup-lv_root 51606140 48279832 704868 99% / tmpfs 756744 0 756744 0% /dev/shm /dev/vda1 495844 27619 442625 6% /boot /dev/mapper/VolGroup-lv_home 27447508 176064 25877168 1% /home [root@localhost deltacloud-aggregator]# ls -lh total 36G -rw-r--r--. 1 dcloud dcloud 36G Nov 24 12:30 condor_refreshd.log -rw-r--r--. 1 dcloud dcloud 36K Nov 23 08:59 dbomatic.log -rw-r--r--. 1 dcloud dcloud 3.1M Nov 22 16:49 image_builder_service.log -rw-r--r--. 1 dcloud dcloud 0 Oct 29 11:03 mongrel.log -rw-r--r--. 1 dcloud dcloud 147M Nov 24 11:45 rails.log -rw-r--r--. 1 dcloud dcloud 390K Nov 21 03:26 rails.log-20101121.gz -rw-rw-rw-. 1 root root 4.8K Nov 22 15:09 thin.log -rw-rw-rw-. 1 root root 556 Nov 21 03:26 thin.log-20101121.gz [root@localhost deltacloud-aggregator]# ^C