Bug 68096 - Improper rotating of logs
Improper rotating of logs
Status: CLOSED CURRENTRELEASE
Product: Red Hat Linux
Classification: Retired
Component: mgetty (Show other bugs)
7.3
All Linux
high Severity high
: ---
: ---
Assigned To: Jason Vas Dias
:
: 76342 (view as bug list)
Depends On:
Blocks:
  Show dependency treegraph
 
Reported: 2002-07-06 04:44 EDT by Andrew E. Mileski
Modified: 2005-10-31 17:00 EST (History)
6 users (show)

See Also:
Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed: 2005-06-03 18:32:28 EDT
Type: ---
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)

  None (edit)
Description Andrew E. Mileski 2002-07-06 04:44:24 EDT
Description of Problem:
  There is an error in /etc/logrotate.d/mgetty

  /var/log/mgetty.log.tty* {
  ...
  }

  The wildcard causes logrotate to rotate rotated logs resulting in:

  mgetty.log.ttyS1      mgetty.log.ttyS1.1.1.1  mgetty.log.ttyS1.2.1
  mgetty.log.ttyS1.1    mgetty.log.ttyS1.1.2    mgetty.log.ttyS1.3
  mgetty.log.ttyS1.1.1  mgetty.log.ttyS1.2

Version-Release number of selected component (if applicable):
  mgetty-1.1.28-3

How Reproducible:
  Always.

Steps to Reproduce:
1. Use mgetty in /etc/inittab

Actual Results:
  See above.

Expected Results:
  Logs with the following filenames:

  mgetty.log.ttyS1      mgetty.log.ttyS1.3
  mgetty.log.ttyS1.1    mgetty.log.ttyS1.4
  mgetty.log.ttyS1.2

Additional Information:
  None.
Comment 1 R P Herrold 2002-07-09 09:57:09 EDT
 This is quite serious -- it caused an 'out of inodes' forkbomb and eats up entire filesystems. 
 
 
 
020708 
   ext3_new_inode: error 28 
On Dec 19, 2001  12:48 -0500, Ray Turcotte wrote: 
> I have been reviewin my message slog and have found the following 
> message: 
> 
> Dec 19 06:27:28 server02 kernel: EXT3-fs error (device sd(8,7)) in 
> ext3_new_inode: error 28 
> 
> What is error 28 and should I be worried about it? 
 
These are system error numbers, 28=ENOSPC.  I guess you ran out of 
inodes? 
 
 
 
[root@server4 log]# df -i 
Filesystem            Inodes   IUsed   IFree IUse% Mounted on 
/dev/sda2             192000   19131  172869   10% / 
/dev/sda1               8032      41    7991    1% /boot 
/dev/sda13            574848    1959  572889    1% /data 
/dev/sda8             127744    3565  124179    3% /home 
/dev/sda6             320640    2784  317856    1% /opt 
none                  128498       1  128497    1% /dev/shm 
/dev/sda9              76320      64   76256    1% /tmp 
/dev/sda5             384000   84408  299592   22% /usr 
/dev/sda11            128520    2061  126459    2% /var 
/dev/sda7             256512  256482      30  100% /var/log 
/dev/sda12            128520    4258  124262    4% /var/spool 
 
[root@server4 log]# 
 
that is, the logfile explosion ate up all of /var/log 
 
Comment 2 R P Herrold 2002-07-09 09:58:05 EDT
 which is of course diabling ... raising priority
Comment 3 Nils Philippsen 2002-08-10 04:49:27 EDT
The bug still exists in the betas (mgetty-1.1.28-6), just had to delete 5000+
rotated log files.

Maybe rename the log file to /var/log/mgetty.tttXYZ.log, then rotate
/var/log/mgetty*.log?
Comment 4 R P Herrold 2002-08-10 14:48:52 EDT
I don't think the globbing in logrotate works that way -- the thought is good,
though.

Really for the present, just rem,oving the d*mn asterisk and ship with the
pre-next-version prior-version update flood seems like a viable approach.
Comment 5 Pekka Savola 2002-10-05 04:41:11 EDT
We've also been bitten by this.. could this *please* be fixed???

Perhaps '/var/log/mgetty.log.tty??' would be ok? (Or plain ttyS?)
Comment 6 Pekka Savola 2002-10-05 04:57:27 EDT
We started using the following (3 new commands):

/var/log/mgetty.log.tty?? {
	rotate 12
	monthly
	notifempty
	nocompress
	missingok
}

IMO; notifempty should at least be added as mgetty logs are often empty and
rotating empty logs makes no sense.
Comment 7 Mike Bird 2002-10-07 20:26:35 EDT
This is nasty, and the fix is simple, so why not roll it out?

Just had to remove over 350,000 log files, 90% of which appeared overnight since we do ten daily rotations.  Ran out of i-nodes for mail messages.
Comment 8 Pekka Savola 2002-10-08 01:38:28 EDT
Totally agree, raising priority.
Comment 9 Bennett Feitell 2003-01-31 21:29:48 EST
I too have been bitten by this running out of inodes on a practically empty
partition.  The bug is pretty nasty since it really doesn't tend to take off
until a machine has been setup and forgotten about for a while.  On a lightly
used machine the bug might take quite some time to rear its ugly head.

I was running a headless keyboardless dedicated fax reciever that suddenly
started acting very strange.  Shortly after running out of inodes a hardware
failure occured (power supply).  

After figuring out how to delete all the rolled over logs, I checked the
/etc/logrotate.d/mgetty file and noticed the wildcard that caused the
exponential? log rollover and set it statically to ttyS0.  There were so many
files that ls, rm, mv and even find failed when asked to work on a wildcard in
the /var/log directory.  

Not being a shell wizard, mc (and patience) came to the rescue.

The machine in question was receiving around 100 multipage faxes a week and
pitching them to a dedicated HPLJ4L, then gzipping and storing the recieved
faxes.  It really was great for a while, the whole setup was built from dumpster
parts and provided an obscenely low cost per page compared to standard plain
paper fax machines.  The machine goes back into production Monday.  This is the
first time I have had a bug bite so hard as to kill one of the machines I tend.
Comment 10 Andrew E. Mileski 2003-02-11 15:17:48 EST
I've been informed by another victim that this bug that I first logged over 7
months ago still exists :(

AFAIK Red Hat is no longer supporting anything older than the previous release,
so I doubt if this bug will be revisited before the next.

The 8.0 "solution" is too darn ugly to be correct in all situations ("good"
solutions are often simple and elegant):

#                                                                              
# Rotate mgetty log files.
#
/var/log/mgetty.log.tty[^.] /var/log/mgetty.log.tty[^.][^.] /var/log/mgetty.log
.tty[^.][^.][^.] /var/log/mgetty.log.tty[^.][^.][^.][^.] /var/log/mgetty.log.tt
y[^.][^.][^.][^.][^.] /var/log/mgetty.log.tty[^.][^.][^.][^.][^.][^.] /var/log/
mgetty.log.tty[^.][^.][^.][^.][^.][^.][^.] /var/log/mgetty.log.tty[^.][^.][^.][
^.][^.][^.][^.][^.] /var/log/mgetty.log.tty[^.][^.][^.][^.][^.][^.][^.][^.][^.]
 /var/log/mgetty.log.tty[^.][^.][^.][^.][^.][^.][^.][^.][^.][^.] /var/log/mgett
y.log.unknown /var/log/mgetty.callback {
        nocompress
        missingok
}
Comment 11 giulioo 2003-02-20 13:50:04 EST
*** Bug 76342 has been marked as a duplicate of this bug. ***
Comment 12 Jason Vas Dias 2005-06-03 18:32:28 EDT
Closing out old bugs here.

The current mgetty logrotate configuration file may be ugly, but
it does work, as it does match only mgetty.log.${tty} files 
suffixed by any legal tty name.

Other solutions would be to change the name of mgetty log
files (not popular with scripts that may depend on current naming),
to use the new logrotate "olddir" directive ONLY for mgetty logs 
(entailing a new directory just for old mgetty logs), or changing
logrotate to use regexps rather than globs .

If you can think of a glob expression as defined by glob(7) that
would match only mgetty.log.tty[^.]+ (in regexp notation) file 
names, please append it to this bug report.  

Note You need to log in before you can comment on or make changes to this bug.