From Bugzilla Helper: User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.4) Gecko/20030624 Description of problem: Request for Enhancement: Merge Suse logrotate "dateext" patches. On Redhat systems, logrotate renames files by numbering and renumbering - messages becomes messages.1, .1 becomes .2, etc. This makes life difficult for rsync backups, which saves each copy of each renumbered file. Ruediger Oertel at Suse (ro) has some patches that enhance the basic Redhat Logrotate with "dateext". This allows a dated extension rather than a numbered one, i.e. messages.20031029 . The old logfiles do not get renamed, just discarded after they get too old. This is a lot easier on rsync, and it also is easier to administer. Ruediger's patches do not change the current logrotate behavior, just adds the new features. His patch to logrotate 3.6.10 can be found at: ftp.suse.com/pub/people/ro/logrotate Suse has been making these patches to RH logrotate for years, and I've been running versions of their code for a few weeks while still using the old logrotate method for testing. It would be nice to fold this into the main line of fedora/redhat and eliminate a needless code fork. Version-Release number of selected component (if applicable): 3.6.10 and others
My understanding is that the patch does not play well with some other logrotate features such as the rotate keyword. Is this correct?
The rotate keyword still appears to function normally. I am running the installed /etc/logrotate.conf file. It contains the default "rotate 4" command, and indeed the normal logfiles (boot.log cron mail.log messages mysqld netconf.log rpmpkgs secure sendfax spooler up2date xferlog ) are all getting rotated in that manner, that is, I have files [cron, cron.1, cron.2, cron.3, cron.4]. pacct is set for "rotate 32" and I have all 33 files, no more or less. If I understand the meaning of the "rotate" statement, the patched version works in the expected way for the old-style number scheme. Of course, logrotate targets configured for "dateext" will need to expire files by date and not by number. That is sensible, and these two ways of doing things should probably remain somewhat orthogonal to prevent surprising interactions. However, I have not tested any corner cases or non-default cases, so I cannot prove there isn't a gotcha in there somewhere. If you have any tests you would like me to try, please let me (and ro ) know. I've found Ruediger to be a very helpful and cooperative fellow, and while I am not so clueful, I would be glad to run tests and help hunt bugs. Thank you for your time; Keith Lofstrom
I have not seen any action on this since November. I answered the question from Elliot Lee - this enhancement does NOT interfere with base functionality. How do I encourage some action on this RFE?
Keith, I think the point may be WHY?..... It seems to be easier for me to type 'cat crom.1' than it is to find the date last week when the logs were rotated and type 'cat crom.20031230'..... That seems like a lot of typing. It may help with some people who want to keep large numbers of log files (52 for a years worth). And archive them to CD or something. But, practically, I think you will have problems getting people use to typeing that much just to get last weeks logs. Good Luck though, James
Why? I have a large set of logs that I keep for a long time. Having them tagged by date accomplishes three things: (1) I can see the dates when I do an 'ls' without having to 'ls -l'. (2) They come sorted in order of increasing date by default. More than ten generations of logs would normally be sorted as log, log.1, log.10, log.11, ..., log.2, etc. For the times when you want to handle logs in a chronological block (e.g., grep for an event and see all occurrences in all logs in chronological order), this would be a boon. (3) Rotating doesn't require renaming all previous generations. Only the current log needs to be renamed. If you have many generations of logs for many sets of apps, then that can get to be a serious burden on logrotate. We were keeping 10 generations of logs for 30-40 Web sites, and logrotate would run half a day or more trying to keep up. So, why not give those of us who *do* keep a year's worth of multiple sets of weekly logs the option to manage them more effectively? It doesn't have any effect at all on users that don't care... Thanks.
This is an *option*. Nobody is forced to use it, and the old behavior will keep working fine. However, if you run rsync for backups or mirroring images, then having the same file and the same data change names from crom.1 to crom.2 means that it needs to be moved and backed up again, for no reason. This is a waste of bandwidth and backup media. Numerous people have requested this. A few have groused that it will cause their code to stop working, but they are neglecting that this option does not come into play unless you purposely configure it differently than the distro out of the box. Most people don't need it, granted. But if you keep forking off small groups whose needs you ignore, eventually you've forked off everyone. SUSE thinks this is important enough that they put in the effort to maintain a separate source tree, for no other reason than that Redhat ignores their request to fold this code back into the main branch of the tree. Perhaps that is distro politics as usual, but it seems like a silly way to grow the Linux business for everyone. Keith
I agree with Keith. This is simply an option that would enhance logrotate for those who want it. Elliott, do you have any information about the problems that may come up with rotate and other key words?
I also like to have this feature. Especially for Keiths backup reasons, but also for properly sorted ls output (although ls -t will do the same). I was planning to build a patch myself but this will save me some time. So please Elliot, consider honoring this request! Thanks -Marcel
I would like this feature as well. I mean, it's obvious. I can't believe it was met with so much resistance. Maybe I'll switch to suse since their developers seem less obtuse. BTW, the mysql version shipped with fedora core 2 stable release in may, is SIX MONTHS OLD. And I don't mean six months compaired to the "testing realease". I mean, mysql 4 stable was released six months before core 2 was released. You're the center of the LAMP architecture and you're screwing up the "M". Hell, I'm switching to suse.
Unfortunately, mysql 4 has licensing issues that prevent it from being included... For including this patch, well, it's not the highest priority. Hopefully once Fedora CVS happens (and it will!) then you'll be able to take over logrotate maintainence from me :)
We've been dieing for this. Please add it to fedora core. If its planned to be added, when? Thanks.
just drop me (ro) a mail for questions about the patch or if you need the patch to be ported to some more recent version (if there is one)
it appears the Fedora CVS is now open http://www.fedora.us/wiki/FedoraCVS Thus, hopefully someone will add it soon. Just out of curiousity, how does the patch rotate out old logs? Is it by the log's mtime or by simply using the largest timestamp in the file names since log.20040801, 20040802.log, 20040803.log, are essentially the same as log.1, log.2, log.3 being incremental integers, just much larger and possibly not sequential (ex. 20040731.log, 20040801.log)?
the current implementation uses the glob result. for "found files = n" and "rotateCount = r" it will remove the first "n - r" files returned by glob. Removing by rotateCount is probably a bad idea anyway, I'm using an extension "maxage" to drop logfiles older than N days, for some cases of use this is even needed, some countries have laws not to keep user related log data longer than a certain amount of time (AFAIR)...
BTW: patches against version 3.7 can be found in the src-rpm at the same location as in comment#1
So since the Fedora CVS is up, does anyone want to incorporate the patches into the main src?
I'm currently rsyncing log files from Australia to the UK, where one of our company's offices is. When the log files get rotated, the whole load get transferred again because of the current logrotate behaviour. I've implemented date-based filenames with my own cron job, but I too would find this very useful in the default logrotate. Any progress?
Created attachment 108349 [details] SUSE's "dateext" logrotate patch I've attached SUSE's "dateext" patch for logrotate 3.7 - extracted from ftp://ftp.suse.com/pub/people/ro/logrotate/logrotate-3.7-31.2.src.rpm SUSE have had the "dateext" capability since December 2001. I think that this fact alone suggests that it just works! (I'm not a C coder, so can't tell you much else!).
Created attachment 108350 [details] "maxage" extension "maxage" extension, from the same source. Again, it's been in SUSE's logrotate since December 2001; again I'd say that's a good enough testimony to its quality. Pleeaasseeee Mr. Red Hat can we have this?
Maintaining large systems this patch can be very useful. Please, I hate creating my own rpms...
The inactivity on this bug is quite difficult to understand. If the maintainer is too busy to review it, can he find another maintainer? If he is not too busy, then can a decision be made? We have here: i) A request that many people want ii) Tested and reliable patches available iii) A 15 month silence from the maintainer (who has been changed in the meantime, but the new maintainer is also silent) This isn't the only entry in bugzilla requesting this; others are older still. This all seems like a bit of a process failure; if there's a good reason for the patch not to be applied, what is it? The net combined effect of i) ii) and iii) above is just that we think "What's the point of reporting things to Red Hat? They just ignore us." There's a big de-motivation here for people who have posted on this bug to keep on bothering using Bugzilla and helping Red Hat improve their products.
I would find this feature very useful to save on backup media and network usage and time. I would even patch all the Red Hat installations I admin (old Red Hat, RHEL and FC), but I'd then have to remember to do it after every server istall (and up2date of logrotate?). Please, Red Hat, include it and save me some work. BTW does anybody know how logwatch behaves when the syslogs have the dateext extensions?
FWIW, there is a request to include these patches in Debian Linux as well: http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=257685 (I too would like to see the inclusion of these patches in Red Hat Linux)
http://people.redhat.com/pvrabec/logrotate-3.7.1-11.src.rpm includes 'dateext' and 'maxage'. I appreciate you test it and let me know it works.
Created attachment 113961 [details] patch with the leftover part of the moveout-patch I guess the rest of the moveout patch (as attached here) could be applied as well. the moveout patch moves some common code into the functions mailLogWrapper and removeLogFile and this leftover patch now makes use of these functions.
The University of Iowa would also like to see these patches included.
Can somebody tell me what this hunk from maxage patch is suppose to do? @@ -644,6 +647,22 @@ globfree(&globResult); free(glob_pattern); } else { + if ( log->rotateAge ) { + struct stat fst_buf; + for (i=1; i <= rotateCount; i++) { + sprintf(oldName, "%s/%s.%d%s%s", dirName, baseName, + rotateCount + 1, fileext, compext); + if(!stat(oldName,&fst_buf) + && (((nowSecs - fst_buf.st_mtime)/60/60/24) + > log->rotateAge)) { + char * mailFilename = (globResult.gl_pathv)[i]; + if (!hasErrors && log->logAddress) + hasErrors = mailLogWrapper(mailFilename, mailCommand, logNum, log); + if (!hasErrors) + hasErrors = removeLogFile(mailFilename); + } + } + } sprintf(oldName, "%s/%s.%d%s%s", dirName, baseName, logStart + rotateCount, fileext, compext);
well, it's really been a while since I wrote that code, but at first look: for all rotated files (for one given target) see if each file is older than the limit set by "rotateAge" and mail the file out if wanted and delete the file.
for (i=1; i <= rotateCount; i++) { sprintf(oldName, "%s/%s.%d%s%s", dirName, baseName, rotateCount + 1, fileext, compext); if(!stat(oldName,&fst_buf) && (((nowSecs - fst_buf.st_mtime)/60/60/24) This is not for all rotated files, is it? If rotate==4 it stats oldname(ex. foo.5) for 4 times. char * mailFilename = (globResult.gl_pathv)[i]; glob() is not callen before this line => globResult is not allocated. I'd like to rewrite it to: if ( log->rotateAge ) { struct stat fst_buf; for (i=1; i <= rotateCount; i++) { sprintf(oldName, "%s/%s.%d%s%s", rotNames->dirName, rotNames->baseName, i, fileext, compext); if(!stat(oldName,&fst_buf) && (((nowSecs - fst_buf.st_mtime)/60/60/24) > log->rotateAge)) { char * mailFilename = oldName; if (!hasErrors && log->logAddress) hasErrors = mailLogWrapper(mailFilename, mailCommand, logNum, log); if (!hasErrors) hasErrors = removeLogFile(mailFilename); } } } What do u think about this? Another question is - see bug #169885. thx. for help
uhmm, the "else" case, MAXAGE without DATEEXT ... well, looks like I did never really try or use that :( yes, your code looks perfectly right.