HOWTO: Use logrotate

One of the most useful tools for log management in UNIX is logrotate, which is part of just about any UNIX distribution. In short, it lets you automatically split, compress and delete log files according to several policies , and is usually employed to rotate common files like /var/log/messages, /var/log/secure and /var/log/system.log.

This HOWTO shows you how to set up log rotation not at a system level, but for a given user (I use it to manage my procmail and Cacti log files, but you can use it for just about anything...)

Let's assume you're user, and that you've set up a daemon to run under your username and spit out the files to ~user/var/log/daemon.log. Your filesystem tree looks like this:

/home/user --+-- etc <- we're going to put logrotate.conf here
             |
             +-- Mail
             ...
             +-- var --+-- lib <- the logrotate status file goes here
                       |
                       +-- log <- the actual log files go here

(If it doesn't, don't worry. I tend to be very picky about such things, and like to use standard file system conventions wherever possible.)

The first step is to create a configuration file. Here is a sample that rotates the log file on a weekly basis, compresses the old log, creates a new zero-byte file and mails us a short report:

$ cat ~/etc/logrotate.conf
# see "man logrotate" for details
# rotate log files weekly
weekly

# keep 4 weeks worth of backlogs
rotate 4

# create new (empty) log files after rotating old ones
# (this is the default, and can be overriden for each log file)
create

# uncomment this if you want your log files compressed
compress

/home/user/var/log/daemon.log {
    create
    mail user@localhost
}

You can, of course, check out man logrotate and add more options (or more files with different options).

Making logrotate actually work, however, requires invoking it from cron. To do that, add it to your crontab specifying the status file with -s and the configuration file you created:

$ crontab -l
0 0 * * * /usr/sbin/logrotate -s /home/user/var/lib/logrotate.status \
/home/user/etc/logrotate.conf > /dev/null 2>&1

(Take care - some systems do not allow "\" to skip to the next line, which means you must enter the logrotate invocation in a single line)

The above invokes logrotate at midnight every day, dumping both standard output and standard error to /dev/null. It will then look at its status file and decide whether or not it is time to actually rotate the log files.