Roll Log4J log files on application startup

A fairly popular feature request for the TimeAndSizeRollingAppender has been to provide the capability to roll a log file when the application starts up. This can be useful if an application is restarted during a period which falls within a logging interval, and when the most recently written log file is smaller than would be needed to trigger a rollover. In response to requests, I've added this feature into the latest release, available from www.simonsite.org.uk. By default, the appender will continue to behave as it has always done. To turn on the rollover on startup feature, somewhat predictably the appender should be configured with the property RollOnStartup set to true.

Daily Rollover Issue Fixed in TimeAndSizeRollingAppender

Thanks to Bernd who reported a bug in handling of the DatePattern parameter. In
one place in the code the parameter was being parsed as if it contained localized pattern characters, but in another localization was being ignored. My bad for having tested only with ENGLISH locales. The code is fixed and, based upon the principle of least surprise, the appender now always defaults to the ENGLISH locale, as per the Javadoc on SimpleDateFormat. To set up any locale other than ENGLISH, simple use the DatePatternLocale parameter with the language and/or country code as described in the appender Javadoc (e.g. "fr_CA", "de", etc). New code at http://www.simonsite.org.uk.

Log4J Log Rolls At Midnight

I found a requirement for the Log4J DailyRollingFileAppender to roll logs at midnight. This doesn't seem to be possible in Log4J 1.2 unless the application itself issues a logging event at midnight. If the application happens to be idle at that time, a log roll isn't going to happen. I guess you could do make the application do this by running a Timer, for example, thereby ensuring that a logging event is issued and forcing the appender to roll the logs.

It seems to me that it would be convenient if the appender could take care of rolling files on a schedule itself, so I've added this behaviour to the Log4J TimeAndSizeRollingAppender hosted on my site at www.simonsite.org.uk.

The new stuff doesn't try to be too clever. It simply looks at the DatePattern config parameter of the appender (this is in common with the DailyRollingFileAppender) and assumes that the end-point of a logging period implied by it is when rolling should happen. So if you have a DatePattern set to 'yyyy-MM-dd' the appender assumes that it should force a file roll at the end of every day.

Compression Added

I received a code contribution for the TimeAndSizeRollingAppender from Eduardo Simioni, providing two new features:

1. Backup log files can be compressed.
2. The scavenger thread that deletes older files can be configured not to run, thereby conserving resources whilst allowing a virtually unlimited number of backups.

I've extended the contribution to support both GZIP and ZIP via configuration. Obviously you can use only one compression algorithm at a time. New source and binaries available in the usual place.

TimeAndSizeRollingAppender Updated

Thanks to Eduardo and Nick who pointed out a couple of bugs in the TimeAndSizeRollingAppender. The first was a start-up bug which meant that the writer was not being initialised with the size of an existing log file. The second was an NPE issue that occurred if a filename was configured without a path. Both are fixed and the new code is available for download at http://www.simonsite.org.uk/resources/lib/log4j-rolling-appender.jar.

Log4J Asynchronous Appender Contrib

I've updated my ActiveAsynchronousAppender contribution to make fuller use of the JSR166 backport-util-concurrent libraries. Take a look at the Javadoc, or download the source and binaries. More info on my home page.

Log4J Asynchronous Appender

I've been working on an alternative to the Log4J AsyncAppender. OK, but why bother when the standard Log4J AsyncAppender works well enough? In a heavily-threaded application the AsyncAppender can block when its buffer is full, and the dispatcher thread is still busy appending LoggingEvents. This is because AsyncAppender grabs all LoggingEvents from its buffer, and appends all those LoggingEvents in a single loop before checking the buffer contents again. Such blocking is analagous to a GC pause.

In trying to minimise such pauses whilst maximising throughput, I've come up with a slightly different solution to the same problem that AsyncAppender tries to address. The ActiveAsynchronousAppender I have developed blocks when the buffer is full, however it only appends a single LoggingEvent from its buffer at a time, before checking the buffer again. As soon as the buffer is below its maximum size, the appender ceases to block.

I've run the new code against the Log4J 1.2.15 unit tests for the AsyncAppender, and the important tests pass. The caveat is that my code is not designed for "non-blocking" operation, i.e. it cannot be configured to discard LoggingEvents when the appender's buffer is full.

What are the benefits of the new code? As with the standard Log4J AsyncAppender, it depends in no small part upon the nature of the application, and tuning the size of the appender's buffer. I don't yet have any hard figures to demonstrate whether my appender code offers any significant advantage over the existing Log4J AsyncAppender. A couple of things I'm looking at:

• Reduced blocking should lead to higher application throughput
• Fewer LoggingEvents on the stack of a daemon thread may mean more reliable logging at the end of an application lifecycle (not sure about this)