Reducing Threads in the TimeAndSizeRollingAppender

Several people have commented that the TimeAndSizeRollingAppender could, where many instances were configured, launch many threads.  With date roll enforcement, backup log file scavenging, and backup log file compression all configured, there could be 3 threads per TASRA instance.  That simply didn’t scale.  The original design assumption that only one or two TASRA instances would be configured per application was clearly broken.

I have begun some work to minimise the number of threads launched where multiple TASRA instances are configured.  The first fruits of that have been released and can be downloaded from www.simonsite.org.uk.

The changes permit an arbitrary number of TASRA instances to share a single thread for all backup log file scavenging and backup log file compression tasks.  Date roll enforcement is still carried out on a separate daemon thread, for now.

The benefit is best illustrated with some numbers; taking an arbitrary 10 TASRA instances as opposed to just one to better illustrate the effect:

TASRA configuration	Daemon thread count per 10 TASRA instances
	20120703-0826 Release	20130130-1840 Release
ScavengeInterval=300000, DateRollEnforced=true, CompressionAlgorithm=ZIP	30	11
ScavengeInterval=300000, DateRollEnforced=false, CompressionAlgorithm=ZIP	20	1
ScavengeInterval=300000, DateRollEnforced=false, CompressionAlgorithm=default (not set)	10	1
ScavengeInterval=-1, DateRollEnforced=false, CompressionAlgorithm=default	0	0

By default, all TASRA instances will now share the same thread.  Optionally, m TASRA instances may be configured to use n threads, where m >= n >= 1.  To do this, a common named object must be shared by all TASRA instances that will share the same thread, and distinct named object(s) for any other TASRA instances that will use different thread(s).  The following sample configuration illustrates two TASRA instances, “appender-1” and “appender-2”, each using its own thread, “executor-1” or “executor-2”, respectively, in this case for scavenging and compression tasks:

       <appender name="file-appender-1"

              class="uk.org.simonsite.log4j.appender.TimeAndSizeRollingAppender">

              <param name="File" value="${tasra.xml.test.log.file.1}" />

              <layout class="org.apache.log4j.SimpleLayout" />

              <param name="CompressionAlgorithm" value="ZIP" />

              <logTaskExecutorService class="uk.org.simonsite.log4j.appender.NamedLoggingTaskExecutorService">

                     <param name="Name" value="executor-1" />

              </logTaskExecutorService>

       </appender>

       <appender name="file-appender-2"

              class="uk.org.simonsite.log4j.appender.TimeAndSizeRollingAppender">

              <param name="File" value="${tasra.xml.test.log.file.2}" />

              <layout class="org.apache.log4j.SimpleLayout" />

<param name="CompressionAlgorithm" value="ZIP" />

              <logTaskExecutorService class="uk.org.simonsite.log4j.appender.NamedLoggingTaskExecutorService">

                     <param name="Name" value="executor-2" />

              </logTaskExecutorService>

       </appender>

Log4J 1.2.17 Support

A handful of things to report this time around.

First and foremost I've widened the testing for both TimeAndSizeRollingAppender and ActiveAsynchronousAppender to include Log4J 1.2.17.  There are only a few changes in 1.2.17 and no changes were required to either the TimeAndSizeRollingAppender or ActiveAsynchronousAppender to be fully compatible.  The Maven 2 POM file dependency still shows up as 1.2.16, however this is a fairly arbitrary choice based upon improvements in threading behaviour in 1.2.16; there's no reason not to edit this for use with 1.2.15 or 1.2.17 if desired.

Secondly, the unit test coverage for the ActiveAsynchronousAppender has been increased considerably.  Some slight inconsistencies in the behaviour of classes targeted towards specific Java versions have been ironed out.

Finally, the www.simonsite.org.uk hosting has been moved.  A problem with IPS tag provisioning, caused by my original hosting provider, resulted in the site going offline for over a week.  This was highly regrettable and I apologise for any inconvenience this may have caused.  The site is back online after having completed the move to its new home.  New versions of the TimeAndSizeRollingAppender and ActiveAsynchronousAppender can be downloaded from www.simonsite.org.uk as before.

I hope the Appenders continue to be useful.

uk.org.simonsite.log4j.appender.TimeAndSizeRollingAppender

I have been thinking for a little while now that the name of the package in which the TimeAndSizeRollingAppender (TASRA) lives is perhaps inappropriate.  From its original inception back in 2008, through successive releases to date, the TASRA has lived in the package org.apache.log4j.appender.  However, development of the TASRA is neither sponsored nor otherwise supported by the Apache Foundation: the code is not bundled with any Log4J distros, it is not managed within an Apache repository, nor is there any reference to it on the Apache websites outside of forum postings and JIRA items that refer to it.  The only connection with Apache is the fact that the TASRA is distributed under the Apache 2.0 license.

At a technical level, a user recently requested a change of package name such that the TASRA could be more readily deployed within a JBoss environment.  I have no experience of JBoss, nor the use of signed JARs which drove the user’s particular request.  However, this has provided a concrete reason to change the name of the package within which the TASRA will reside.

Provided there is no significant objection from users all future releases the TASRA will live in the uk.org.simonsite.log4j.appender package.

For most users, the only difference will be that configuration must change to use the new fully-qualified class name (uk.org.simonsite.log4j.appender.TimeAndSizeRollingAppender).  For advanced users, import declarations will need to be updated and code recompiled.  Outside of that the TASRA will continue to work as before.

I intend to make this change soon.