Friday, July 6, 2012

Element-Level Caching of Collection Mapping Methods

Note:  This article applies to Spring 3.0 and EhCache 2.5
Update:  Allow non 1:1 mapping
Update:  Uses Spring 3.1's Caching abstractions
Update:  Refactored for clarity

Annotation Based Collection Caching

If you have ever used the ehcache-spring-annotations package Spring caching abstraction then you know what an awesome thing method-level caching is.  If you haven't used it, go check it out now!  To summarise (poorly), you annotate a method with "@Cacheable" and the package uses proxying to wrap the method invocation in a cache template.

For example:
@Cacheable("foo")
public SomeObject getObjectFromServer( String parameter )
{
  return someLengthyRestCall( parameter );
}

During execution, you would get a program flow similar to this:
key = generateKey( methodSignature, parameter );
if( cache.contains( key )) return cache.get( key );
value = getObjectFromServer( parameter );
cache.put( key, value );
return value;

It's more complicated, obviously, but the benefits and easy of this pattern are obvious.  In addition, you can leverage all the power and flexibility of ehcache to do your actual caching.  I apply this to all my MVC service implementations for instant web-service caching.

Limitations

One issue I had, though, was when working with collection-to-collection mapping methods.  This is a common pattern (for me, anyway) where a list of type A is converted to a list of type B in an idempotent stateless manner.

List<A> getAfromB( List<B> list )
{
  List<A> result= new ArrayList<A>( list.size() );
  for( B b: list ) result.add( getAfromB( b ));
  return result;
}

Another common pattern is unordered mapping method:
Collection<A> getAfromB( Collection<B> coll )
{
  Collection<A> result= new HashSet<A>( coll.size() );
  for( B b: list ) result.add( getAfromB( b ));
  return result;
}

If you annotate such a method as @Cacheable it will only cache complete result mappings, which can still be useful if you need to map {A,B,C} -> {X,Y,Z} on a regular basis.  What would really be neat, though, is if caching were applied to each element individually, with only the unknown values being passed on for resolution.

Enter the Aspect

This is a perfect application of AOP (aspect-oriented programming).  Although I'm no AOP expert, I was able to get my feet wet and enable just such a solution in only about two hours, thanks to the excellent documentation provided with the Spring.  This is a pretty bare-bones implementation, but it illustrates the important AOP bits.

Annotation

First we must declare a new custom annotation with which we will mark any method that meets are requirements.  By using explicit annotation-based configuration we give responsibility over the proper use of this aspect to the programmer.  In this configuration, we allow unordered Collection:Collection mapping by specifying which field in the result objects contains the request key.  An ordered List:List mapping is also possible by specifying "IMPLICIT" as the keyField.

@Target(ElementType.METHOD)
@Retention(RetentionPolicy.RUNTIME)
public @interface CollectionCacheable
{
  public static final String IMPLICIT = "##_implicit_##";
  String cacheName();      // EhCache to use
  String keyPrefix();      // This plus ID is unique key
  String keyField();       // ID field in result object
  Class<?> implClass default ArrayList.class;
}

Advice Class

Next we create the actual advice class.  We use @Aspect annotation to make it an aspect, and add a setter to allow injection of a Spring CacheManager object. We also have the key generator and convenience methods for casting.
@Aspect
public class CollectionCacheAspect
{
  // Object used as placeholder when weaving new and cached results
  private static final Object HOLDER = new Object();
  // Object used as part of the key when caching the 'null' object.
  private static final Serializable NULL_KEY = new Long( Long.MIN_VALUE );

  // CacheManager, configured elsewhere
  private CacheManager cacheManager;

  @Required
  public void setCacheManager( CacheManager cacheManager )
  {
    this.cacheManager = cacheManager;
  }

  @SuppressWarnings( "unchecked" )
  public static <T> List<T> cast(List<?> p)
  {
    return (List<T>) p;
  }

  @SuppressWarnings( "unchecked" )
  public static <T> Class<T> cast(Class<?> p)
  {
    return (Class<T>) p;
  }
 
  public Serializable generateKey( String keyPrefix, Object input )
  {
    return 31 * (long)keyPrefix.hashCode() + input.hashCode();
  }

Next we add @Pointcut configuration, which will decide whether to treat the proxied call as an individual, ordered List:List or unordered Collection:Collection operation. 
@Around("@annotation(config) && args(arg) ")
public Object doCollectionCache( ProceedingJoinPoint pjp,
                                 CollectionCache config,
                                 Object arg ) throws Throwable
{
  // Get annotation configuration
  @SuppressWarnings( "unchecked" )
  Class<?> implClass = (Class<Collection<Object>>)config.implClass();
  String cacheName = config.cacheName();
  String keyPrefix = config.keyPrefix();
  String keyField = config.keyField();
  // Get Cache
  Cache cache = cacheManager.getCache( cacheName );
  if( cache == null ) {
    throw new AopInvocationException( "CollectionCachePut:  Cache '"+
                                                            cacheName +
                                                            "' does not seem to exist?" );
  }

  // Call appropriate implementation based on run-time scenario
  Object result;
  if( CollectionCache.IMPLICIT.equals( keyField )) {
    if( List.class.isInstance( arg ) &&
        List.class.isAssignableFrom( implClass )) {
      // IMPLICIT mode (special handling for List->List)
      Class<List<Object>> listClass = cast( implClass );
      result = cacheOrdered( pjp, cache, keyPrefix, listClass, (List<?>) arg );
    } else {
      // Normal single-item cache where arg is the key
      result = cacheSingle( pjp, cache, keyPrefix, keyField, arg );
    }
  } else if( Collection.class.isInstance( arg )) {
    // UNORDERED mode (uses explicit field from result objects)
    Class<Collection<Object>> collClass = cast( implClass );
    result = cacheUnordered( pjp, cache, keyPrefix, keyField,
                             collClass, (Collection<?>)arg );
  } else {
    // SINGLE mode
    result = cacheSingle( pjp, cache, keyPrefix, keyField, arg );
  }
  return result;
}


Single Element Operation 

Since we want non-Collection requests to share the same cache as the Collection calls, we must provide the ability to operate on a single element.  This also handles the special "null" case.

private Object cacheSingle( ProceedingJoinPoint pjp, Cache cache,
                            String keyPrefix, String keyField, Object input )
  throws Throwable
{
  // Determine key
  Object value;
  Object suffix = ( input == null ) ? NULL_KEY : input;
  Serializable key = generateKey( keyPrefix, suffix );
  // Check cache
  ValueWrapper wrapper = cache.get( key );
  // Return cached, or fetch actual value
  if( wrapper != null ) {
    value = wrapper.get();
  } else {
    value = pjp.proceed( new Object[] { input } );
    // Cache fetched value if not null
    if( value != null ) {
      cache.put( key, value );
    }
  }
  return value;
}


Unordered Operation

Unordered mapping is the simpler of the two multi-value modes of operation, since we need not worry about maintaining the order of the request since the cache key is explicitly found in the result values.

private Collection<?> cacheUnordered( ProceedingJoinPoint pjp, Cache cache,
                                      String keyPrefix, String keyField,
                                      Class<Collection<Object>> implClass,
                                      Collection<?> input ) throws Throwable
{
  // Holder for intermediary results
  Collection<Object> hits = new ArrayList<Object>( input.size() );
  // Holder for our misses, which we'll pass on to the original target
  Collection<Object> misses = implClass.newInstance();
  for( Object in: input ) {
    // Search cache for each element; nulls always miss
    ValueWrapper wrapper = null;
    // Put found value in "hits", else put missed key in "misses"
    if( in != null ) {
      Serializable key = generateKey( keyPrefix, in );
      wrapper = cache.get( key );
      if( wrapper == null ) {
        misses.add( in );
      } else {
        hits.add( wrapper.get() );
      }
    } else {
      misses.add( in );
    }
  }

  // Pass our cache misses to original target
  Collection<Object> results = Collections.<Object>emptyList();
  if( misses.size() > 0 ) {
    results = cast( (List<?>)pjp.proceed( new Object[] { misses } ));
  }

  // Cache results
  for( Object value: results ) {
    // Pull key from explicit field
    Object suffix = PropertyAccessorFactory.forBeanPropertyAccess( value )
                    .getPropertyValue( keyField );
    if( suffix != null ) {
      Serializable key=generateKey( keyPrefix, suffix );
      cache.put( key, value );
    }
    // Merge new values into result collection
    hits.add( value );
  }
  return hits;
}


Ordered Operation

Ordered mapping is more difficult. We use the previously defined HOLDER object to mark placeholders in the output List where we will put the results of cache misses from the target method.

private List<?> cacheOrdered( ProceedingJoinPoint pjp, Cache cache, 
                              String keyPrefix, Class<List<Object>> implClass,
                              List<?> input ) throws Throwable
{
  // Holder for intermediary results
  List<Object> hits = new ArrayList<Object>( input.size() );
  // Holder for our misses, which we'll pass on to the original target method
  List<Object> misses = implClass.newInstance();
  for( int i=0; i<input.size(); i++ ) {
    // Search cache for each element; nulls always miss
    ValueWrapper wrapper = null;
    Object in = input.get( i );
    if( in != null ) {
      // Check cache for this object
      Serializable key = generateKey( keyPrefix, in );
      wrapper = cache.get( key );
    }
    if( wrapper == null ) {
      // If element is not found, put HOLDER Object and load the 'misses' list
      hits.add( HOLDER );
      misses.add( in );
    } else {
      // If element is found, then add cached value to intermediary results
      hits.add( wrapper.get() );
    }
  }

  // Pass our cache misses to original target
  List<Object> results = Collections.<Object>emptyList();
  if( misses.size() > 0 ) {
    results = cast( (List<?>)pjp.proceed( new Object[] { misses } ));
  }

  if( results.size() != misses.size() ) {
    // If our result size does not match input size, we cannot cache new values
    // as we do not know the associated key.  Just merge the lists and return.
    for( Object h: hits ) {
      if( h != HOLDER ) {
        results.add( h );
      }
    }
    return results;

  } else {
    // We'll reuse this list for our output
    misses.clear();
    // Iterate intermediary results
    Iterator<?> iter = results.iterator();
    for( int i=0; i<hits.size(); i++ ) {
      Object h = hits.get( i );
      if( h == HOLDER ) {
        if( iter.hasNext() ) {
          // Each place-holder will have its actual value in the results list
          // at the same location (ie. Nth HOLDER is in results[N]
          Object value = iter.next();
          misses.add( value );
          // Cache new non-null values
          if( input.get( i ) != null ) {
            Serializable key=generateKey( keyPrefix, input.get( i ));
            cache.put( key, value );
          }
        }
      } else {
        // This was a cache hit earlier so just use it
        misses.add( h );
      }
    }
  }
  return misses;
}

Cache Evictions

Evictions is just a simpler application of the above concepts.

The annotation:

@Target(ElementType.METHOD)
@Retention(RetentionPolicy.RUNTIME)
public @interface CollectionEvict
{
  public static final String IMPLICIT = CollectionCache.IMPLICIT;
  String cacheName();
  String keyPrefix() default "";
  String keyField();
  boolean removeAll() default false;
}

A pointcut to handle the special "no-args-remove-all" scenario:
@Before("@annotation(config)" )
public void doCollectionEvict( CollectionEvict config ) throws Throwable
{
  if( !config.removeAll() ) {
      // No keys and (removeAll == false)?  Nothing to do here.
      return;
  }
  doCollectionEvict( config, null );
}

A pointcut to choose which mode of operation (ordered, unordered, implicit, etc)
@Before("@annotation(config) && args(arg) ")
public void doCollectionEvict( CollectionEvict config,
                               Object arg ) throws Throwable
{
  // Get annotation configuration
  String cacheName = config.cacheName();
  String keyPrefix = config.keyPrefix();
  String keyField  = config.keyField();
  boolean removeAll = config.removeAll();

  // Get Cache
  Cache cache = cacheManager.getCache( cacheName );
  if( cache == null ) {
    throw new AopInvocationException( "CollectionCacheEvict:  Cache '"+ cacheName +"' does not seem to exist?" );
  }

  if( removeAll ) {
    // Evict all items
    cache.clear();

  } else if( List.class.isInstance( arg )) {
    // Evict as list
    for( Object in: (List<?>)arg ) {
      evict( cache, keyPrefix, keyField, in );
    }

  } else {
    // Evict as object
    if( arg == null ) {
      evict( cache, keyPrefix, keyField, NULL_KEY );
    } else {
      evict( cache, keyPrefix, keyField, arg );
    }
  }
}
And the actual eviction logic:
private void evict( Cache cache, String keyPrefix,
                    String keyField, Object input )
{
  // Key is based upon strategy marked by presence of keyField parameter
  // If parameter is present (ie. is not "" ) then cache by explicit field
  final boolean implicitKey = CollectionCache.IMPLICIT.equals( keyField );
  if( input != null ) {
    Object suffix = implicitKey ? input :
                    PropertyAccessorFactory.forBeanPropertyAccess( input )
                    .getPropertyValue( keyField );
    Serializable key = generateKey( keyPrefix, suffix );
    cache.evict( key );
  }
}


Spring Configuration

Wiring it all together with Spring is:

<!-- Enable AOP -->
<aop:aspectj-autoproxy/>
<!-- The EhCacheManager is usually created within Hibernate startup, so we must
indicate we want the shared singleton instance. -->
<bean id="mvcEhCache" class="org.springframework.cache.ehcache.EhCacheManagerFactoryBean">
    <property name="shared" value="true"/>
</bean>
<!-- Spring's abstract CacheManager
<bean id="cacheManager" class="org.springframework.cache.ehcache.EhCacheCacheManager">
  <property name="cacheManager" ref="mvcEhCache"/>
</bean>
<!-- Our Aspect -->
<bean id="collectionCacheAspect" class="CollectionCacheAspect">
  <property name="cacheManager" ref="cacheManager"/>
</bean>

Putting it All Together

Now we can annotate any appropriate class and get element-level caching:

@CollectionCache( cacheName="listCache", keyPrefix="AtoB", keyField=CollectionCacheable.IMPLICIT )
List<A> getAfromB( List<B> list )
{
  List<A> result= new ArrayList<A>( list.size() );
  for( B b: list ) result.add( getAfromB( b ));
  return result;
}

@CollectionEvict( cacheName="listCache", keyPrefix="AtoB", keyField=CollectionEvict.IMPLICIT )
void evictB( B item ) {}

Conclusions

This, of course, is just part of a full solution.  It should be easy to add additional annotations and functionality to allow adding of individual items to the same cache, and for triggering removal of elements either via a list or individually.  It's a long way from being as feature-filled or rigorous as the original ehcache-spring-annotations package but solves a specific problem and is a good introduction to AOP in Spring.

No comments:

Post a Comment