009-spring cache-自己定制緩存接入。CacheManager和Cache

一、概述

現狀：目前緩存框架底層使用redis，但是進行了統一包裝，相當於對外一個新緩存框架，提供了redis基礎功能

問題：緩存混亂亂，由程序員自己set，get。清理不徹底。線上出問題。

需求：項目需要使用spring cache統一對service進行緩存處理。團隊統一的緩存管理，以及同外部一致的緩存策略方案

1.1、實現自定義緩存方案

　　參看緩存實現圖：006-spring cache-緩存實現-01-SimpleCacheConfiguration、RedisCacheConfiguration

　　參看redis 實現架構

　　　　

　　SpringCache本質上是一個對緩存使用的抽象，將存儲的具體實現方案從緩存執行動作及流程中提取出來。緩存流程中面向的兩個抽象接口是CacheManager、Cache。其中Cache提供了緩存操作的讀取/寫入/移除等方法，本着面向抽象編程的原則，內部將緩存對象統一封裝成ValueWrapper。Cache接口代碼如下：

public interface Cache {    
    String getName();  //緩存的名字    
    Object getNativeCache(); //得到底層使用的緩存，如Ehcache    
    ValueWrapper get(Object key); //根據key得到一個ValueWrapper，然后調用其get方法獲取值    
    <T> T get(Object key, Class<T> type);//根據key，和value的類型直接獲取value    
    void put(Object key, Object value);//往緩存放數據    
    void evict(Object key);//從緩存中移除key對應的緩存    
    void clear(); //清空緩存    
    
    interface ValueWrapper { //緩存值的Wrapper    
        Object get(); //得到真實的value    
    }    
}

由於在應用中可能定義多個Cache，因此提供了CacheManager抽象，用於緩存的管理，接口代碼如下：

public interface CacheManager {    
    Cache getCache(String name); //根據Cache名字獲取Cache     
    Collection<String> getCacheNames(); //得到所有Cache的名字    
}

　　任何實現了這兩個接口的緩存方案都可以直接配置進SpringCache使用。其自帶的SimpleCacheManager、ConcurrentMapCache是如此；使用ehcache作為存儲實現的EhCacheCacheManager、EhCacheCache也是如此。可以自己實現CacheManager與Cache並將其集成進來。

二、自定義緩存實現

2.1、自定義緩存實現-基礎版本【需要自己手動寫緩存空間】

1、Cache接口，增刪改差

　　為了方便展示，自定義緩存實現方案只實現最簡單的功能，cache內部使用ConcurrentHashMap做為存儲方案，使用默認實現SimpleValueWrapper，MyCache代碼如下：

package com.aaa.test.config;

import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.cache.Cache;
import org.springframework.cache.support.SimpleValueWrapper;
import org.springframework.context.annotation.Configuration;

import java.util.Map;
import java.util.concurrent.Callable;
import java.util.concurrent.ConcurrentHashMap;

@Configuration
public class MyCache implements Cache {
    final static Logger logger = LoggerFactory.getLogger(MyCache.class);

    String name;
    Map<Object, Object> store = new ConcurrentHashMap<Object, Object>();

    public MyCache() {
    }

    public MyCache(String name) {
        this.name = name;
    }

    @Override
    public String getName() {
        return this.name;
    }
    public void setName(String name){
        this.name = name;
    }


    @Override
    public Object getNativeCache() {
        return store;
    }


    @Override
    public ValueWrapper get(Object key) {
        ValueWrapper result = null;
        Object thevalue = store.get(key);
        if(thevalue!=null) {
            logger.info("["+name+"]got cache, key:"+key);
            result = new SimpleValueWrapper(thevalue);
        }else{
            logger.info("["+name+"]missing cache, key:"+key);
        }
        return result;
    }


    @SuppressWarnings("unchecked")
    @Override
    public <T> T get(Object key, Class<T> type) {
        ValueWrapper vw = get(key);
        if(vw==null){
            return null;
        }
        return (T)vw.get();
    }


    @SuppressWarnings("unchecked")
    @Override
    public <T> T get(Object key, Callable<T> valueLoader) {
        ValueWrapper vw = get(key);
        if(vw==null){
            return null;
        }
        return (T)vw.get();
    }


    @Override
    public void put(Object key, Object value) {
        store.put(key, value);
    }


    @Override
    public Cache.ValueWrapper putIfAbsent(Object key, Object value) {
        Object existing = this.store.putIfAbsent(key, value);
        return (existing != null ? new SimpleValueWrapper(existing) : null);
    }


    @Override
    public void evict(Object key) {
        store.remove(key);
    }


    @Override
    public void clear() {
        store.clear();
    }
}

2、MyCacheManager實現-緩存管理器

public class MyCacheManager extends AbstractCacheManager {
    private Collection<? extends MyCache> caches;

    public void setCaches(Collection<? extends MyCache> caches) {
        this.caches = caches;
    }

    @Override
    protected Collection<? extends MyCache> loadCaches() {
        return this.caches;
    }
}

3、注入使用，配置緩存空間

@Bean(name="myCacheManager")
    public CacheManager myCacheManager(){
        MyCacheManager myCacheManager = new MyCacheManager();
        List<MyCache> caches = new ArrayList<MyCache>();
        MyCache mycache = new MyCache("mycache");
        MyCache mycache2 = new MyCache("mycache2");
        caches.add(mycache);
        caches.add(mycache2);
        myCacheManager.setCaches(caches);
        return myCacheManager;
    }

2.2、 自定義緩存實現-基礎版本【優化命名，使用注解自動化添加】

　　通過上述實現，發現緩存管理器需要手工注入緩存空間。

　　解決思路：在項目啟動，類初始化或者調用方法處，獲取創建命名空間。此處使用后置，調用方法前創建命名空間。

知識點、

　　使用aop攔截，獲取自定義注解上的參數值

　　實現Ordered，已提前緩存注解攔截

1、將上述第三步，修改

@Configuration
@EnableCaching
public class SimpleCacheManagerConfiguration {
    @Bean
    public CacheManager cacheManager() {
        MyCacheManager cacheManager = new MyCacheManager();
        List<MyCache> caches = new ArrayList<MyCache>();
//        caches.add(new MyCache("cache"));
//        cacheManager.setCaches(caches);
        return cacheManager;
    }
}

2、添加 Aspect

package com.aaa.test.aspect;

import com.aaa.test.config.MyCache;
import com.aaa.test.config.MyCacheManager;
import org.aspectj.lang.JoinPoint;
import org.aspectj.lang.annotation.Aspect;
import org.aspectj.lang.annotation.Before;
import org.aspectj.lang.reflect.MethodSignature;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.cache.CacheManager;
import org.springframework.cache.annotation.CacheConfig;
import org.springframework.cache.annotation.CacheEvict;
import org.springframework.cache.annotation.CachePut;
import org.springframework.cache.annotation.Cacheable;
import org.springframework.core.Ordered;
import org.springframework.stereotype.Component;

import java.lang.annotation.Annotation;
import java.util.Arrays;
import java.util.Collection;
import java.util.HashSet;
import java.util.Set;

@Aspect
@Component
public class CacheAspect implements Ordered {
    @Autowired
    private CacheManager cacheManager;

    @Before("@annotation(org.springframework.cache.annotation.Cacheable)")
    public void beforeCacheable(JoinPoint pjp) throws Throwable {
        MethodSignature ms = (MethodSignature) pjp.getSignature();
        setCacheNames(ms, CacheConfig.class);
        setCacheNames(ms, Cacheable.class);
    }

    @Before("@annotation(org.springframework.cache.annotation.CachePut)")
    public void beforeCachePut(JoinPoint pjp) throws Throwable {
        MethodSignature ms = (MethodSignature) pjp.getSignature();
        setCacheNames(ms, CacheConfig.class);
        setCacheNames(ms, CachePut.class);
    }

    @Before("@annotation(org.springframework.cache.annotation.CacheEvict)")
    public void beforeCacheEvict(JoinPoint pjp) throws Throwable {
        MethodSignature ms = (MethodSignature) pjp.getSignature();
        setCacheNames(ms, CacheConfig.class);
        setCacheNames(ms, CacheEvict.class);
    }

    private void setCacheNames(MethodSignature ms, Class cls) {
        Annotation[] annotationsClass = ms.getDeclaringType().getAnnotationsByType(cls);
        Annotation[] annotationsMethod = ms.getMethod().getAnnotationsByType(cls);
        Set<Annotation> set=new HashSet<>();
        set.addAll(Arrays.asList(annotationsClass));
        set.addAll(Arrays.asList(annotationsMethod));

        Set<MyCache> setCache = new HashSet<>();
        Collection<? extends MyCache> caches = ((MyCacheManager) cacheManager).getCaches();
        setCache.addAll(caches);
        for (Annotation item : set) {
            if (item instanceof CacheConfig) {
                CacheConfig config = (CacheConfig) item;
                for (String s : config.cacheNames()) {
                    setCache.add(new MyCache(s));
                }
            } else if (item instanceof Cacheable) {
                Cacheable config = (Cacheable) item;
                String[] strings = config.cacheNames();
                String[] values = config.value();
                Set<String> nameSet =new HashSet<>();
                nameSet.addAll(Arrays.asList(strings));
                nameSet.addAll(Arrays.asList(values));
                for (String s : nameSet) {
                    setCache.add(new MyCache(s));
                }
            }
        }
        ((MyCacheManager) cacheManager).setCaches(setCache);
        ((MyCacheManager) cacheManager).initializeCaches();
    }

    // 優先執行
    @Override
    public int getOrder() {
        return -999;
    }

}

2.3、實現類Redis緩存

　　參看上圖，進行重新划分

　　 

 　　1、原生、繼承實現復用；

　　2、當前的緩存體系，即被包裝的緩存；

　　3、編碼開發的部分，要重新復寫一下Cache、CacheManager；

      4、忽略，springboot redis 配置類

開發步驟 

1、目前配置的有如下兩個：

maven-archetypes-webapp 默認模式，常用

maven-archetypes-webapp-cache 使用redis緩存，沒被包裝的，需要編寫service 代碼添加spring cache注解

現狀：公司統一包裝了緩存，如redis等，此時直接沒法使用，需要自定義。但是還想使用spring cache。

創建基礎項目

　　使用：maven-archetypes-webapp-cache 構建基礎項目

mvn archetype:generate \
-DgroupId=com.aaa.test -DartifactId=test-custom-cache-demo -Dversion=1.0.0-SNAPSHOT \
-DarchetypeGroupId=com.github.bjlhx15 -DarchetypeArtifactId=maven-archetypes-webapp-cache -DarchetypeVersion=0.0.2 \
-X -DarchetypeCatalog=local -DinteractiveMode=false

 

2、定義緩存

|--customcache
    |--CacheAspect
    |--CustomRedisCache
    |--CustomRedisCacheConfiguration
    |--CustomRedisCacheManager

 

　　如名稱為：CustomRedisCache繼承AbstractValueAdaptingCache，緩存實現

public class CustomRedisCache extends AbstractValueAdaptingCache {

    private final String name;
    private final CustomRedisDbService cacheService;
    private final RedisCacheConfiguration cacheConfig;
    private final ConversionService conversionService;


    public CustomRedisCache(String name, JimClientService cacheService) {
        super(true);
        this.name = name;
        this.cacheService = cacheService;
        this.cacheConfig = RedisCacheConfiguration.defaultCacheConfig();
        this.conversionService = cacheConfig.getConversionService();
    }

    //獲根據key取緩存,如果返回null，則要讀取持久層
    @Override
    protected Object lookup(Object key) {
        byte[] value = null;
        try {
            value = cacheService.hGet(name.getBytes("UTF-8"), createAndConvertCacheKey(key));
        } catch (UnsupportedEncodingException e) {
            e.printStackTrace();
        }
        if (value == null) {
            return null;
        }
        return deserializeCacheValue(value);
//        return value;
    }

    //獲取 緩存 空間名
    @Override
    public String getName() {
        return name;
    }

    @Override
    public Object getNativeCache() {
        return this.cacheService;
    }

    @Override
    public synchronized <T> T get(Object key, Callable<T> valueLoader) {
        ValueWrapper result = get(key);

        if (result != null) {
//            JSON.parseObject(result,T);
            return (T) result.get();
        }

        T value = valueFromLoader(key, valueLoader);
        put(key, value);
        return value;
    }

    //從持久層讀取value，然后存入緩存。允許value = null
    @Override
    public void put(Object key, Object value) {
        Object cacheValue = value;
        if (!isAllowNullValues() && cacheValue == null) {
            throw new IllegalArgumentException(String.format(
                    "Cache '%s' does not allow 'null' values. Avoid storing null via '@Cacheable(unless=\"#result == null\")' or configure RedisCache to allow 'null' via RedisCacheConfiguration.",
                    name));
        }
        try {
            cacheService.hSet(name.getBytes("UTF-8"), createAndConvertCacheKey(key), serializeCacheValue(value));
        } catch (UnsupportedEncodingException e) {
            e.printStackTrace();
        }
    }

    //如果傳入key對應的value已經存在，就返回存在的value，不進行替換。如果不存在，就添加key和value，返回null.
    @Override
    public ValueWrapper putIfAbsent(Object key, Object value) {
        Object cacheValue = value;

        if (!isAllowNullValues() && cacheValue == null) {
            return get(key);
        }

        Boolean result = null;
        try {
            result = cacheService.hSet(name.getBytes("UTF-8"), createAndConvertCacheKey(key), serializeCacheValue(cacheValue));
        } catch (UnsupportedEncodingException e) {
            e.printStackTrace();
        }


        if (result == null || result == false) {
            return null;
        }
        return get(key);
        //return new SimpleValueWrapper(fromStoreValue(deserializeCacheValue(result)));
    }

    @Override
    public void evict(Object key) {
        try {
            cacheService.hDel(name.getBytes("UTF-8"),createAndConvertCacheKey(key));
        } catch (UnsupportedEncodingException e) {
            e.printStackTrace();
        }
    }

    protected String convertKey(Object key) {

        TypeDescriptor source = TypeDescriptor.forObject(key);
        if (conversionService.canConvert(source, TypeDescriptor.valueOf(String.class))) {
            return conversionService.convert(key, String.class);
        }

        Method toString = ReflectionUtils.findMethod(key.getClass(), "toString");

        if (toString != null && !Object.class.equals(toString.getDeclaringClass())) {
            return key.toString();
        }

        throw new IllegalStateException(
                String.format("Cannot convert %s to String. Register a Converter or override toString().", source));
    }
    @Override
    public void clear() {
        cacheService.deleteRedisByKey(name);
    }

    protected String createCacheKey(Object key) {

        String convertedKey = convertKey(key);

        if (!cacheConfig.usePrefix()) {
            return convertedKey;
        }

        return prefixCacheKey(convertedKey);
    }
    protected byte[] serializeCacheKey(String cacheKey) {
        return ByteUtils.getBytes(cacheConfig.getKeySerializationPair().write(cacheKey));
    }

    private byte[] createAndConvertCacheKey(Object key) {
        return serializeCacheKey(createCacheKey(key));
    }

    @Nullable
    protected Object deserializeCacheValue(byte[] value) {
//        return null;//
         return cacheConfig.getValueSerializationPair().read(ByteBuffer.wrap(value));
    }

    protected byte[] serializeCacheValue(Object value) {
        return ByteUtils.getBytes(cacheConfig.getValueSerializationPair().write(value));
    }

    private static <T> T valueFromLoader(Object key, Callable<T> valueLoader) {
        try {
            return valueLoader.call();
        } catch (Exception e) {
            throw new ValueRetrievalException(key, valueLoader, e);
        }
    }

    private String prefixCacheKey(String key) {

        // allow contextual cache names by computing the key prefix on every call.
        return cacheConfig.getKeyPrefixFor(name) + key;
    }
}

　　緩存管理器 CustomRedisCacheManager

public class CustomRedisCacheManager extends AbstractCacheManager {
    private Collection<? extends CustomRedisCache> caches;

    /**
     * Specify the collection of Cache instances to use for this CacheManager.
     */
    public void setCaches(Collection<? extends CustomRedisCache> caches) {
        this.caches = caches;
    }

    @Override
    protected Collection<? extends CustomRedisCache> loadCaches() {
        return this.caches;
    }

    public Collection<? extends CustomRedisCache> getCaches() {
        return caches;
    }
}

　　配置類

@Configuration
public class CustomRedisCacheConfiguration {

    @Autowired
    private CustomRedisService cacheService;

    @Bean
    public CacheManager cacheManager() {
        CustomRedisCacheManager cacheManager = new CustomRedisCacheManager();
        List<CustomRedisCache> caches = new ArrayList<CustomRedisCache>();
//        caches.add(new JimDbCache("AccountBalance", cacheService));
        cacheManager.setCaches(caches);

        return cacheManager;
    }
}

　　其實此時這樣就可以使用了，但是每次增加命名空間，上述注釋的地方

　　此處使用注解，管理命名空間

@Aspect
@Component
public class CacheAspect implements Ordered {
    @Autowired
    private CacheManager cacheManager;
    @Autowired
    private CustomRedisDbService cacheService;

    @Before("@annotation(org.springframework.cache.annotation.Cacheable)")
    public void beforeCacheable(JoinPoint pjp) throws Throwable {
        MethodSignature ms = (MethodSignature) pjp.getSignature();
        setCacheNames(ms, CacheConfig.class);
        setCacheNames(ms, Cacheable.class);
    }

    @Before("@annotation(org.springframework.cache.annotation.CachePut)")
    public void beforeCachePut(JoinPoint pjp) throws Throwable {
        MethodSignature ms = (MethodSignature) pjp.getSignature();
        setCacheNames(ms, CacheConfig.class);
        setCacheNames(ms, CachePut.class);
    }

    @Before("@annotation(org.springframework.cache.annotation.CacheEvict)")
    public void beforeCacheEvict(JoinPoint pjp) throws Throwable {
        MethodSignature ms = (MethodSignature) pjp.getSignature();
        setCacheNames(ms, CacheConfig.class);
        setCacheNames(ms, CacheEvict.class);
    }

    private void setCacheNames(MethodSignature ms, Class cls) {
        Annotation[] annotationsClass = ms.getDeclaringType().getAnnotationsByType(cls);
        Annotation[] annotationsMethod = ms.getMethod().getAnnotationsByType(cls);
        Set<Annotation> set=new HashSet<>();
        set.addAll(Arrays.asList(annotationsClass));
        set.addAll(Arrays.asList(annotationsMethod));

        Set<CustomRedisCache> setCache = new HashSet<>();
        Collection<? extends CustomRedisCache> caches = ((CustomRedisCacheManager) cacheManager).getCaches();
        setCache.addAll(caches);
        for (Annotation item : set) {
            if (item instanceof CacheConfig) {
                CacheConfig config = (CacheConfig) item;
                for (String s : config.cacheNames()) {
                    setCache.add(new CustomRedisCache(s,cacheService));
                }
            } else if (item instanceof Cacheable) {
                Cacheable config = (Cacheable) item;
                String[] strings = config.cacheNames();
                String[] values = config.value();
                Set<String> nameSet =new HashSet<>();
                nameSet.addAll(Arrays.asList(strings));
                nameSet.addAll(Arrays.asList(values));
                for (String s : nameSet) {
                    setCache.add(new CustomRedisCache(s,cacheService));
                }
            }
        }
        ((CustomRedisCacheManager) cacheManager).setCaches(setCache);
        ((CustomRedisCacheManager) cacheManager).initializeCaches();
    }

    // 優先執行
    @Override
    public int getOrder() {
        return -999;
    }

} 

實現說明：

1、CustomRedisCache，redis實現中使用的是string類型，這里使用的是set類型

　　主要原因：刪除這里如果使用String類型，刪除使用的是pattern模式，找到每個集群每個點，使用模式刪除

　　String優點：1、使用Stringkey，不同key分布在不同節點，更加負載化，降低緩存雪崩概率，使用Set的話，key在一個節點。

參看：006-spring cache-緩存原理-SimpleCacheConfiguration、RedisCacheConfiguration原理實現

2、緩存空間加載方式

　　上述使用注解攔截器，加載緩存空間名

　　redis使用→RedisCacheConfiguration→CacheProperties，內使用：private List<String> cacheNames = new ArrayList<>();