為每個請求分配traceId的兩種方式及父子線程本地變量傳遞

 需求背景

　　有時候我們需要某個請求下的所有的traceId都是一致的，以獲得統一解析的日志文件。便於排查問題。

　　為每一個請求分配同一個traceId據我所知有兩種方式：MDC和ThreadLocal，MDC的內部實現也是ThreadLocal，下面分別介紹這兩種方式。

 

一、MDC

　　MDC(Mapped Diagnostic Contexts)，翻譯過來就是：映射的診斷上下文。意思是：在日志中（映射的）請求ID（requestId），可以作為我們定位（診斷）問題的關鍵字（上下文）。

　　有了MDC工具，只要在接口或切面植入 put 和 remove 代碼，就可以在定位問題時，根據映射的唯一 requestID 快速過濾出某次請求的所有日志。

　　另外，當客戶端請求到來時，可以將客戶端id，ip，請求參數等信息保存在MDC中，同時在logback.xml中配置，那么會自動在每個日志條目中包含這些信息。

　　slf4j的MDC機制其內部基於ThreadLocal實現，可參見ThreadLocal這篇博客：https://www.cnblogs.com/yangyongjie/p/10574591.html

　　MDC類基本原理其實非常簡單，其內部持有一個ThreadLocal實例，用於保存context數據，MDC提供了put/get/clear等幾個核心接口，用於操作ThreadLocal中的數據；ThreadLocal中的K-V，可以在logback.xml中聲明，即在layout中通過聲明“%X{Key}”來打印MDC中保存的此key對應的value在日志中。

　　在使用MDC時需要注意一些問題，這些問題通常也是ThreadLocal引起的，比如我們需要在線程退出之前清除（clear）MDC中的數據；在線程池中使用MDC時，那么需要在子線程退出之前清除數據；可以調用MDC.clear()方法。

　　MDC部分源碼：

package org.slf4j;
public class MDC {
  // 將上下文的值作為 MDC 的 key 放到線程上下的 map 中
  public static void put(String key, String val);
  // 通過 key 獲取上下文標識
  public static String get(String key);
  // 通過 key 移除上下文標識
  public static void remove(String key);
  // 清除 MDC 中所有的 entry
  public static void clear();
}

 

　1、請求沒有子線程的情況下代碼實現：

　　1）使用Aop攔截請求

/**
 * 為每一個的HTTP請求添加線程號
 *
 * @author yangyongjie
 * @date 2019/9/2
 * @desc
 */
@Aspect
@Component
public class LogAspect {

    private static final String STR_THREAD_ID = "threadId";

    @Pointcut(value = "@annotation(org.springframework.web.bind.annotation.RequestMapping)")
    private void webPointcut() {
        // doNothing
    }

    /**
     * 為所有的HTTP請求添加線程號
     *
     * @param joinPoint
     * @throws Throwable
     */
    @Around(value = "webPointcut()")
    public void around(ProceedingJoinPoint joinPoint) throws Throwable {
        // 方法執行前加上線程號
        MDC.put(STR_THREAD_ID, UUID.randomUUID().toString().replaceAll("-", ""));
        // 執行攔截的方法
        joinPoint.proceed();
        // 方法執行結束移除線程號
        MDC.remove(STR_THREAD_ID);
    }
}

　　2）log4j日志配置

log4j.appender.stdout.layout.ConversionPattern=[%-5p]%d{yyyy-MM-dd HH:mm:ss.SSS}[%t]%X{threadId}[%c:%L] - %m%n

　　需要注意日志紅色中字符串 threadId 需要和 日志攔截中MDC put的key是一樣的。

 

　2、請求有子線程的情況

　　slf4j的MDC機制其內部基於ThreadLocal實現，可參見Java基礎下的 ThreadLocal這篇博客，https://www.cnblogs.com/yangyongjie/p/10574591.html。所以我們調用 MDC.put()方法傳入

　　的請求ID只在當前線程有效。所以，主線程中設置的MDC數據，在其子線程（線程池）中是無法獲取的。那么主線程如何將MDC數據傳遞給子線程？　

　　官方建議

　　　　1）在父線程新建子線程之前調用MDC.getCopyOfContextMap()方法將MDC內容取出來傳給子線程

　　　　2）子線程在執行操作前先調用MDC.setContextMap()方法將父線程的MDC內容設置到子線程　

　　　　

　　代碼實現

　　1）使用Aop攔截請求，與上面相同

　　2）log4j日志配置與上面相同

　　3）裝飾器模式裝飾子線程，有兩種方式：

　　　　方式一：使用裝飾器模式，對Runnable接口進行一層裝飾，在創建MDCRunnable類對Runnable接口進行一層裝飾。

在創建MDCRunnable類時保存當前線程的MDC值，再執行run()方法

　　　　裝飾器MDCRunnable裝飾Runnable：

import org.slf4j.MDC;

import java.util.Map;

/**
 * 裝飾器模式裝飾Runnable，傳遞父線程的線程號
 *
 * @author yangyongjie
 * @date 2020/3/9
 * @desc
 */
public class MDCRunnable implements Runnable {

    private Runnable runnable;

    /**
     * 保存當前主線程的MDC值
     */
    private final Map<String, String> mainMdcMap;

    public MDCRunnable(Runnable runnable) {
        this.runnable = runnable;
        this.mainMdcMap = MDC.getCopyOfContextMap();
    }

    @Override
    public void run() {
        // 將父線程的MDC值賦給子線程
        for (Map.Entry<String, String> entry : mainMdcMap.entrySet()) {
            MDC.put(entry.getKey(), entry.getValue());
        }
        // 執行被裝飾的線程run方法
        runnable.run();
        // 執行結束移除MDC值
        for (Map.Entry<String, String> entry : mainMdcMap.entrySet()) {
            MDC.put(entry.getKey(), entry.getValue());
        }
    }

}

　　使用MDCRunnable代替Runnable：

        // 異步線程打印日志，用MDCRunnable裝飾Runnable
        new Thread(new MDCRunnable(new Runnable() {
            @Override
            public void run() {
                logger.debug("log in other thread");
            }
        })).start();

        // 異步線程池打印日志，用MDCRunnable裝飾Runnable
        EXECUTOR.execute(new MDCRunnable(new Runnable() {
            @Override
            public void run() {
                logger.debug("log in other thread pool");
            }
        }));
        EXECUTOR.shutdown();

 

　　　　方式二：裝飾線程池

/**
 *  裝飾ThreadPoolExecutor，將父線程的MDC內容傳給子線程
 * @author yangyongjie
 * @date 2020/3/19
 * @desc
 */
public class MDCThreadPoolExecutor extends ThreadPoolExecutor {

    private static final Logger LOGGER= LoggerFactory.getLogger(MDCThreadPoolExecutor.class);

    public MDCThreadPoolExecutor(int corePoolSize, int maximumPoolSize, long keepAliveTime, TimeUnit unit, BlockingQueue<Runnable> workQueue, ThreadFactory threadFactory, RejectedExecutionHandler handler) {
        super(corePoolSize, maximumPoolSize, keepAliveTime, unit, workQueue, threadFactory, handler);
    }

    @Override
    public void execute(final Runnable runnable) {
        // 獲取父線程MDC中的內容，必須在run方法之前，否則等異步線程執行的時候有可能MDC里面的值已經被清空了，這個時候就會返回null
        final Map<String, String> context = MDC.getCopyOfContextMap();
        super.execute(new Runnable() {
            @Override
            public void run() {
                // 將父線程的MDC內容傳給子線程
                MDC.setContextMap(context);
                try {
                    // 執行異步操作
                    runnable.run();
                } finally {
                    // 清空MDC內容
                    MDC.clear();
                }
            }
        });
    }
}

　　用MDCThreadPoolExecutor 代替ThreadPoolExecutor ：

private static final MDCThreadPoolExecutor MDCEXECUTORS=new MDCThreadPoolExecutor(1,10,60,TimeUnit.SECONDS,new LinkedBlockingQueue<Runnable>(600), new CustomThreadFactory("mdcThreadPoolTest"), new RejectedExecutionHandler() {
            @Override
            public void rejectedExecution(Runnable r, ThreadPoolExecutor executor) {
                // 打印日志，並且重啟一個線程執行被拒絕的任務
                LOGGER.error("Task：{},rejected from：{}", r.toString(), executor.toString());
                // 直接執行被拒絕的任務，JVM另起線程執行
                r.run();
            }
        });

        LOGGER.info("父線程日志");
        MDCEXECUTORS.execute(new Runnable() {
            @Override
            public void run() {
                LOGGER.info("子線程日志");
            }
        });

 

二、ThreadLocal方式

ThreadLocal可以用於在同一個線程內，跨類、跨方法傳遞數據。因此可以用來透傳全局上下文

　1、沒有子線程的情況

　　1）創建線程的請求上下文

/**
 * 線程上下文，一個線程內所需的上下文變量參數，使用ThreadLocal保存副本
 *
 * @author yangyongjie
 * @date 2019/9/12
 * @desc
 */
public class ThreadContext {
    /**
     * 每個線程的私有變量，每個線程都有獨立的變量副本，所以使用private static final修飾，因為都需要復制進入本地線程
     */
    private static final ThreadLocal<ThreadContext> THREAD_LOCAL = new ThreadLocal<ThreadContext>() {
        @Override
        protected ThreadContext initialValue() {
            return new ThreadContext();
        }
    };

    public static ThreadContext currentThreadContext() {
        /*ThreadContext threadContext = THREAD_LOCAL.get();
        if (threadContext == null) {
            THREAD_LOCAL.set(new ThreadContext());
            threadContext = THREAD_LOCAL.get();
        }
        return threadContext;*/
        return THREAD_LOCAL.get();
    }

    public static void remove() {
        THREAD_LOCAL.remove();
    }

    private String threadId;

    public String getThreadId() {
        return threadId;
    }

    public void setThreadId(String threadId) {
        this.threadId = threadId;
    }

    @Override
    public String toString() {
        return JacksonJsonUtil.toString(this);
    }
}

　　2）使用Aop攔截請求，給每個請求線程ThreadLocalMap添加本地變量ThreadContext 對象並為對象的threadId屬性賦值。

/**
 * 為每一個的HTTP請求添加線程號
 *
 * @author yangyongjie
 * @date 2019/9/2
 * @desc
 */
@Aspect
@Component
public class LogAspect {

    private static final Logger LOGGER = LoggerFactory.getLogger(LogAspect.class);

    @Pointcut(value = "@annotation(org.springframework.web.bind.annotation.RequestMapping)")
    private void webPointcut() {
        // doNothing
    }

    /**
     * 為所有的HTTP請求添加線程號
     *
     * @param joinPoint
     * @throws Throwable
     */
    @Around(value = "webPointcut()")
    public Object around(ProceedingJoinPoint joinPoint) throws Throwable {
        // 方法執行前加上線程號，並將線程號放到線程本地變量中
        ThreadContext.currentThreadContext().setThreadId(StringUtil.uuid());
        // 執行攔截的方法
        Object result;
        try {
            result = joinPoint.proceed();
        } finally {
            // 方法執行結束移除線程號，並移除線程本地變量,防止內存泄漏
            ThreadContext.remove();
        }
        return result;
    }
}

　　3）獲取線程號

String threadId = ThreadContext.currentThreadContext().getThreadId();

 

　2、請求有子線程的情況

　　首先了解一下InheritableThreadLocal<T>，它是對ThreadLocal<T> 的擴展和繼承，它的數據ThreadLocal.ThreadLocalMap保存在Thread的inheritableThreadLocals變量中，同時如果我們在當前線程開啟一個新線程，而且當前線程存在inheritableThreadLocals變量，那么子線程會copy一份當前線程（父線程）中的這個變量持有的值。

　　源碼：

　　Thread類的inheritableThreadLocals 屬性：

 ThreadLocal.ThreadLocalMap inheritableThreadLocals = null;

　　Thread的構造方法會調用init方法，而init方法的inheritableThreadLocals參數默認為true：

    public Thread() {
        init(null, null, "Thread-" + nextThreadNum(), 0);
    }

    public Thread(Runnable target) {
        init(null, target, "Thread-" + nextThreadNum(), 0);
    }

     private void init(ThreadGroup g, Runnable target, String name,
                      long stackSize) {
        init(g, target, name, stackSize, null, true);
    }

　　那么如果當前線程的inheritableThreadLocals變量不為空，就可以將當前線程的變量繼續往下傳遞給它創建的子線程：

private void init(ThreadGroup g, Runnable target, String name,
                      long stackSize, AccessControlContext acc,
                      boolean inheritThreadLocals) {
        if (name == null) {
            throw new NullPointerException("name cannot be null");
        }

        this.name = name;

        Thread parent = currentThread();
        SecurityManager security = System.getSecurityManager();
        if (g == null) {
            /* Determine if it's an applet or not */

            /* If there is a security manager, ask the security manager
               what to do. */
            if (security != null) {
                g = security.getThreadGroup();
            }

            /* If the security doesn't have a strong opinion of the matter
               use the parent thread group. */
            if (g == null) {
                g = parent.getThreadGroup();
            }
        }

        /* checkAccess regardless of whether or not threadgroup is
           explicitly passed in. */
        g.checkAccess();

        /*
         * Do we have the required permissions?
         */
        if (security != null) {
            if (isCCLOverridden(getClass())) {
                security.checkPermission(SUBCLASS_IMPLEMENTATION_PERMISSION);
            }
        }

        g.addUnstarted();

        this.group = g;
        this.daemon = parent.isDaemon();
        this.priority = parent.getPriority();
        if (security == null || isCCLOverridden(parent.getClass()))
            this.contextClassLoader = parent.getContextClassLoader();
        else
            this.contextClassLoader = parent.contextClassLoader;
        this.inheritedAccessControlContext =
                acc != null ? acc : AccessController.getContext();
        this.target = target;
        setPriority(priority);
        if (inheritThreadLocals && parent.inheritableThreadLocals != null)
            this.inheritableThreadLocals =
                ThreadLocal.createInheritedMap(parent.inheritableThreadLocals);
        /* Stash the specified stack size in case the VM cares */
        this.stackSize = stackSize;

        /* Set thread ID */
        tid = nextThreadID();
    }

這里是重點，當inheritThreadLocals 參數為true，且創建此線程的線程即parent（父線程）的inheritableThreadLocals不為空的時候，就會在創建當前線程的時候將父線程的 inheritableThreadLocals 復制到 此新建的子線程。                                    

if (inheritThreadLocals && parent.inheritableThreadLocals != null)
            this.inheritableThreadLocals =
                ThreadLocal.createInheritedMap(parent.inheritableThreadLocals);

createInheritedMap()方法就是調用ThreadLocalMap的私有構造方法來產生一個實例對象，把父線程的不為null的線程變量都拷貝過來：

    /**
     * Factory method to create map of inherited thread locals.
     * Designed to be called only from Thread constructor.
     *
     * @param  parentMap the map associated with parent thread
     * @return a map containing the parent's inheritable bindings
     */
    static ThreadLocalMap createInheritedMap(ThreadLocalMap parentMap) {
        return new ThreadLocalMap(parentMap);
    }

        /**
         * Construct a new map including all Inheritable ThreadLocals
         * from given parent map. Called only by createInheritedMap.
         *
         * @param parentMap the map associated with parent thread.
         */
        private ThreadLocalMap(ThreadLocalMap parentMap) {
            Entry[] parentTable = parentMap.table;
            int len = parentTable.length;
            setThreshold(len);
            table = new Entry[len];

            for (int j = 0; j < len; j++) {
                Entry e = parentTable[j];
                if (e != null) {
                    @SuppressWarnings("unchecked")
                    ThreadLocal<Object> key = (ThreadLocal<Object>) e.get();
                    if (key != null) {
                        Object value = key.childValue(e.value);
                        Entry c = new Entry(key, value);
                        int h = key.threadLocalHashCode & (len - 1);
                        while (table[h] != null)
                            h = nextIndex(h, len);
                        table[h] = c;
                        size++;
                    }
                }
            }
        }

 

實際應用代碼實現：

/**
 * 線程上下文，一個線程內所需的上下文變量參數，使用InheritableThreadLocal保存副本，可以將副本傳遞給子線程
 * @author yangyongjie
 * @date 2020/3/20
 * @desc
 */
public class InheritableThreadContext {

    private static final InheritableThreadLocal<ThreadContext> INHERITABLE_THREAD_LOCAL = new InheritableThreadLocal<ThreadContext>() {
        @Override
        protected ThreadContext initialValue() {
            return new ThreadContext();
        }
    };

    public static ThreadContext currentThreadContext() {
        /*ThreadContext threadContext = INHERITABLE_THREAD_LOCAL.get();
        if (threadContext == null) {
            INHERITABLE_THREAD_LOCAL.set(new ThreadContext());
            threadContext = INHERITABLE_THREAD_LOCAL.get();
        }
        return threadContext;*/
        return INHERITABLE_THREAD_LOCAL.get();
    }

    public static void remove() {
        INHERITABLE_THREAD_LOCAL.remove();
    }

    private String threadId;

    public String getThreadId() {
        return threadId;
    }

    public void setThreadId(String threadId) {
        this.threadId = threadId;
    }

    @Override
    public String toString() {
        return JacksonJsonUtil.toString(this);
    }
}

使用示例：

        String uuid = UUID.randomUUID().toString().replaceAll("-", "");
        // 當前線程的本地變量（為當前線程的inheritableThreadLocals屬性賦值）
        InheritableThreadContext.currentThreadContext().setThreadId(uuid);
        LOGGER.info("[{}]父線程日志",InheritableThreadContext.currentThreadContext().getThreadId());
        // 創建子線程
        new Thread(new Runnable() {
            @Override
            public void run() {
                LOGGER.info("[{}]子線程日志",InheritableThreadContext.currentThreadContext().getThreadId());
                // 移除子線程本地變量
                InheritableThreadContext.remove();
            }
        }).start();
        // 線程池中的線程也是線程工廠通過new Thread創建的
        EXECUTORS.execute(new Runnable() {
            @Override
            public void run() {
                LOGGER.info("[{}]線程池子線程日志",InheritableThreadContext.currentThreadContext().getThreadId());
                // 移除子線程本地變量
                InheritableThreadContext.remove();
            }
        });
        // 移除父線程本地變量
        InheritableThreadContext.remove();

運行結果：

[INFO ]2020-03-20 13:53:24.056[main] - [4eb5063b1dd84448807cb94f7d4df87a]父線程日志
[INFO ]2020-03-20 13:53:26.133[Thread-8] - [4eb5063b1dd84448807cb94f7d4df87a]子線程日志
[INFO ]2020-03-20 13:53:26.137[From CustomThreadFactory-inheritable-worker-1] - [4eb5063b1dd84448807cb94f7d4df87a]線程池子線程日志

不使用InheritableThreadLocal而使用ThreadLocal的結果（將上面使用示例中的InheritableThreadContext替換為ThreadContext）：

[INFO ]2020-03-20 13:57:54.592[main] - [926f164b97914d29a3638c945c125d44]父線程日志
[INFO ]2020-03-20 13:57:56.214[Thread-8] - [null]子線程日志
[INFO ]2020-03-20 13:57:56.215[From CustomThreadFactory-worker-1] - [null]線程池子線程日志

 

注意：使用ThreadLocal和InheritableThreadLocal透傳上下文時，需要注意線程間切換、異常傳輸時的處理，避免在傳輸過程中因處理不當而導致的上下文丟失。

 

Spring中對InheritableThreadLocal的使用：

　　Spring中對一個request范圍的數據進行傳遞時，如當有請求到來需要將 HttpServletRequest的值進行傳遞，用的就是InheritableThreadLocal。

　　Spring的RequestContextHolder部分源碼：

public abstract class RequestContextHolder  {

    private static final ThreadLocal<RequestAttributes> requestAttributesHolder =
            new NamedThreadLocal<RequestAttributes>("Request attributes");

    private static final ThreadLocal<RequestAttributes> inheritableRequestAttributesHolder =
            new NamedInheritableThreadLocal<RequestAttributes>("Request context");


    /**
     * Reset the RequestAttributes for the current thread.
     */
    public static void resetRequestAttributes() {
        requestAttributesHolder.remove();
        inheritableRequestAttributesHolder.remove();
    }

    /**
     * Bind the given RequestAttributes to the current thread,
     * <i>not</i> exposing it as inheritable for child threads.
     * @param attributes the RequestAttributes to expose
     * @see #setRequestAttributes(RequestAttributes, boolean)
     */
    public static void setRequestAttributes(RequestAttributes attributes) {
        setRequestAttributes(attributes, false);
    }

    /**
     * Bind the given RequestAttributes to the current thread.
     * @param attributes the RequestAttributes to expose,
     * or {@code null} to reset the thread-bound context
     * @param inheritable whether to expose the RequestAttributes as inheritable
     * for child threads (using an {@link InheritableThreadLocal})
     */
    public static void setRequestAttributes(RequestAttributes attributes, boolean inheritable) {
        if (attributes == null) {
            resetRequestAttributes();
        }
        else {
            if (inheritable) {
                inheritableRequestAttributesHolder.set(attributes);
                requestAttributesHolder.remove();
            }
            else {
                requestAttributesHolder.set(attributes);
                inheritableRequestAttributesHolder.remove();
            }
        }
    }

    /**
     * Return the RequestAttributes currently bound to the thread.
     * @return the RequestAttributes currently bound to the thread,
     * or {@code null} if none bound
     */
    public static RequestAttributes getRequestAttributes() {
        RequestAttributes attributes = requestAttributesHolder.get();
        if (attributes == null) {
            attributes = inheritableRequestAttributesHolder.get();
        }
        return attributes;
    }
}

NamedInheritableThreadLocal繼承了InheritableThreadLocal ，添加了一個name屬性：

public class NamedInheritableThreadLocal<T> extends InheritableThreadLocal<T> {

    private final String name;


    /**
     * Create a new NamedInheritableThreadLocal with the given name.
     * @param name a descriptive name for this ThreadLocal
     */
    public NamedInheritableThreadLocal(String name) {
        Assert.hasText(name, "Name must not be empty");
        this.name = name;
    }

    @Override
    public String toString() {
        return this.name;
    }

}

從RequestContextListener可以看到當請求事件到來時，需要將HttpServletRequest的值通過RequestContextHolder進行傳遞：

public class RequestContextListener implements ServletRequestListener {
    ...
    public void requestInitialized(ServletRequestEvent requestEvent) {
        if (!(requestEvent.getServletRequest() instanceof HttpServletRequest)) {
            throw new IllegalArgumentException(
                    "Request is not an HttpServletRequest: " + requestEvent.getServletRequest());
        }
        HttpServletRequest request = (HttpServletRequest) requestEvent.getServletRequest();
        ServletRequestAttributes attributes = new ServletRequestAttributes(request);
        request.setAttribute(REQUEST_ATTRIBUTES_ATTRIBUTE, attributes);
        LocaleContextHolder.setLocale(request.getLocale());
        RequestContextHolder.setRequestAttributes(attributes);
    }
    ...
}

 

 

　總結

　　1）若是日志線程號需要通過線程上下文傳遞時，使用MDC加日志文件配置的方式；有子線程時，使用MDCRunnable裝飾Runnable

　　2）若是用戶信息需要隨着請求上下文傳遞時，使用ThreadLocal的方式；有子線程時，使用InheritableThreadLocal

　　3）若是上面兩個信息都需要傳遞，那就結合一起使用。日志使用MDC，其他信息使用ThreadLocal。

 

END