前言
最近做的一個項目因為安全審計需要，需要做安全改造。其中自然就包括XSS和CSRF漏洞安全整改。關於這兩個網絡安全漏洞的詳細說明，可以參照我本篇博客最后的參考鏈接。當然，我這里並不是想寫一篇安全方面的專題。我要講的是在做了XSS漏洞修復之后引發的一系列事件。
超時
本地測試的時候隨便點了些頁面，然后debug跟了下代碼未發現任何問題。上線之后用戶反饋有的頁面打不開，自己去線上體驗發現大部分頁面正常，但是存在部分客戶反饋的頁面打開直接超時報錯。
事件緊急處理
XSS這個漏洞修復開始不是經過我處理的，上線之后由於編碼規則太嚴格（后面我會講我們使用的解決方案），導致前台傳入的JSON字符串中的引號全被轉碼，造成后台解析報錯。
而我成為了那個救(bei)火(guo)英(xia)雄，需要立馬解決這個問題補丁升級。我看了下以前實現的代碼，有考慮到通過一個XML白名單文件，配置忽略XSS編碼的請求URI。最直接的辦法，是直接把我這個請求的URI加入XML白名單配置，然后補丁替換白名單文件重啟服務。
但是當時我在修改的時候，考慮到可能不止這一個需要過濾的白名單，如果純粹啟動時加載的XML白名單列表。到時候還有別的URI需要忽略，那我豈不是還要再發增量補丁......
於是當時我修改的時候順便增加了一個能力，白名單可以直接在界面配置，並且每次獲取白名單列表的時候動態從數據庫獲取（考慮到實時請求較大，我調用的系統已有接口提供的支持緩存的查詢方法）。正因為有這個“后門”我直接在線上配置了這個參數，先暫且把線上問題解決。
問題探索
解決問題第一步，自然是分析線上日志。發現線上日志的確對請求的URI中的參數做了XSS編碼處理。
那問題就回到我們XSS漏洞修復的實現方式:AntiSamy(見參考鏈接)。其中我們的XssRequestWrapper 源碼如下：
public class XssRequestWrapper extends HttpServletRequestWrapper {
private static Logger log = LoggerFactory.getLogger(XssRequestWrapper.class);
private static Policy policy = null;
static {
String path = XssRequestWrapper.class.getClassLoader().getResource("security/antisamy-tinymce.xml").getFile();
log.info("policy_filepath:" + path);
if (path.startsWith("file")) {
//以file:開頭
path = path.substring(6);
}
try {
policy = Policy.getInstance(path);
} catch (PolicyException e) {
e.printStackTrace();
}
}
public XssRequestWrapper(HttpServletRequest request) {
super(request);
}
// 隊請求參數進行安全轉碼
public String getParameter(String paramString) {
String str = super.getParameter(paramString);
if (StringUtils.isBlank(str)) {
return null;
}
return xssClean(str, paramString);
}
// 隊請求頭進行安全轉碼
public String getHeader(String paramString) {
String str = super.getHeader(paramString);
if (StringUtils.isBlank(str))
return null;
return xssClean(str, paramString);
}
@SuppressWarnings({"rawtypes","unchecked"})
public Map<String, String[]> getParameterMap() {
Map<String, String[]> request_map = super.getParameterMap();
Iterator iterator = request_map.entrySet().iterator();
log.debug("getParameterMap size:{}", request_map.size());
while (iterator.hasNext()) {
Map.Entry me = (Map.Entry) iterator.next();
String paramsKey = (String) me.getKey();
String[] values = (String[]) me.getValue();
for (int i = 0; i < values.length; i++) {
values[i] = xssClean(values[i], paramsKey);
}
}
return request_map;
}
public String[] getParameterValues(String paramString) {
String[] arrayOfString1 = super.getParameterValues(paramString);
if (arrayOfString1 == null)
return null;
int i = arrayOfString1.length;
String[] arrayOfString2 = new String[i];
for (int j = 0; j < i; j++)
arrayOfString2[j] = xssClean(arrayOfString1[j], paramString);
return arrayOfString2;
}
public final static String KEY_FILTER_STR = "'";
public final static String SUFFIX = "value";
//需要過濾的地方
private String xssClean(String value, String paramsKey) {
String keyFilterStr = KEY_FILTER_STR;
String param = paramsKey.toLowerCase();
if (param.endsWith(SUFFIX)) { // 如果參數名 name="xxxvalue"
if (value.contains(keyFilterStr)) {
value = value.replace(keyFilterStr, "‘");
}
}
AntiSamy antiSamy = new AntiSamy();
try {
final CleanResults cr = antiSamy.scan(value, policy);
// 安全的HTML輸出
return cr.getCleanHTML();
} catch (ScanException e) {
log.error("antiSamy scan error", e);
} catch (PolicyException e) {
log.error("antiSamy policy error", e);
}
return value;
}
}
可以看到了我們項目組最終采用的策略配置文件是:antisamy-tinymce.xml，這種策略只允許傳送純文本到后台（這樣做真的好嗎？個人覺得這個規則太過嚴格）,並且對請求頭和請求參數都做了XSS轉碼。請注意這里，我們相對於參考鏈接中源碼不同的處理方式在於：我們對請求頭也進行了編碼處理。
那么看來問題就在於編碼導致的效率低下，於是我在getHeader和getParameter方法中都打了斷點。在揭曉結果之前，我說說我當時的猜測：因為當時用戶反饋的有問題的頁面是有很多查詢條件的，我開始的猜測是應該是傳入后台的參數過多導致編碼影響效率。然而，現實總是無情地打臉，不管你天真不天真。
Debug發現getParameter調用的此時算正常，而getHeader處的斷點沒完沒了的進來（最終結果證明，進來了幾千次）......
原因分析
還是那句話，沒有什么是源碼解決不了的，如果有，那么請Debug源碼。😄
我們項目是傳統的SpringMVC項目，那么當然我們要從org.springframework.web.servlet.DispatcherServlet入手了。DispatcherServlet其實也是一個Servlet，他的繼承關系如下:
DispatcherServlet extends FrameworkServlet
FrameworkServlet extends HttpServletBean implements ApplicationContextAware
HttpServletBean extends HttpServlet implements EnvironmentCapable, EnvironmentAware
HttpServlet extends GenericServlet
GenericServlet
implements Servlet, ServletConfig, java.io.Serializable
可以看到，實際上SpringMVC的DispatcherServlet最終也是通過doGet和doPost來對請求進行轉發，而最終其實都到了DispatcherServlet的doService.該方法的源碼如下:
/**
* Exposes the DispatcherServlet-specific request attributes and delegates to {@link #doDispatch}
* for the actual dispatching.
*/
@Override
protected void doService(HttpServletRequest request, HttpServletResponse response) throws Exception {
if (logger.isDebugEnabled()) {
String resumed = WebAsyncUtils.getAsyncManager(request).hasConcurrentResult() ? " resumed" : "";
logger.debug("DispatcherServlet with name '" + getServletName() + "'" + resumed +
" processing " + request.getMethod() + " request for [" + getRequestUri(request) + "]");
}
// Keep a snapshot of the request attributes in case of an include,
// to be able to restore the original attributes after the include.
Map<String, Object> attributesSnapshot = null;
if (WebUtils.isIncludeRequest(request)) {
attributesSnapshot = new HashMap<String, Object>();
Enumeration<?> attrNames = request.getAttributeNames();
while (attrNames.hasMoreElements()) {
String attrName = (String) attrNames.nextElement();
if (this.cleanupAfterInclude || attrName.startsWith("org.springframework.web.servlet")) {
attributesSnapshot.put(attrName, request.getAttribute(attrName));
}
}
}
// Make framework objects available to handlers and view objects.
request.setAttribute(WEB_APPLICATION_CONTEXT_ATTRIBUTE, getWebApplicationContext());
request.setAttribute(LOCALE_RESOLVER_ATTRIBUTE, this.localeResolver);
request.setAttribute(THEME_RESOLVER_ATTRIBUTE, this.themeResolver);
request.setAttribute(THEME_SOURCE_ATTRIBUTE, getThemeSource());
FlashMap inputFlashMap = this.flashMapManager.retrieveAndUpdate(request, response);
if (inputFlashMap != null) {
request.setAttribute(INPUT_FLASH_MAP_ATTRIBUTE, Collections.unmodifiableMap(inputFlashMap));
}
request.setAttribute(OUTPUT_FLASH_MAP_ATTRIBUTE, new FlashMap());
request.setAttribute(FLASH_MAP_MANAGER_ATTRIBUTE, this.flashMapManager);
try {
// 重點在這，都會進入doDispatch方法
doDispatch(request, response);
}
finally {
if (!WebAsyncUtils.getAsyncManager(request).isConcurrentHandlingStarted()) {
// Restore the original attribute snapshot, in case of an include.
if (attributesSnapshot != null) {
restoreAttributesAfterInclude(request, attributesSnapshot);
}
}
}
}
可以看到doService方法，最終還是會進入到doDispatch中，該方法的源碼如下:
/**
* Process the actual dispatching to the handler.
* <p>The handler will be obtained by applying the servlet's HandlerMappings in order.
* The HandlerAdapter will be obtained by querying the servlet's installed HandlerAdapters
* to find the first that supports the handler class.
* <p>All HTTP methods are handled by this method. It's up to HandlerAdapters or handlers
* themselves to decide which methods are acceptable.
* @param request current HTTP request
* @param response current HTTP response
* @throws Exception in case of any kind of processing failure
*/
protected void doDispatch(HttpServletRequest request, HttpServletResponse response) throws Exception {
HttpServletRequest processedRequest = request;
HandlerExecutionChain mappedHandler = null;
boolean multipartRequestParsed = false;
WebAsyncManager asyncManager = WebAsyncUtils.getAsyncManager(request);
try {
ModelAndView mv = null;
Exception dispatchException = null;
try {
processedRequest = checkMultipart(request);
multipartRequestParsed = (processedRequest != request);
// Determine handler for the current request.
// 根據請求獲取處理的Handler
mappedHandler = getHandler(processedRequest);
if (mappedHandler == null || mappedHandler.getHandler() == null) {
noHandlerFound(processedRequest, response);
return;
}
// Determine handler adapter for the current request.
HandlerAdapter ha = getHandlerAdapter(mappedHandler.getHandler());
// Process last-modified header, if supported by the handler.
String method = request.getMethod();
boolean isGet = "GET".equals(method);
if (isGet || "HEAD".equals(method)) {
long lastModified = ha.getLastModified(request, mappedHandler.getHandler());
if (logger.isDebugEnabled()) {
logger.debug("Last-Modified value for [" + getRequestUri(request) + "] is: " + lastModified);
}
if (new ServletWebRequest(request, response).checkNotModified(lastModified) && isGet) {
return;
}
}
if (!mappedHandler.applyPreHandle(processedRequest, response)) {
return;
}
// Actually invoke the handler.
mv = ha.handle(processedRequest, response, mappedHandler.getHandler());
if (asyncManager.isConcurrentHandlingStarted()) {
return;
}
applyDefaultViewName(processedRequest, mv);
mappedHandler.applyPostHandle(processedRequest, response, mv);
}
catch (Exception ex) {
dispatchException = ex;
}
catch (Throwable err) {
// As of 4.3, we're processing Errors thrown from handler methods as well,
// making them available for @ExceptionHandler methods and other scenarios.
dispatchException = new NestedServletException("Handler dispatch failed", err);
}
processDispatchResult(processedRequest, response, mappedHandler, mv, dispatchException);
}
catch (Exception ex) {
triggerAfterCompletion(processedRequest, response, mappedHandler, ex);
}
catch (Throwable err) {
triggerAfterCompletion(processedRequest, response, mappedHandler,
new NestedServletException("Handler processing failed", err));
}
finally {
if (asyncManager.isConcurrentHandlingStarted()) {
// Instead of postHandle and afterCompletion
if (mappedHandler != null) {
mappedHandler.applyAfterConcurrentHandlingStarted(processedRequest, response);
}
}
else {
// Clean up any resources used by a multipart request.
if (multipartRequestParsed) {
cleanupMultipart(processedRequest);
}
}
}
}
這段處理的重點也在我中文注釋的地方，我們繼續跟進getHandler方法：
protected HandlerExecutionChain getHandler(HttpServletRequest request) throws Exception {
// 遍歷HandlerMapping，找到請求對應的Handler具體信息
for (HandlerMapping hm : this.handlerMappings) {
if (logger.isTraceEnabled()) {
logger.trace(
"Testing handler map [" + hm + "] in DispatcherServlet with name '" + getServletName() + "'");
}
HandlerExecutionChain handler = hm.getHandler(request);
if (handler != null) {
return handler;
}
}
return null;
}
// 最終會到AbstractHandlerMethodMapping#getHandlerInternal
protected HandlerMethod getHandlerInternal(HttpServletRequest request) throws Exception {
// 分析請求URI （去掉Context）
String lookupPath = getUrlPathHelper().getLookupPathForRequest(request);
if (logger.isDebugEnabled()) {
logger.debug("Looking up handler method for path " + lookupPath);
}
this.mappingRegistry.acquireReadLock();
try {
// 重頭戲： 根據請求URI去找對應的處理器 （具體到類和方法）
HandlerMethod handlerMethod = lookupHandlerMethod(lookupPath, request);
if (logger.isDebugEnabled()) {
if (handlerMethod != null) {
logger.debug("Returning handler method [" + handlerMethod + "]");
}
else {
logger.debug("Did not find handler method for [" + lookupPath + "]");
}
}
return (handlerMethod != null ? handlerMethod.createWithResolvedBean() : null);
}
finally {
this.mappingRegistry.releaseReadLock();
}
}
注意其中的中文注釋，我們通過解析請求中的URI，然后根據請求URI去查找對應的處理器，也就是進行適配的關鍵一步：
/**
* Look up the best-matching handler method for the current request.
* If multiple matches are found, the best match is selected.
* @param lookupPath mapping lookup path within the current servlet mapping
* @param request the current request
* @return the best-matching handler method, or {@code null} if no match
* @see #handleMatch(Object, String, HttpServletRequest)
* @see #handleNoMatch(Set, String, HttpServletRequest)
*/
protected HandlerMethod lookupHandlerMethod(String lookupPath, HttpServletRequest request) throws Exception {
List<Match> matches = new ArrayList<Match>();
// 直接根據請求的URI去找對應的處理類（此時前台請求URI必須與后台注解配置的RequestMapping完全一致）
List<T> directPathMatches = this.mappingRegistry.getMappingsByUrl(lookupPath);
if (directPathMatches != null) {
addMatchingMappings(directPathMatches, matches, request);
}
if (matches.isEmpty()) {
// 注意這里： SpringMVC做了個無奈的容錯處理，如果沒有完全匹配的話，就遍歷所有請求URI找到大致匹配的 --- 這也是本次問題出現的原因
// No choice but to go through all mappings...
addMatchingMappings(this.mappingRegistry.getMappings().keySet(), matches, request);
}
if (!matches.isEmpty()) {
Comparator<Match> comparator = new MatchComparator(getMappingComparator(request));
Collections.sort(matches, comparator);
if (logger.isTraceEnabled()) {
logger.trace("Found " + matches.size() + " matching mapping(s) for [" +
lookupPath + "] : " + matches);
}
Match bestMatch = matches.get(0);
if (matches.size() > 1) {
if (CorsUtils.isPreFlightRequest(request)) {
return PREFLIGHT_AMBIGUOUS_MATCH;
}
Match secondBestMatch = matches.get(1);
if (comparator.compare(bestMatch, secondBestMatch) == 0) {
Method m1 = bestMatch.handlerMethod.getMethod();
Method m2 = secondBestMatch.handlerMethod.getMethod();
throw new IllegalStateException("Ambiguous handler methods mapped for HTTP path '" +
request.getRequestURL() + "': {" + m1 + ", " + m2 + "}");
}
}
handleMatch(bestMatch.mapping, lookupPath, request);
return bestMatch.handlerMethod;
}
else {
return handleNoMatch(this.mappingRegistry.getMappings().keySet(), lookupPath, request);
}
}
其實到這里，我們已經找到真相了，但是為什么循環遍歷請求URI會導致getHeader方法超頻調用呢？我們繼續跟進：
private void addMatchingMappings(Collection<T> mappings, List<Match> matches, HttpServletRequest request) {
for (T mapping : mappings) {
T match = getMatchingMapping(mapping, request);
if (match != null) {
matches.add(new Match(match, this.mappingRegistry.getMappings().get(mapping)));
}
}
}
// RequestMappingInfoHandlerMapping
/**
* Check if the given RequestMappingInfo matches the current request and
* return a (potentially new) instance with conditions that match the
* current request -- for example with a subset of URL patterns.
* @return an info in case of a match; or {@code null} otherwise.
*/
@Override
protected RequestMappingInfo getMatchingMapping(RequestMappingInfo info, HttpServletRequest request) {
return info.getMatchingCondition(request);
}
// RequestMappingInfo
/**
* Checks if all conditions in this request mapping info match the provided request and returns
* a potentially new request mapping info with conditions tailored to the current request.
* <p>For example the returned instance may contain the subset of URL patterns that match to
* the current request, sorted with best matching patterns on top.
* @return a new instance in case all conditions match; or {@code null} otherwise
*/
@Override
public RequestMappingInfo getMatchingCondition(HttpServletRequest request) {
RequestMethodsRequestCondition methods = this.methodsCondition.getMatchingCondition(request);
ParamsRequestCondition params = this.paramsCondition.getMatchingCondition(request);
// 問題就在於headersCondition的 getMatchingCondition 方法的調用
HeadersRequestCondition headers = this.headersCondition.getMatchingCondition(request);
ConsumesRequestCondition consumes = this.consumesCondition.getMatchingCondition(request);
ProducesRequestCondition produces = this.producesCondition.getMatchingCondition(request);
if (methods == null || params == null || headers == null || consumes == null || produces == null) {
return null;
}
PatternsRequestCondition patterns = this.patternsCondition.getMatchingCondition(request);
if (patterns == null) {
return null;
}
RequestConditionHolder custom = this.customConditionHolder.getMatchingCondition(request);
if (custom == null) {
return null;
}
return new RequestMappingInfo(this.name, patterns,
methods, params, headers, consumes, produces, custom.getCondition());
}
// HeadersRequestCondition
public HeadersRequestCondition getMatchingCondition(HttpServletRequest request) {
// 最終調用的是CorsUtils#isCorsRequest
if (CorsUtils.isPreFlightRequest(request)) {
return PRE_FLIGHT_MATCH;
}
for (HeaderExpression expression : expressions) {
if (!expression.match(request)) {
return null;
}
}
return this;
}
// CorsUtils
/**
* Returns {@code true} if the request is a valid CORS one.
*/
public static boolean isCorsRequest(HttpServletRequest request) {
// XSS的編碼是通過Filter對請求中的Header進行編碼的，所以每次遍歷URI都會調用一次請求頭編碼
return (request.getHeader(HttpHeaders.ORIGIN) != null);
}
/**
* Returns {@code true} if the request is a valid CORS pre-flight one.
*/
public static boolean isPreFlightRequest(HttpServletRequest request) {
return (isCorsRequest(request) && HttpMethod.OPTIONS.matches(request.getMethod()) &&
request.getHeader(HttpHeaders.ACCESS_CONTROL_REQUEST_METHOD) != null);
}
好了，真相終於浮出水面了。就是因為如果前台請求URI沒有完全匹配后台配置的話會導致每次請求校驗都會對請求頭中參數進行編碼。而項目中請求的URI有幾千個，假設每個請求頭平均有3個參數。那么，一次請求編碼可能上萬次...... 你說超時不超時？
解決方案
最終和領導討論確認去掉對header的XSS編碼處理。我們業務上不存在將Header的參數入庫的情況。
深思
善於思考的小伙伴一定會問了： 為什么會有請求URI不匹配呢？如果不匹配以前為什么能正常請求到呢？
哈哈，我們繼續看下文：
通過最后的getMatchingCondition方法，我們可以看到要想最終能找到一個匹配的請求的URI。上面幾個condition必須至少要滿足一個。通過debug我發現，最終在我們項目中匹配的是patternsCondition。那么這condition的具體實現是咋樣的呢？直接見源碼：
/**
* Checks if any of the patterns match the given request and returns an instance
* that is guaranteed to contain matching patterns, sorted via
* {@link PathMatcher#getPatternComparator(String)}.
* <p>A matching pattern is obtained by making checks in the following order:
* <ul>
* <li>Direct match
* <li>Pattern match with ".*" appended if the pattern doesn't already contain a "."
* <li>Pattern match
* <li>Pattern match with "/" appended if the pattern doesn't already end in "/"
* </ul>
* @param request the current request
* @return the same instance if the condition contains no patterns;
* or a new condition with sorted matching patterns;
* or {@code null} if no patterns match.
*/
public PatternsRequestCondition getMatchingCondition(HttpServletRequest request) {
if (this.patterns.isEmpty()) {
return this;
}
String lookupPath = this.pathHelper.getLookupPathForRequest(request);
List<String> matches = getMatchingPatterns(lookupPath);
return matches.isEmpty() ? null :
new PatternsRequestCondition(matches, this.pathHelper, this.pathMatcher, this.useSuffixPatternMatch,
this.useTrailingSlashMatch, this.fileExtensions);
}
/**
* Find the patterns matching the given lookup path. Invoking this method should
* yield results equivalent to those of calling
* {@link #getMatchingCondition(javax.servlet.http.HttpServletRequest)}.
* This method is provided as an alternative to be used if no request is available
* (e.g. introspection, tooling, etc).
* @param lookupPath the lookup path to match to existing patterns
* @return a collection of matching patterns sorted with the closest match at the top
*/
public List<String> getMatchingPatterns(String lookupPath) {
List<String> matches = new ArrayList<String>();
for (String pattern : this.patterns) {
String match = getMatchingPattern(pattern, lookupPath);
if (match != null) {
matches.add(match);
}
}
Collections.sort(matches, this.pathMatcher.getPatternComparator(lookupPath));
return matches;
}
private String getMatchingPattern(String pattern, String lookupPath) {
if (pattern.equals(lookupPath)) {
return pattern;
}
// 如果使用后綴匹配模式 且后台配置的URI沒有后綴(不包含.)，且前台請求的URI中包含. 則在后台配置的URI原來的匹配模式上加上 .* 再與前台請求URI進行匹配進行匹配
if (this.useSuffixPatternMatch) {
if (!this.fileExtensions.isEmpty() && lookupPath.indexOf('.') != -1) {
for (String extension : this.fileExtensions) {
if (this.pathMatcher.match(pattern + extension, lookupPath)) {
return pattern + extension;
}
}
}
else {
boolean hasSuffix = pattern.indexOf('.') != -1;
if (!hasSuffix && this.pathMatcher.match(pattern + ".*", lookupPath)) {
return pattern + ".*";
}
}
}
if (this.pathMatcher.match(pattern, lookupPath)) {
return pattern;
}
if (this.useTrailingSlashMatch) {
if (!pattern.endsWith("/") && this.pathMatcher.match(pattern + "/", lookupPath)) {
return pattern +"/";
}
}
return null;
}
注意中文注釋的部分。我們項目中這種不匹配的情況是后台只配置了/aaa/aaaa 而前台配置的是/aaa/aaaa.json，自然符合前面的模式匹配。自然，也就不會匹配不到后台請求。
總結
每一次采坑，都能夠讓人前進。通過這次XSS事件，我這邊還是有不少收獲的:
- 多想想后門： 實現功能的時候要多考慮可能出現的意外情況，實現功能很簡單，解決功能可能導致的問題卻是一個值的深入思考的方向
- 源碼debug的重要性： 當有一定的編程經驗之后，很多問題需要耐心debug才能解決。通過這個debug的過程，不僅能夠了解底層的實現流程，也熟悉了大神寫的代碼。 何樂而不為呢？
- SpringMVC前台請求最好與后台配置完全匹配: 通過這篇文章相信大家也看到了，不匹配的后果。SpringMVC源碼中的省略號已經充分展示了他對你這種行為的無語 😢
這次的博客就到這里，一是篇幅太長了。怕吃太多消化不良，二來留點懸念，我們下回分解（雖然不知道下回是啥時候了，哈哈哈 我盡快.....）
參考鏈接
AntiSamy: https://blog.csdn.net/qq_35946990/article/details/74982760
XSS與CSRF: https://www.jianshu.com/p/64a413ada155