本文基於 Pinpoint 2.1.0 版本 本文大部分內容來自:俠夢的開發筆記 ,但是原文的版本和我的不一致,放在2.1.0是跑不起來的,但是大概邏輯和思路基本一致。
一、接入預警大概思路
官方預警相關文檔: https://pinpoint-apm.github.io/pinpoint/2.1.0/alarm.html
在pinpoint 2.X中引入了默認的告警實現類DefaultAlarmMessageSender
。所以我們只需要實現短信發送的接口即可。
上述類封裝了發送郵件和短信的方法,目前短信方法的實現仍是空,不過打印了一句話。
logger.info("can not send sms message.");
然而,郵件發送是有一個實現類來幫我們做告警郵件發送的。他就是:SpringSmtpMailSender。
我們接入預警的大概思路就是實現短信發送,然后當短信發送的時候,調用我們寫的模塊,然后實現微信/釘釘預警。 微信釘釘預警都是通過一個接口給到我們進行調用即可。下面我就說說微信預警(釘釘預警一致)。
我們接入微信預警分為三大塊:
- 加入 微信預警模塊
- 將調用 短信發送改成調用 微信預警
- 增加一個 bean 引入配置
- 配置文件增加微信預警URL 配置
二、具體實現
2.1、加入預警模塊
將這個Pinpoint2DingTalkSmsSender.java
文件加入到/web/src/main/java/com/navercorp/pinpoint/web/alarm
文件內容如下所示。
package com.navercorp.pinpoint.web.alarm;
import com.navercorp.pinpoint.web.alarm.checker.AlarmChecker;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.batch.core.StepExecution;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.http.HttpEntity;
import org.springframework.http.HttpHeaders;
import org.springframework.http.MediaType;
import org.springframework.http.ResponseEntity;
import org.springframework.util.StringUtils;
import org.springframework.web.client.RestTemplate;
import java.util.HashMap;
import java.util.List;
import java.util.Map;
import java.util.Optional;
public class Pinpoint2DingTalkSmsSender implements SmsSender {
private final Logger logger = LoggerFactory.getLogger(this.getClass());
@Override
public void sendSms(AlarmChecker checker, int sequenceCount, StepExecution stepExecution) {
if(StringUtils.isEmpty(dingTalkUrl)){
logger.warn("web.ding.talk.url is not set!");
return;
}
List smsMessage = checker.getSmsMessage();
String textMsg = String.join("\r\n", smsMessage);
send(textMsg);
}
@Autowired
private RestTemplate restTemplate;
private String dingTalkUrl;
private static String DING_ALARM_PREFIX= "【告警】";
public Pinpoint2DingTalkSmsSender(){
logger.info("init DingTalkSmsSender,{}",dingTalkUrl);
}
public void send(String textMsg){
HttpHeaders headers = new HttpHeaders();
headers.setContentType(MediaType.APPLICATION_JSON);
Map map = dingMap(textMsg);
HttpEntity<Map<String, Object>> request = new HttpEntity<Map<String, Object>>(map, headers);
ResponseEntity<DingResponse> responseEntity = restTemplate.postForEntity(dingTalkUrl, request, DingResponse.class);
DingResponse dingResponse = Optional.ofNullable(responseEntity).map(ResponseEntity::getBody).orElse(null);
logger.info("send alarm msg: {},dingtalk result: {} ",map,dingResponse);
}
public static Map dingMap(String message){
Map textMap = new HashMap();
textMap.put("content",DING_ALARM_PREFIX + message);
Map result = new HashMap();
result.put("msgtype","text");
result.put("text",textMap);
return result;
}
public RestTemplate getRestTemplate() {
return restTemplate;
}
public void setRestTemplate(RestTemplate restTemplate) {
this.restTemplate = restTemplate;
}
public String getDingTalkUrl() {
return dingTalkUrl;
}
public void setDingTalkUrl(String dingTalkUrl) {
this.dingTalkUrl = dingTalkUrl;
}
static class DingResponse{
private int errcode;
private String errmsg;
public int getErrcode() {
return errcode;
}
public void setErrcode(int errcode) {
this.errcode = errcode;
}
public String getErrmsg() {
return errmsg;
}
public void setErrmsg(String errmsg) {
this.errmsg = errmsg;
}
@Override
public String toString() {
return "DingResponse{" +
"errcode=" + errcode +
", errmsg='" + errmsg + '\'' +
'}';
}
}
}
2.2、開啟微信預警調用
web/src/main/java/com/navercorp/pinpoint/web/alarm/DefaultAlarmMessageSender.java
將調用 EmptySmsSender
注釋。 增加調用 Pinpoint2DingTalkSmsSender
//this.smsSender = smsSender.orElseGet(EmptySmsSender::new);
this.smsSender = smsSender.orElseGet(Pinpoint2DingTalkSmsSender::new);
2.3、增加一個 bean 引入配置
在 applicationContext-web.xml
中增加一個bean
<bean id="dingTalkAndWechatMessageSender" class="com.navercorp.pinpoint.web.alarm.Pinpoint2DingTalkSmsSender" >
<property name="dingTalkUrl" value="${web.wechat.alarm.url:}"/>
</bean>
2.4、配置文件增加微信預警URL 配置
該步驟可以在web容器啟動后,然后進入到容器中編輯配置文件,然后重啟 web 容器即可。
在pinpoint-web.properties
增加配置
web.wechat.alarm.url=https://oapi.dingtalk.com/robot/send?
三、實現預警
3.1、創建用戶和創建用戶組
3.2、創建預警規則
預警的是針對不同的應用(服務)來配置,可以設置對應的統計規則,和閾值,接收預警的只能是用戶組,不能是用戶, 預警的方式可以選擇 sms 和 email.
補充下預警規則, 默認策略,是每3分鍾統計一次,統計最近5分鍾的數據,這個可以更改,具體更改方式見 :
四、郵件預警
見官網 https://pinpoint-apm.github.io/pinpoint/2.1.0/alarm.html
五、預警相關問題和注意事項
5.1、告警發送異常,缺少參數
原因是:我們在配置預警的用戶信息的時候,並沒有全都配置,雖然有些配置不是必填的,但是我們還是需要都填入則預警信息發送不出來,並且出現下面的錯誤。
11-18 10:24:00.000 [ scheduler-3] ERROR o.s.s.s.TaskUtils$LoggingErrorHandler -- Unexpected error occurred in scheduled task
java.lang.IllegalStateException: org.springframework.batch.core.repository.JobInstanceAlreadyCompleteException: A job instance already exists and is complete for parameters={schedule.date=1605666240000}. If you want to run this job again, change the parameters.
at com.navercorp.pinpoint.web.batch.JobLaunchSupport.run(JobLaunchSupport.java:52) ~[classes!/:2.1.0]
at com.navercorp.pinpoint.web.batch.BatchJobLauncher.alarmJob(BatchJobLauncher.java:50) ~[classes!/:2.1.0]
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:1.8.0_212]
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[?:1.8.0_212]
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:1.8.0_212]
at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_212]
at org.springframework.scheduling.support.ScheduledMethodRunnable.run(ScheduledMethodRunnable.java:84) ~[spring-context-5.2.7.RELEASE.jar!/:5.2.7.RELEASE]
at org.springframework.scheduling.support.DelegatingErrorHandlingRunnable.run(DelegatingErrorHandlingRunnable.java:54) [spring-context-5.2.7.RELEASE.jar!/:5.2.7.RELEASE]
at org.springframework.scheduling.concurrent.ReschedulingRunnable.run(ReschedulingRunnable.java:93) [spring-context-5.2.7.RELEASE.jar!/:5.2.7.RELEASE]
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) [?:1.8.0_212]
at java.util.concurrent.FutureTask.run(FutureTask.java:266) [?:1.8.0_212]
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180) [?:1.8.0_212]
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293) [?:1.8.0_212]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [?:1.8.0_212]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [?:1.8.0_212]
at java.lang.Thread.run(Thread.java:748) [?:1.8.0_212]
Caused by: org.springframework.batch.core.repository.JobInstanceAlreadyCompleteException: A job instance already exists and is complete for parameters={schedule.date=1605666240000}. If you want to run this job again, change the parameters.
at org.springframework.batch.core.repository.support.SimpleJobRepository.createJobExecution(SimpleJobRepository.java:131) ~[spring-batch-core-4.2.4.RELEASE.jar!/:4.2.4.RELEASE]
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:1.8.0_212]
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[?:1.8.0_212]
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:1.8.0_212]
at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_212]
at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:344) ~[spring-aop-5.2.7.RELEASE.jar!/:5.2.7.RELEASE]
at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:198) ~[spring-aop-5.2.7.RELEASE.jar!/:5.2.7.RELEASE]
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163) ~[spring-aop-5.2.7.RELEASE.jar!/:5.2.7.RELEASE]
at org.springframework.transaction.interceptor.TransactionAspectSupport.invokeWithinTransaction(TransactionAspectSupport.java:367) ~[spring-tx-5.2.7.RELEASE.jar!/:5.2.7.RELEASE]
at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:118) ~[spring-tx-5.2.7.RELEASE.jar!/:5.2.7.RELEASE]
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186) ~[spring-aop-5.2.7.RELEASE.jar!/:5.2.7.RELEASE]
at org.springframework.batch.core.repository.support.AbstractJobRepositoryFactoryBean$1.invoke(AbstractJobRepositoryFactoryBean.java:181) ~[spring-batch-core-4.2.4.RELEASE.jar!/:4.2.4.RELEASE]
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186) ~[spring-aop-5.2.7.RELEASE.jar!/:5.2.7.RELEASE]
at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:212) ~[spring-aop-5.2.7.RELEASE.jar!/:5.2.7.RELEASE]
at com.sun.proxy.$Proxy96.createJobExecution(Unknown Source) ~[?:?]
at org.springframework.batch.core.launch.support.SimpleJobLauncher.run(SimpleJobLauncher.java:137) ~[spring-batch-core-4.2.4.RELEASE.jar!/:4.2.4.RELEASE]
at com.navercorp.pinpoint.web.batch.JobLaunchSupport.run(JobLaunchSupport.java:50) ~[classes!/:2.1.0]
... 15 more
如果出現上面的報錯會一致出現,清除的方式,我是進行清除數據庫(慎重)。