背景：此前以javaagent的方式集成了prometheus jvm客戶端，jmx-exporter，現需要在此基礎上加幾個自定義參數：
Counter newcounter = Counter.build()
.name(key)
.help(key + "-help")
.register();
運行后，prometheus取不到這個key的數據，深入調查。
jmx-exporter這個工程寫了premain方法：
進行了一些Exporter的注冊，特別是jvm的一些指標，接下去看下注冊的代碼：
最終由CollectorRegistery 靜態成員變量 defaultRegistry來注冊，也就是說一個類加載器所有exporter的注冊都統一進行
javaagent由appclassloader加載，其它所有jar包由Saturn Job classloader加載，這個類加載器父加載器為null，則與appclassloader相互之間類是不可見的。
故javaagent和主業務指標不能共用一個httpserver對prometheus提供metric
此前的register代碼雖然也注冊了，但與jvm參數等所在不同的類加載器，兩者注冊到了兩個注冊中心，且這個新的未以http的形式發布給prometheus，用jvm的那個exporter由於兩者類加載器隔離自然是跟蹤不到這些自定義key的。
解決：
1 模仿jmx-exporter，新開一個httpserver：
new HTTPServer(socket, CollectorRegistry.defaultRegistry, true);
public class PromController {
private static final Logger LOGGER = LoggerFactory.getLogger(PromController.class);
private static final ConcurrentHashMap<String, Counter> mapCounter = new ConcurrentHashMap<>();
//線程池
private static final ExecutorService POOL = new ThreadPoolExecutor(
2, 4, 0L, TimeUnit.MILLISECONDS, new ArrayBlockingQueue<Runnable>(20),
new RejectedExecutionHandler() {
@Override
public void rejectedExecution(Runnable r, ThreadPoolExecutor executor) {
LOGGER.warn("放棄線程");
}
});
private static HTTPServer server = null;
static {
int cc = 0;
int port = 9892;
while (true) {
if(cc > 10) {
LOGGER.error("放棄監聽prom：{}", port);
server = null;
break;
}
InetSocketAddress socket = new InetSocketAddress("0.0.0.0", port);
try {
server = new HTTPServer(socket, CollectorRegistry.defaultRegistry, true);
LOGGER.info("成功監聽prom：{}", port);
break;
} catch (IOException e) {
LOGGER.error("{} {}", e.getMessage(), ++cc);
port += 10;
}
}
}
public static void main(String []args) throws InterruptedException {
while (true) {
putPromCounter("xxx");
Thread.sleep(1000);
}
}
public static void putPromCounter(final String key) {
if(server == null)
return;
POOL.submit(new Runnable() {
@Override
public void run() {
// try {
// Class<?> cCollectorRegistry = Thread.currentThread().getContextClassLoader().loadClass("io.prometheus.jmx.shaded.io.prometheus.client.CollectorRegistry");
// Class<?> cCounter = Thread.currentThread().getContextClassLoader().loadClass("io.prometheus.jmx.shaded.io.prometheus.client.Counter");
// Class<?> cCounterBuilder = Thread.currentThread().getContextClassLoader().loadClass("io.prometheus.jmx.shaded.io.prometheus.client.Counter$Builder");
// LOGGER.info("CollectorRegistry: {}", cCollectorRegistry.getProtectionDomain().getCodeSource().getLocation());
// LOGGER.info("Counter: {}", cCounter.getProtectionDomain().getCodeSource().getLocation());
// LOGGER.info("CounterBuilder: {}", cCounterBuilder.getProtectionDomain().getCodeSource().getLocation());
// } catch (ClassNotFoundException e) {
// LOGGER.info("no class ");
// }
Counter counter = mapCounter.get(key);
if(counter == null) {
LOGGER.info("new prom counter {}", key);
Counter newcounter = Counter.build()
.name(key)
.help(key + "-help")
.register();
mapCounter.putIfAbsent(key, newcounter);
counter = mapCounter.get(key);
}
counter.inc();
}
});
}
}
2 prometheus服務端加入對此服務器的拉取
84 - job_name: 'jmx-executor-sym' 85 scrape_interval: 10s 86 static_configs: 87 - targets: ['192.168.57.234:8892'] 88 labels: 89 instance: executor-sym
當然，也可以直接使用prom的client maven：
基於Grafana和Prometheus的監視系統(3):java客戶端使用
https://www.jianshu.com/p/60c6d6cb4c49
附屬一個jmx-exporter源碼分析：https://blog.csdn.net/qqqq0199181/article/details/83792364