背景：大家都在說Skywalking Agent實現的核心原理就是
Java Agent+字節碼操作。但是這兩者是如何起作用的，一直比較模糊，所以通過這篇博客來記錄一下解疑答惑的過程。
通過這篇博客應該可以解答如下幾個問題：
- Skywalking具體使用什么來操作字節碼；
- Skywalking的插件是怎么開發的；
- Skywalking是怎么讓插件生效的；
ByteBuddy入門
ByteBuddy封裝了一系列API來輕松創建一個agent，下面通過AgentBuilder來創建一個簡單的agent：假設我們定義了一個注解ToString，我們匹配所有標注@ToString的類，修改toString方法，讓其返回"transformed"。
public class ToStringAgent {
public static void premain(String arguments, Instrumentation instrumentation) {
new AgentBuilder.Default()
.type(isAnnotatedWith(ToString.class))
.transform(new AgentBuilder.Transformer() {
@Override
public DynamicType.Builder transform(DynamicType.Builder builder,
TypeDescription typeDescription,
ClassLoader classloader) {
return builder.method(named("toString"))
.intercept(FixedValue.value("transformed"));
}
}).installOn(instrumentation);
}
}
- type 接受 一個ElementMatcher 匹配 ToString注解
- transform 接受AgentBuilder.Transformer 來描述修改邏輯，每有一個類加載時，觸發transformer邏輯，對類進行匹配和修改。
- installOn ,將修改，應用到instrumentation中。
自定義Skywalking插件
1、引入依賴
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>com.itmuch.skywalking</groupId>
<artifactId>apm-string-replace-plugin</artifactId>
<packaging>jar</packaging>
<version>1.0.0-SNAPSHOT</version>
<properties>
<skywalking.version>6.6.0</skywalking.version>
<shade.package>org.apache.skywalking.apm.dependencies</shade.package>
<shade.net.bytebuddy.source>net.bytebuddy</shade.net.bytebuddy.source>
<shade.net.bytebuddy.target>${shade.package}.${shade.net.bytebuddy.source}</shade.net.bytebuddy.target>
</properties>
<dependencies>
<dependency>
<groupId>org.apache.skywalking</groupId>
<artifactId>apm-agent-core</artifactId>
<version>${skywalking.version}</version>
<scope>provided</scope>
</dependency>
<dependency>
<groupId>org.apache.skywalking</groupId>
<artifactId>apm-util</artifactId>
<version>${skywalking.version}</version>
<scope>provided</scope>
</dependency>
<dependency>
<groupId>org.apache.commons</groupId>
<artifactId>commons-lang3</artifactId>
<version>3.4</version>
<scope>provided</scope>
</dependency>
</dependencies>
<build>
<plugins>
<plugin>
<artifactId>maven-shade-plugin</artifactId>
<executions>
<execution>
<phase>package</phase>
<goals>
<goal>shade</goal>
</goals>
<configuration>
<shadedArtifactAttached>false</shadedArtifactAttached>
<createDependencyReducedPom>true</createDependencyReducedPom>
<createSourcesJar>true</createSourcesJar>
<shadeSourcesContent>true</shadeSourcesContent>
<relocations>
<relocation>
<pattern>${shade.net.bytebuddy.source}</pattern>
<shadedPattern>${shade.net.bytebuddy.target}</shadedPattern>
</relocation>
</relocations>
</configuration>
</execution>
</executions>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-compiler-plugin</artifactId>
<configuration>
<source>6</source>
<target>6</target>
</configuration>
</plugin>
</plugins>
</build>
</project>
編寫插件定義
public class ToStringInstrumentation extends ClassInstanceMethodsEnhancePluginDefine {
@Override
protected ClassMatch enhanceClass() {
// 指定想要監控的類
return NameMatch.byName("java.lang.String");
}
@Override
public ConstructorInterceptPoint[] getConstructorsInterceptPoints() {
return new ConstructorInterceptPoint[0];
}
@Override
public InstanceMethodsInterceptPoint[] getInstanceMethodsInterceptPoints() {
// 指定想要監控的實例方法，每個實例方法對應一個InstanceMethodsInterceptPoint
return new InstanceMethodsInterceptPoint[]{
@Override
public ElementMatcher<MethodDescription> getMethodsMatcher() {
// 實例方法名稱
return ElementMatchers.named("toString");
}
@Override
public String getMethodsInterceptor() {
// 該實例方法的監控攔截器類名全路徑
return "com.itmuch.skywalking.plugin.tostring.ToStringInterceptor";
}
@Override
public boolean isOverrideArgs() {
return false;
}
};
}
@Override
public StaticMethodsInterceptPoint[] getStaticMethodsInterceptPoints() {
// 指定想要監控的靜態方法，每一個方法對應一個StaticMethodsInterceptPoint
return new StaticMethodsInterceptPoint[0];
}
}
編寫攔截器
public class ToStringInterceptor implements InstanceMethodsAroundInterceptor {
@Override
public void beforeMethod(Class aClass, Method method, Object[] argumentsTypes, Class<?>[] classes, MethodInterceptResult methodInterceptResult) {
System.out.print("在toString之前執行beforeMethod方法");
}
@Override
public Object afterMethod(Class aClass, Method method, Object[] objects, Class<?>[] classes, Object o) {
return o + "在toString之后執行afterMethod";
}
@Override
public void handleMethodException(Class aClass, Method method, Object[] objects, Class<?>[] classes, Throwable throwable) {
System.out.print(throwable.getMessage());
}
}
上面的操作步驟完全仿照Skywalking源碼中的插件的定義。
使用的哪種字節碼操作框架
首先找到入口類premain-class。
apm-sniffer/apm-agent/pom.xml：
<properties>
<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
<!-- javaagent對應的premain方法 -->
<premain.class>org.apache.skywalking.apm.agent.SkyWalkingAgent</premain.class>
<can.redefine.classes>true</can.redefine.classes>
<can.retransform.classes>true</can.retransform.classes>
<shade.net.bytebuddy.source>net.bytebuddy</shade.net.bytebuddy.source>
<shade.net.bytebuddy.target>${shade.package}.${shade.net.bytebuddy.source}</shade.net.bytebuddy.target>
</properties>
找到了入口類SkyWalkingAgent，那來看看他的源碼：
public static void premain(String agentArgs, Instrumentation instrumentation) throws PluginException {
final PluginFinder pluginFinder;
try {
//1、初始化一些核心配置
SnifferConfigInitializer.initializeCoreConfig(agentArgs);
} catch (Exception e) {
// try to resolve a new logger, and use the new logger to write the error log here
LogManager.getLogger(SkyWalkingAgent.class)
.error(e, "SkyWalking agent initialized failure. Shutting down.");
return;
} finally {
// refresh logger again after initialization finishes
LOGGER = LogManager.getLogger(SkyWalkingAgent.class);
}
try {
//2、加載插件
pluginFinder = new PluginFinder(new PluginBootstrap().loadPlugins());
} catch (AgentPackageNotFoundException ape) {
LOGGER.error(ape, "Locate agent.jar failure. Shutting down.");
return;
} catch (Exception e) {
LOGGER.error(e, "SkyWalking agent initialized failure. Shutting down.");
return;
}
//3、初始化字節碼操作框架ByteBuddy
final ByteBuddy byteBuddy = new ByteBuddy().with(TypeValidation.of(Config.Agent.IS_OPEN_DEBUGGING_CLASS));
AgentBuilder agentBuilder = new AgentBuilder.Default(byteBuddy).ignore(
nameStartsWith("net.bytebuddy.")
.or(nameStartsWith("org.slf4j."))
.or(nameStartsWith("org.groovy."))
.or(nameContains("javassist"))
.or(nameContains(".asm."))
.or(nameContains(".reflectasm."))
.or(nameStartsWith("sun.reflect"))
.or(allSkyWalkingAgentExcludeToolkit())
.or(ElementMatchers.isSynthetic()));
JDK9ModuleExporter.EdgeClasses edgeClasses = new JDK9ModuleExporter.EdgeClasses();
try {
agentBuilder = BootstrapInstrumentBoost.inject(pluginFinder, instrumentation, agentBuilder, edgeClasses);
} catch (Exception e) {
LOGGER.error(e, "SkyWalking agent inject bootstrap instrumentation failure. Shutting down.");
return;
}
try {
agentBuilder = JDK9ModuleExporter.openReadEdge(instrumentation, agentBuilder, edgeClasses);
} catch (Exception e) {
LOGGER.error(e, "SkyWalking agent open read edge in JDK 9+ failure. Shutting down.");
return;
}
if (Config.Agent.IS_CACHE_ENHANCED_CLASS) {
try {
agentBuilder = agentBuilder.with(new CacheableTransformerDecorator(Config.Agent.CLASS_CACHE_MODE));
LOGGER.info("SkyWalking agent class cache [{}] activated.", Config.Agent.CLASS_CACHE_MODE);
} catch (Exception e) {
LOGGER.error(e, "SkyWalking agent can't active class cache.");
}
}
agentBuilder.type(pluginFinder.buildMatch())
//4、重點：描述如何修改字節碼
.transform(new Transformer(pluginFinder))
.with(AgentBuilder.RedefinitionStrategy.RETRANSFORMATION)
.with(new RedefinitionListener())
.with(new Listener())
.installOn(instrumentation);
try {
ServiceManager.INSTANCE.boot();
} catch (Exception e) {
LOGGER.error(e, "Skywalking agent boot failure.");
}
Runtime.getRuntime()
.addShutdownHook(new Thread(ServiceManager.INSTANCE::shutdown, "skywalking service shutdown thread"));
}
通過上面編號3處的代碼可以看到使用的是ByteBuddy字節碼操作框架。
如何修改字節碼
我們都知道通過Javaagent來修改字節碼離不開ClassFileTransformer，同樣的ByteBuddy也根據transform來描述如何修改字節碼，如上面的編號4處的代碼。
private static class Transformer implements AgentBuilder.Transformer {
private PluginFinder pluginFinder;
Transformer(PluginFinder pluginFinder) {
this.pluginFinder = pluginFinder;
}
/**
* 轉換方法
* @param builder
* @param typeDescription 要被修改的Class類型
* @param classLoader
* @param module
* @return
*/
@Override
public DynamicType.Builder<?> transform(final DynamicType.Builder<?> builder,
final TypeDescription typeDescription,
final ClassLoader classLoader,
final JavaModule module) {
//1、找到typeDescription要被哪些插件增強
List<AbstractClassEnhancePluginDefine> pluginDefines = pluginFinder.find(typeDescription);
if (pluginDefines.size() > 0) {
DynamicType.Builder<?> newBuilder = builder;
EnhanceContext context = new EnhanceContext();
for (AbstractClassEnhancePluginDefine define : pluginDefines) {
//2、使用插件增強
DynamicType.Builder<?> possibleNewBuilder = define.define(
typeDescription, newBuilder, classLoader, context);
if (possibleNewBuilder != null) {
newBuilder = possibleNewBuilder;
}
}
if (context.isEnhanced()) {
LOGGER.debug("Finish the prepare stage for {}.", typeDescription.getName());
}
return newBuilder;
}
LOGGER.debug("Matched class {}, but ignore by finding mechanism.", typeDescription.getTypeName());
return builder;
}
}
重點看看編號2處的代碼如何用插件增強目標類。
public DynamicType.Builder<?> define(TypeDescription typeDescription, DynamicType.Builder<?> builder,
ClassLoader classLoader, EnhanceContext context) throws PluginException {
//1、插件的全路徑名
String interceptorDefineClassName = this.getClass().getName();
//2、要被增強類的全路徑名
String transformClassName = typeDescription.getTypeName();
if (StringUtil.isEmpty(transformClassName)) {
LOGGER.warn("classname of being intercepted is not defined by {}.", interceptorDefineClassName);
return null;
}
LOGGER.debug("prepare to enhance class {} by {}.", transformClassName, interceptorDefineClassName);
WitnessFinder finder = WitnessFinder.INSTANCE;
/**
* find witness classes for enhance class
*/
//3、一些插件需要依賴一些外部的類,他會找是否有對應依賴的類,如果沒有就直接 return null,不再去修改字節碼了
String[] witnessClasses = witnessClasses();
if (witnessClasses != null) {
for (String witnessClass : witnessClasses) {
if (!finder.exist(witnessClass, classLoader)) {
LOGGER.warn("enhance class {} by plugin {} is not working. Because witness class {} is not existed.", transformClassName, interceptorDefineClassName, witnessClass);
return null;
}
}
}
//4、一些插件需要依賴一些外部的方法,他會找是否有對應依賴的方法,如果沒有就直接 return null,不再去修改字節碼了
List<WitnessMethod> witnessMethods = witnessMethods();
if (!CollectionUtil.isEmpty(witnessMethods)) {
for (WitnessMethod witnessMethod : witnessMethods) {
if (!finder.exist(witnessMethod, classLoader)) {
LOGGER.warn("enhance class {} by plugin {} is not working. Because witness method {} is not existed.", transformClassName, interceptorDefineClassName, witnessMethod);
return null;
}
}
}
/**
* find origin class source code for interceptor
*/
//5、真正的增強
DynamicType.Builder<?> newClassBuilder = this.enhance(typeDescription, builder, classLoader, context);
context.initializationStageCompleted();
LOGGER.debug("enhance class {} by {} completely.", transformClassName, interceptorDefineClassName);
return newClassBuilder;
}
上面編號5處的方法才是真正的增強邏輯，繼續跟進去看看。
protected DynamicType.Builder<?> enhance(TypeDescription typeDescription, DynamicType.Builder<?> newClassBuilder,
ClassLoader classLoader, EnhanceContext context) throws PluginException {
//1、增強靜態方法
newClassBuilder = this.enhanceClass(typeDescription, newClassBuilder, classLoader);
//2、增強實例方法
newClassBuilder = this.enhanceInstance(typeDescription, newClassBuilder, classLoader, context);
return newClassBuilder;
}
這里分兩個步驟，增強靜態方法和增強實例方法，這里以編號2處的舉例，繼續跟進去。
private DynamicType.Builder<?> enhanceInstance(TypeDescription typeDescription,
DynamicType.Builder<?> newClassBuilder, ClassLoader classLoader,
EnhanceContext context) throws PluginException {
//1、構造器攔截點，在執行構造器之前也有可能會執行其他邏輯來增強構造方法
ConstructorInterceptPoint[] constructorInterceptPoints = getConstructorsInterceptPoints();
//2、實例方法攔截點，對實例方法的增強
InstanceMethodsInterceptPoint[] instanceMethodsInterceptPoints = getInstanceMethodsInterceptPoints();
String enhanceOriginClassName = typeDescription.getTypeName();
boolean existedConstructorInterceptPoint = false;
if (constructorInterceptPoints != null && constructorInterceptPoints.length > 0) {
existedConstructorInterceptPoint = true;
}
boolean existedMethodsInterceptPoints = false;
if (instanceMethodsInterceptPoints != null && instanceMethodsInterceptPoints.length > 0) {
existedMethodsInterceptPoints = true;
}
/**
* nothing need to be enhanced in class instance, maybe need enhance static methods.
*/
if (!existedConstructorInterceptPoint && !existedMethodsInterceptPoints) {
return newClassBuilder;
}
/**
* Manipulate class source code.<br/>
*
* new class need:<br/>
* 1.Add field, name {@link #CONTEXT_ATTR_NAME}.
* 2.Add a field accessor for this field.
*
* And make sure the source codes manipulation only occurs once.
*
*/
if (!typeDescription.isAssignableTo(EnhancedInstance.class)) {
if (!context.isObjectExtended()) {
newClassBuilder = newClassBuilder.defineField(
CONTEXT_ATTR_NAME, Object.class, ACC_PRIVATE | ACC_VOLATILE)
.implement(EnhancedInstance.class)
.intercept(FieldAccessor.ofField(CONTEXT_ATTR_NAME));
context.extendObjectCompleted();
}
}
/**
* 3、enhance constructors 增強構造方法
*/
if (existedConstructorInterceptPoint) {
for (ConstructorInterceptPoint constructorInterceptPoint : constructorInterceptPoints) {
if (isBootstrapInstrumentation()) {
newClassBuilder = newClassBuilder.constructor(constructorInterceptPoint.getConstructorMatcher())
.intercept(SuperMethodCall.INSTANCE.andThen(MethodDelegation.withDefaultConfiguration()
.to(BootstrapInstrumentBoost
.forInternalDelegateClass(constructorInterceptPoint
.getConstructorInterceptor()))));
} else {
newClassBuilder = newClassBuilder.constructor(constructorInterceptPoint.getConstructorMatcher())
.intercept(SuperMethodCall.INSTANCE.andThen(MethodDelegation.withDefaultConfiguration()
.to(new ConstructorInter(constructorInterceptPoint
.getConstructorInterceptor(), classLoader))));
}
}
}
/**
* 4、enhance instance methods 增強實例方法
*/
if (existedMethodsInterceptPoints) {
for (InstanceMethodsInterceptPoint instanceMethodsInterceptPoint : instanceMethodsInterceptPoints) {
String interceptor = instanceMethodsInterceptPoint.getMethodsInterceptor();
if (StringUtil.isEmpty(interceptor)) {
throw new EnhanceException("no InstanceMethodsAroundInterceptor define to enhance class " + enhanceOriginClassName);
}
ElementMatcher.Junction<MethodDescription> junction = not(isStatic()).and(instanceMethodsInterceptPoint.getMethodsMatcher());
if (instanceMethodsInterceptPoint instanceof DeclaredInstanceMethodsInterceptPoint) {
junction = junction.and(ElementMatchers.<MethodDescription>isDeclaredBy(typeDescription));
}
if (instanceMethodsInterceptPoint.isOverrideArgs()) {
if (isBootstrapInstrumentation()) {
newClassBuilder = newClassBuilder.method(junction)
.intercept(MethodDelegation.withDefaultConfiguration()
.withBinders(Morph.Binder.install(OverrideCallable.class))
.to(BootstrapInstrumentBoost.forInternalDelegateClass(interceptor)));
} else {
newClassBuilder = newClassBuilder.method(junction)
.intercept(MethodDelegation.withDefaultConfiguration()
.withBinders(Morph.Binder.install(OverrideCallable.class))
.to(new InstMethodsInterWithOverrideArgs(interceptor, classLoader)));
}
} else {
if (isBootstrapInstrumentation()) {
newClassBuilder = newClassBuilder.method(junction)
.intercept(MethodDelegation.withDefaultConfiguration()
.to(BootstrapInstrumentBoost.forInternalDelegateClass(interceptor)));
} else {
//5、一般情況下會走到這里
newClassBuilder = newClassBuilder.method(junction)
.intercept(MethodDelegation.withDefaultConfiguration()
.to(new InstMethodsInter(interceptor, classLoader)));
}
}
}
}
return newClassBuilder;
}
看到上述編號5的時候是不是就覺得熟悉了，這不就是ByteBuddy創建agent的代碼嗎，按照這個邏輯InstMethodsInter這個類就是委托類，他里面的intercept方法才是真正增強/代理的核心。
@RuntimeType
public Object intercept(@This Object obj, @AllArguments Object[] allArguments, @SuperCall Callable<?> zuper,
@Origin Method method) throws Throwable {
//1、要增強的對象
EnhancedInstance targetObject = (EnhancedInstance) obj;
MethodInterceptResult result = new MethodInterceptResult();
try {
//2、在真正方法調用之前進行增強
interceptor.beforeMethod(targetObject, method, allArguments, method.getParameterTypes(), result);
} catch (Throwable t) {
LOGGER.error(t, "class[{}] before method[{}] intercept failure", obj.getClass(), method.getName());
}
Object ret = null;
try {
if (!result.isContinue()) {
ret = result._ret();
} else {
//3、真正的方法調用
ret = zuper.call();
}
} catch (Throwable t) {
try {
//4、真正的方法調用之后處理異常
interceptor.handleMethodException(targetObject, method, allArguments, method.getParameterTypes(), t);
} catch (Throwable t2) {
LOGGER.error(t2, "class[{}] handle method[{}] exception failure", obj.getClass(), method.getName());
}
throw t;
} finally {
try {
//5、真正的方法調用之后進行增強
ret = interceptor.afterMethod(targetObject, method, allArguments, method.getParameterTypes(), ret);
} catch (Throwable t) {
LOGGER.error(t, "class[{}] after method[{}] intercept failure", obj.getClass(), method.getName());
}
}
return ret;
}
那這里會問這個interceptor到底是什么呢，往回跟蹤代碼
InstanceMethodsInterceptPoint[] instanceMethodsInterceptPoints = getInstanceMethodsInterceptPoints();
for (InstanceMethodsInterceptPoint instanceMethodsInterceptPoint : instanceMethodsInterceptPoints) {
String interceptor = instanceMethodsInterceptPoint.getMethodsInterceptor();
看到這個getInstanceMethodsInterceptPoints是不是又很熟悉呢，這其實就是我們上面自定義插件的時候實現的ClassInstanceMethodsEnhancePluginDefine里面的方法。
那現在問題清楚了，總的來說可以用下面的圖描述：
總結
其實Skywalking只是通過ByteBuddy修改字節碼，在目標方法前后執行增強代碼，但是具體增強的邏輯還是在各自的插件里面定義。只能感嘆插件機制真強大。