调试遇到的问题
按照给定的源码，外部插件是没有加加载的。因为外部插件有两种加载方式。一个是以jar包方式，一个是plugin.xml方式加装。调试发现如果以jar形式加载，需要将jar包放到/ui/plugins目录下（而不是源代码目录/plugins目录）而这种方式导致有些插件pdi-xml-plugin插件加载会报错Unexpected error loading class: General - org/pentaho/metaverse/api/analyzer/kettle/step/IStepExternalResourceConsumer
， 分析发现pdi-xml-plugin的pom.xml文件发现依赖pentaho-metaverse-api包，但是看了下Eclipse 中Maven Dependencies有这个包。我猜可能这种加载jar包找不到这个包（我也验证过，将maven仓库中的这个包拷贝到这个插件libs目录下确实不报没有这个错，但是报其他包的错。感觉拷贝maven包的方式肯定不行，而且其他插件也需要这个包也要拷贝，造成包重复，也违背了maven仓库管理原则），如是要考虑其他方法。但是看了经过mvn命令行编译好的.源代码目录pentaho-kettle-9.2.0.3-R\assemblies\client\target下的pdi-ce-9.2.0.3-477.zip解压后，我们看到如下目录
lib:所有maven仓库的包
libswt:不同平台下的swt包(linux,osx64,win64)
plugins:所有外部插件
ui:都是些*.xul文件
launcher：里含有launcher.jar文件和launcher.properties文件而launcher.properties中指定了lib，libswt路径
main=org.pentaho.di.ui.spoon.Spoon
libraries=../test:../lib:../libswt
classpath=../classes:../:../ui:../ui/images:../lib
system-property.pentaho.installed.licenses.file=${PENTAHO_INSTALLED_LICENSE_PATH}
在main方法中 KettleEnvironment.init()之前添加了
System.setProperty("KETTLE_PLUGIN_BASE_FOLDERS","E:\\DevOps\\ETL\\pentaho-kettle-9.2.0.3-R\\plugins");
重新cd到源码目录，用如下命令编译项目
mvn clean package
运行调试都报错
Caused by: java.lang.NoClassDefFoundError: com/google/api/client/json/JsonFactory
at org.pentaho.googledrive.lifecycle.GoogleDrivePluginLifecycleListener.onEnvironmentInit(GoogleDrivePluginLifecycleListener.java:49)
at org.pentaho.di.core.lifecycle.KettleLifecycleSupport.onEnvironmentInit(KettleLifecycleSupport.java:116)
at org.pentaho.di.core.lifecycle.KettleLifecycleSupport.onEnvironmentInit(KettleLifecycleSupport.java:107)
at org.pentaho.di.core.KettleEnvironment.initLifecycleListeners(KettleEnvironment.java:184)
at org.pentaho.di.core.KettleEnvironment.init(KettleEnvironment.java:157)
调试分析异常
经验证去掉System.setProperty("KETTLE_PLUGIN_BASE_FOLDERS","E:\DevOps\ETL\pentaho-kettle-9.2.0.3-R\plugins");这句话打包程序运行就不会报错。
如果不加入这句话，需要将plugins拷贝到pentaho-kettle-9.2.0.3-R\ui下，否则jar包就不会加载，本来将编译好的plugins目录及文件拷贝到pentaho-kettle-9.2.0.3-R\ui下调试，发现kettile-json-plugin-core插件报错
General - ERROR (version Unknown) : Unexpected error loading class for plugin JsonOutputAnalyzerPlugin
General - ERROR (version Unknown) : org.pentaho.di.core.exception.KettlePluginException:
General - Unexpected error loading class:
General - org/pentaho/metaverse/api/analyzer/kettle/step/IStepExternalResourceConsumer
猜测缺少包，但maven仓库中有这个包，只是这个插件下lib下没有这个包（而且这个不像内部插件调试环境有Maven Dependciies库，外部插件虽然也有Maven Dependciies库，但是以jar包形式加载这个时候还没有进入调试环境，所有即使有Maven Dependciies库也没用，还必须各插件下lib目录有所有的依赖包）
Spoon.bat指定了jre,launcher.jar
因为lib所有jar包都是公用的，所以可以加载外部的jar包插件不报错.
遗留问题1
这里需要理解的是有些插件下面lib目录有些依赖的jar包，有些没有，有些不完整。
从Spoon.java启动调试的不好地方
1.调试插件有些插件缺依赖的jar包，需要拷贝依赖包到编译的插件下的lib目录下
2.system下的karaf（osgi插件），比如编译号的data-integration\system\karaf\system\org\pentaho\di\plugins下的file-open-save-plugin插件加载不了，导致内部插件Text file input插件中的【Browse】打不开对话框，经调试无法osgi的file-open-save-plugin插件
可以从/kettle-ui-swt/src/main/java/org/pentaho/di/ui/core/events/dialog/SelectionAdapterFileDialog.java中的widgetSelectedHelper方法中的
extensionPointWrapper.callExtensionPoint( log,
KettleExtensionPoint.SpoonOpenSaveNew.id, fileDialogOperation );
目前最佳的做法
将整个程序编译打包后的代码pentaho-kettle-9.2.0.3-R\assemblies\client\target下的pdi-ce-9.2.0.3-477.zip解压，将plugins整个目录拷贝到pentaho-kettle-9.2.0.3-R\ui目录下，虽然有些插件报错，但是插件都加载出来，而且可以调试，后续就是解决这些报错问题。
public class PluginRegistryPluginType extends BasePluginType implements PluginTypeInterface {
private static PluginRegistryPluginType INSTANCE = new PluginRegistryPluginType();
public PluginRegistryPluginType() {
super( RegistryPlugin.class, "Plugin Extensions", "Plugin Registry Extension Types" );
populateFolders( "pluginRegistry" );
}
@Override
protected String getXmlPluginFile() {
// This property is set by the import-export.bat/sh command line script to disable this native resource file from
// processing.
if ( !Utils.isEmpty( getProperty( "pentaho.disable.karaf", "" ) ) ) {
return "KettleFileDisabled.xmldisabledxyzabc"; //we return a file that does not exist to suppress exceptions.
}
return Const.XML_FILE_KETTLE_REGISTRY_EXTENSIONS;
}
XML_FILE_KETTLE_REGISTRY_EXTENSIONS = "kettle-registry-extensions.xml"内容如下
<registry-extensions>
<registry-extension id="PdiOsgiBridge">
<description>PDI-OSGI-Bridge Extension</description>
<tooltip/>
<classname>org.pentaho.di.osgi.registryExtension.OSGIPluginRegistryExtension</classname>
<meta-classname/>
<version-browser-classname/>
</registry-extension>
</registry-extensions>
注意
PluginRegistryExtension
ExtensionPointPluginType
项目中有两种实现了PluginRegistryExtension接口类的
org.pentaho.di.osgi.registryExtension.OSGIPluginRegistryExtension
org.pentaho.di.bigdata.ShimDependentPluginRegistryPlugin
这里主要说明OSGIPluginRegistryExtension
PluginRegistry.java中registerType方法中
for ( PluginRegistryExtension ext : extensions ) {
ext.searchForType( pluginType );
}
PluginRegistryExtension接口代码如下
public interface PluginRegistryExtension {
void init( PluginRegistry registry );
void searchForType( PluginTypeInterface pluginType );
String getPluginId( Class<? extends PluginTypeInterface> pluginType, Object
pluginClass );
}
OSGI需另行研究
org.pentaho.di.osgi.OSGIPluginType
通过Everything工具高级搜索OSGIPluginRegistryExtension，找到了pdi-osgi-bridge-core-9.2.0.3-477.jar包，同时又去maven仓库中找到了对应的源代码包pdi-osgi-bridge-core-9.2.0.3-477-sources.jar解压就可以看到OSGIPluginRegistryExtension.java代码实现,同时包含了OsgPlugin及OsgPluginType实现
@RegistryPlugin(id = "OSGIRegistryPlugin", name = "OSGI")
public class OSGIPluginRegistryExtension implements PluginRegistryExtension {
private static OSGIPluginRegistryExtension INSTANCE;
private OSGIPluginTracker tracker = OSGIPluginTracker.getInstance();
private Logger logger = LoggerFactory.getLogger( getClass() );
private KarafBoot boot = new KarafBoot();
private StatusGetter<Boolean> kettleClientEnvironmentInitialized = new StatusGetter<Boolean>() {
@Override public Boolean get() {
return KettleClientEnvironment.isInitialized();
}
};
private AtomicBoolean initializedKaraf = new AtomicBoolean( false );
public OSGIPluginRegistryExtension() {
INSTANCE = this;
}
public static OSGIPluginRegistryExtension getInstance() {
if ( INSTANCE == null ) {
throw new IllegalStateException( "Kettle is supposed to construct this first" );
}
return INSTANCE;
}
// FOR UNIT TEST ONLY
protected static void setInstance( OSGIPluginRegistryExtension instance ) {
INSTANCE = instance;
}
// FOR UNIT TEST ONLY
protected void setTracker( OSGIPluginTracker tracker ) {
this.tracker = tracker;
}
// FOR UNIT TEST ONLY
protected void setLogger( Logger logger ) {
this.logger = logger;
}
// FOR UNIT TEST ONLY
protected void setKettleClientEnvironmentInitialized( StatusGetter<Boolean> kettleClientEnvironmentInitialized ) {
this.kettleClientEnvironmentInitialized = kettleClientEnvironmentInitialized;
}
@VisibleForTesting
void setKarafBoot( KarafBoot boot ){
this.boot = boot;
}
public KarafBoot getKarafBoot(){
return boot;
}
public synchronized void init( final PluginRegistry registry ) {
if ( PentahoSystem.getInitializedStatus() != PentahoSystem.SYSTEM_INITIALIZED_OK && !initializedKaraf.getAndSet( true )) {
String userDir = System.getProperty("pentaho.user.dir", ".");
IApplicationContext context = new StandaloneApplicationContext( userDir, userDir );
PentahoSystem.init(context);
boot.startup( null );
}
PluginRegistry.addPluginType( OSGIPluginType.getInstance() );
tracker.registerPluginClass( PluginInterface.class );
tracker.addPluginLifecycleListener( PluginInterface.class,
new PluginRegistryOSGIServiceLifecycleListener( registry ) );
}
@Override
public void searchForType( PluginTypeInterface pluginType ) {
tracker.registerPluginClass( pluginType.getClass() );
}
@Override
public String getPluginId( Class<? extends PluginTypeInterface> pluginType, Object pluginClass ) {
try {
return (String) tracker.getBeanPluginProperty( pluginType, pluginClass, "ID" );
} catch ( Exception e ) {
logger.error( e.getMessage(), e );
}
return null;
}
}
参考
Missing plugins found while loading a transformation on Kettle
Embed Pentaho Data Integration
Extend Pentaho Data Integration
Kettle — 自定义插件重点理解
eclipse远程调试时出现：Failed to connect to remote VM. Connection refused. Connection refused: connect
Kettle环境初始化源码分析(KettleEnvironment.init())
关于Kettle使用es批量导出插件支持ES5/ES6/ES7的说明
Pentaho Kettle 8.3 源码编译打包及 Debug 调试运行(图文教程)重点看
https://community.hitachivantara.com/communities/developers