java項目調用kettleJob和Trans


1、調用本地Job和Trans

  較簡單不用多說沒有遇到任何問題,以下是代碼:

import org.pentaho.di.core.KettleEnvironment;
import org.pentaho.di.core.database.DatabaseMeta;
import org.pentaho.di.core.exception.KettleException;
import org.pentaho.di.core.util.EnvUtil;
import org.pentaho.di.job.Job;
import org.pentaho.di.job.JobMeta;
import org.pentaho.di.repository.Repository;
import org.pentaho.di.trans.Trans;
import org.pentaho.di.trans.TransMeta;
import org.pentaho.di.repository.RepositoryDirectoryInterface;
import org.pentaho.di.repository.kdr.KettleDatabaseRepository;
import org.pentaho.di.repository.kdr.KettleDatabaseRepositoryMeta;
public class MyUtils {

    public static void main(String[] args) {

        String[] params = {"1","content","d:\\test1.txt"};
        runTransfer(params, "D:\\一起浪\\數據資源管理平台\\kettle\\test.ktr");
    }
  
    /**  
     * 運行轉換文件方法 
     * @param params 多個參數變量值 
     * @param ktrPath 轉換文件的路徑,后綴ktr
     */  
    public static void runTransfer(String[] params, String ktrPath) {  
        Trans trans = null;  
        try {  
            // // 初始化  
            // 轉換元對象  
            KettleEnvironment.init();// 初始化  
            EnvUtil.environmentInit();  
            TransMeta transMeta = new TransMeta(ktrPath);  
            // 轉換  
            trans = new Trans(transMeta);  
              
            // 執行轉換  
            trans.execute(params);  
            // 等待轉換執行結束  
            trans.waitUntilFinished();  
            // 拋出異常  
            if (trans.getErrors() > 0) {  
                throw new Exception(  
                        "There are errors during transformation exception!(傳輸過程中發生異常)");  
            }  
        } catch (Exception e) {  
            e.printStackTrace();  
        }  
    }  
  
    /** 
     * java 調用 kettle 的job 
     *  
     * @param jobname 
     *            如: String fName= "D:\\kettle\\informix_to_am_4.ktr"; 
     */  
    public static void runJob(String[] params, String jobPath) {  
        try {  
            KettleEnvironment.init();  
            // jobname 是Job腳本的路徑及名稱  
            JobMeta jobMeta = new JobMeta(jobPath, null);  
            Job job = new Job(null, jobMeta);  
            // 向Job 腳本傳遞參數,腳本中獲取參數值:${參數名}  
            // job.setVariable(paraname, paravalue);  
            job.setVariable("id", params[0]);  
            job.setVariable("content", params[1]);  
            job.setVariable("file", params[2]);  
            job.start();  
            job.waitUntilFinished();  
            if (job.getErrors() > 0) {  
                throw new Exception(  
                        "There are errors during job exception!(執行job發生異常)");  
            }  
        } catch (Exception e) {  
            e.printStackTrace();  
        }  
    }  
}
調用本地Job和Trans

2、連接oracle資源庫

  調用資源庫中的Trans,並在本地執行,以下是代碼:

    
    /** 
     * 執行存儲在數據庫資源庫中的轉換 
     *  
     * @Description: 
     * @throws KettleException 
     * @author Jeffy 
     * @since:2016-9-23
     */  
    public static void executeTrans(String transName) throws KettleException {  
        //初始化kettle環境
        KettleEnvironment.init();
        //創建資源庫對象,此時的對象還是一個空對象
        KettleDatabaseRepository repository = new KettleDatabaseRepository();
        //創建資源庫數據庫對象,類似我們在spoon里面創建資源庫
        DatabaseMeta dataMeta = new DatabaseMeta("ETL", "Oracle", "Native(JDBC)", "127.0.0.1", "DBname", "1521",
                "username", "password"); 
        //資源庫元對象,名稱參數,id參數,描述等可以隨便定義
        KettleDatabaseRepositoryMeta kettleDatabaseMeta = 
        new KettleDatabaseRepositoryMeta("ETL", "ETL", "ETL description",dataMeta);
        //給資源庫賦值
        repository.init(kettleDatabaseMeta);
        //連接資源庫
        repository.connect("admin","admin");
        //根據變量查找到模型所在的目錄對象,此步驟很重要。
        RepositoryDirectoryInterface directory = repository.findDirectory("/test");
        //創建ktr元對象
        TransMeta transMeta = ((Repository) repository).loadTransformation(transName, directory, null, true, null ) ;
        //執行參數
        String[] params = {"1","07bb40f7200448","d:\\haha.txt"};
        //創建ktr
        Trans trans = new Trans(transMeta);
        //執行ktr
        trans.execute(params);
        //等待執行完畢
        trans.waitUntilFinished();
        
        if(trans.getErrors()>0)
        {                   
            System.err.println("Transformation run Failure!");
        }
        else
        {
            System.out.println("Transformation run successfully!");
        }
    }  

  過程中,KettleEnvironment.init();語句報錯一般由於jar包沒有引全,可根據錯誤的包名,去找到應該引入卻未引入的jar包。另外,loadTransformation時遇到一個比較特別的錯誤:

2016/10/08 10:32:19 - ETL - ERROR (version 6.1.0.1-196, build 1 from 2016-04-07 12.08.49 by buildguy) : 當讀共享文件時發生錯誤(繼續加載): org.pentaho.di.core.exception.KettleDatabaseException:
2016/10/08 10:32:19 - ETL - Error determining value metadata from SQL resultset metadata
2016/10/08 10:32:19 - ETL - For input string: "4294967295"
2016/10/08 10:32:19 - ETL - ERROR (version 6.1.0.1-196, build 1 from 2016-04-07 12.08.49 by buildguy) : org.pentaho.di.core.exception.KettleDatabaseException:
2016/10/08 10:32:19 - ETL - Error determining value metadata from SQL resultset metadata
2016/10/08 10:32:19 - ETL - For input string: "4294967295"
2016/10/08 10:32:19 - ETL -
2016/10/08 10:32:19 - ETL -     at org.pentaho.di.core.row.value.ValueMetaBase.getValueFromSQLType(ValueMetaBase.java:4588)
2016/10/08 10:32:19 - ETL -     at org.pentaho.di.core.database.Database.getValueFromSQLType(Database.java:2267)
2016/10/08 10:32:19 - ETL -     at org.pentaho.di.core.database.Database.getRowInfo(Database.java:2229)
2016/10/08 10:32:19 - ETL -     at org.pentaho.di.core.database.Database.getRow(Database.java:2341)
2016/10/08 10:32:19 - ETL -     at org.pentaho.di.core.database.Database.getLookup(Database.java:2713)
2016/10/08 10:32:19 - ETL -     at org.pentaho.di.core.database.Database.getLookup(Database.java:2703)
2016/10/08 10:32:19 - ETL -     at org.pentaho.di.core.database.Database.getLookup(Database.java:2699)
2016/10/08 10:32:19 - ETL -     at org.pentaho.di.repository.kdr.delegates.KettleDatabaseRepositoryConnectionDelegate.getTransAttributeRow(KettleDatabaseRepositoryConnectionDelegate.java:619)
2016/10/08 10:32:19 - ETL -     at org.pentaho.di.repository.kdr.delegates.KettleDatabaseRepositoryConnectionDelegate.getTransAttributeString(KettleDatabaseRepositoryConnectionDelegate.java:761)
2016/10/08 10:32:19 - ETL -     at org.pentaho.di.repository.kdr.delegates.KettleDatabaseRepositoryTransDelegate.getTransAttributeString(KettleDatabaseRepositoryTransDelegate.java:1481)
2016/10/08 10:32:19 - ETL -     at org.pentaho.di.repository.kdr.delegates.KettleDatabaseRepositoryTransDelegate.readTransSharedObjects(KettleDatabaseRepositoryTransDelegate.java:1490)
2016/10/08 10:32:19 - ETL -     at org.pentaho.di.repository.kdr.delegates.KettleDatabaseRepositoryTransDelegate.loadTransformation(KettleDatabaseRepositoryTransDelegate.java:500)
2016/10/08 10:32:19 - ETL -     at org.pentaho.di.repository.kdr.KettleDatabaseRepository.loadTransformation(KettleDatabaseRepository.java:278)
2016/10/08 10:32:19 - ETL -     at com.inspur.commons.MyUtils.executeTrans(MyUtils.java:107)
2016/10/08 10:32:19 - ETL -     at com.inspur.commons.NewTest.main(NewTest.java:126)
2016/10/08 10:32:19 - ETL - Caused by: java.lang.NumberFormatException: For input string: "4294967295"
2016/10/08 10:32:19 - ETL -     at java.lang.NumberFormatException.forInputString(NumberFormatException.java:65)
2016/10/08 10:32:19 - ETL -     at java.lang.Integer.parseInt(Integer.java:495)
2016/10/08 10:32:19 - ETL -     at java.lang.Integer.parseInt(Integer.java:527)
2016/10/08 10:32:19 - ETL -     at oracle.jdbc.driver.OracleResultSetMetaData.getPrecision(OracleResultSetMetaData.java:331)
2016/10/08 10:32:19 - ETL -     at org.pentaho.di.core.row.value.ValueMetaBase.getOriginalColumnMetadata(ValueMetaBase.java:4608)
2016/10/08 10:32:19 - ETL -     at org.pentaho.di.core.row.value.ValueMetaBase.getValueFromSQLType(ValueMetaBase.java:4561)
2016/10/08 10:32:19 - ETL -     ... 14 more
2016/10/08 10:32:19 - ETL - ERROR (version 6.1.0.1-196, build 1 from 2016-04-07 12.08.49 by buildguy) : 一個數據庫錯誤發生在從資源庫文件讀取轉換時
2016/10/08 10:32:19 - ETL - org.pentaho.di.core.exception.KettleDatabaseException:
2016/10/08 10:32:19 - ETL - ERROR executing query
2016/10/08 10:32:19 - ETL -
2016/10/08 10:32:19 - ETL - Error determining value metadata from SQL resultset metadata
2016/10/08 10:32:19 - ETL - For input string: "4294967295"

這是由於oracle的jar包版本不對,原本引入的是ojdbc14.jar,去kettle目錄\pdi-ce-6.1.0.1-196\data-integration\lib里找到ojdbc14-10.2.0.1.0.jar,替換掉原來的jar包就可以了。

3、遠程調用

nohup  ./carte.sh  127.0.0.1  8080 > ./log/carte.log &


免責聲明!

本站轉載的文章為個人學習借鑒使用,本站對版權不負任何法律責任。如果侵犯了您的隱私權益,請聯系本站郵箱yoyou2525@163.com刪除。



 
粵ICP備18138465號   © 2018-2025 CODEPRJ.COM