使用 Coordinator job 可以執行定時任務和時間觸發執行
需要注意的是 Oozie 默認使用的時區與中國時區不是一致的,需要進行一點修改
1.關於時區
a.修改 core-site.xml 文件(運行需要)需要清除編譯文件,重啟 tomcat 服務(不能是UTC+0800)
<property>
<name>oozie.processing.timezone</name>
<value>GMT+0800</value>
</property>
b.修改 $OOZIE_HOME/oozie-server/webapps/oozie/oozie-console.js 文件(Web顯示需要),無需重啟
function getTimeZone() {
Ext.state.Manager.setProvider(new Ext.state.CookieProvider());
return Ext.state.Manager.get("TimezoneId","GMT+0800");
}
2.拷貝 example 文件 cron-schedule
3.編輯 job.properties 文件(注意時間格式)
nameNode=hdfs://cen-ubuntu.cenzhongman.com:8020
jobTracker=localhost:8032
queueName=default
oozieAppsRoot=oozie-apps
oozie.coord.application.path=${nameNode}/user/cen/${oozieAppsRoot}/cron-schedule
start=2017-07-30T14:40+0800
end=2017-07-30T14:59+0800
workflowAppUri=${nameNode}/user/cen/${oozieAppsRoot}/cron-schedule
4.編輯 workflow.xml 文件(內容酌情添加,這里什么也不做)(修改了版本號)
<workflow-app xmlns="uri:oozie:workflow:0.5" name="no-op-wf">
<start to="end"/>
<end name="end"/>
</workflow-app>
5.編輯 coordinator.xml 文件(支持兩種定時任務方式,下文詳細說明)
<coordinator-app name="cron-coord" frequency="0/1 * * * *" start="${start}" end="${end}" timezone="GMT+0800"
xmlns="uri:oozie:coordinator:0.4">
<action>
<workflow>
<app-path>${workflowAppUri}</app-path>
<configuration>
<property>
<name>jobTracker</name>
<value>${jobTracker}</value>
</property>
<property>
<name>nameNode</name>
<value>${nameNode}</value>
</property>
<property>
<name>queueName</name>
<value>${queueName}</value>
</property>
</configuration>
</workflow>
</action>
</coordinator-app>
注意事項
- 修改時區寫法
- 修改版本號
- coordinator.xml 文件在本地文件系統中讀取,HDFS 中無需修改不影響
6.上傳文件至 HDFS 文件系統
7.執行任務
export OOZIE_URL=http://cen-ubuntu:11000/oozie/
bin/oozie job --config oozie-apps/cron-schedule/job.properties -run
關於定時方式
方式一:官方定義方式
| EL Constant | Value | Example |
|---|---|---|
| ${coord:minutes(int n)} | n | ${coord:minutes(45)} --> 45 |
| ${coord:hours(int n)} | n * 60 | ${coord:hours(3)} --> 180 |
| ${coord:days(int n)} | variable | ${coord:days(2)} --> minutes in 2 full days from the current date |
| ${coord:months(int n)} | variable | ${coord:months(1)} --> minutes in a 1 full month from the current date |
| ${cron syntax} | variable | ${0,10 15 * * 2-6} --> a job that runs every weekday at 3:00pm and 3:10pm UTC time |
方式二:corntab方式
| Field name | Allowed Values | Allowed Special Characters |
|---|---|---|
| Minutes | 0-59 | , - * / |
| Hours | 0-23 | , - * / |
| Day-of-month | 1-31 | , - * ? / L W |
| Month | 1-12 or JAN-DEC | , - * / |
| Day-of-Week | 1-7 or SUN-SAT | , - * ? / L # |
Example
| Cron Expression | Meaning |
|---|---|
| 10 9 * * * | Runs everyday at 9:10am |
| 10,30,45 9 * * * | Runs everyday at 9:10am, 9:30am, and 9:45am |
| 0 * 30 JAN 2-6 | Runs at 0 minute of every hour on weekdays and 30th of January |
| 0/20 9-17 * * 2-5 | Runs every Mon, Tue, Wed, and Thurs at minutes 0, 20, 40 from 9am to 5pm |
| 1 2 L-3 * * | Runs every third-to-last day of month at 2:01am |
| 1 2 6W 3 ? | Runs on the nearest weekday to March, 6th every year at 2:01am |
| 1 2 * 3 3#2 | Runs every second Tuesday of March at 2:01am every year |
| 0 10,13 * * MON-FRI | Runs every weekday at 10am and 1pm |
注1:開啟了檢查頻率,導致5分鍾以內的頻率運行失敗
-
錯誤提示:Error: E1003 : E1003: Invalid coordinator application attributes, Coordinator job with frequency [2] minutes is faster than allowed maximum of 5 minutes (oozie.service.coord.check.maximum.frequency is set to true)
-
錯誤原因:開啟了檢查頻率,導致5分鍾以內的頻率運行失敗
-
解決:關閉頻率檢查功能 配置oozie-site.xml文件
<property> <name>oozie.service.coord.check.maximum.frequency</name> <value>false</value> </property>
