apache flink kubernetes 運行試用

類似docker-compose 運行模式，使用的是deploy 的模式

deploy yaml 文件

deploy-k8s-yaml
apiVersion: extensions/v1beta1
kind: Deployment
metadata:  
  name: jobmanager
  namespace: big-data
  labels:
    app: jobmanager
spec:
  replicas: 1
  template:
    metadata:
      labels:
        app: jobmanager
    spec:
      containers:
      - name: jobmanager
        image: flink
        imagePullPolicy: IfNotPresent
        ports:
        - containerPort: 6123
        - containerPort: 8081
        args: ["jobmanager"]
        env:
        - name: JOB_MANAGER_RPC_ADDRESS
          value: jobmanager

---

# 暴露service 方便taskmanager 連接
 apiVersion: v1
kind: Service
metadata:
  name: jobmanager
  namespace: big-data
  labels:
    app: jobmanager
spec:
  type: NodePort
  ports:
  - port: 8081
    targetPort: 8081
    protocol: TCP
    name: http
  - port: 6123
    targetPort: 6123
    protocol: TCP
    name: rpc
  - port: 6124
    targetPort: 6124
    protocol: TCP
    name: blob
  - port: 6125
    targetPort: 6125
    protocol: TCP
    name: query
  selector:
    app: jobmanager
---

# 訪問地址使用ingress
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: jobmanager
  namespace: big-data
spec:
  rules:
  - host: flink.data.com
    http:
      paths:
      - backend:
          serviceName: jobmanager
          servicePort: 8081
        path: /

---

apiVersion: extensions/v1beta1
kind: Deployment
metadata:  
  name: taskmanager
  namespace: big-data
  labels:
    app: taskmanager
spec:
  replicas: 3
  template:
    metadata:
      labels:
        app: taskmanager
    spec:
      containers:
      - name: taskmanager
        image: flink
        imagePullPolicy: IfNotPresent
        ports:
        - containerPort: 6121
        - containerPort: 6122
        args: ["taskmanager"]
        env:
        - name: JOB_MANAGER_RPC_ADDRESS
          value: jobmanager

部署

kubectl apply -f  deploy-k8s-yaml

訪問

簡單batchjob demo

具體代碼參考 https://github.com/rongfengliang/flink-docker-compose-demo/tree/master/flink-app

運行效果

參考資料

https://github.com/rongfengliang/flink-docker-compose-demo
https://github.com/apache/flink/tree/master/flink-examples/flink-examples-batch/src/main/java/org/apache/flink/examples/java/wordcount