[k8s]一些技巧性的yaml和dockerfile-docker學習思路

我會把一些dockerfile和yaml的技巧性東西不定期搜集到這里

docker學習思路

三部曲:

• 1,vm會搭建服務
• 2,docker會跑服務
• 3,k8s集群會調度該服務

存儲

• vm1 vm2 共享存儲,vm1的容器掛了直接調度到vm2(docker存儲驅動實現)

網絡

• 物理機安裝flannel,使兩台docker上容器能通(物理機/容器搭建etcd集群)
• 物理機搭建openvswitch,實現兩台docker

監控

• cadvisor會跑
• 物理機安裝grafana,物理機安裝promethus(實現nodeexplore和mysqlexplore)
• 容器跑promethus +cadvisor+grafana
• 容器跑cadvisor+influendb+grafana
• 容器跑elk,物理機跑filebeat搜集容器日志

pod的一些技巧

參考: https://jimmysong.io/kubernetes-handbook/appendix/tricks.html

• 在容器中獲取 Pod 的IP
• 指定容器的啟動參數
• 讓Pod調用宿主機的docker能力
• 使用Init container初始化應用配置
• 使容器內時間與宿主機同步
• 在Pod中獲取宿主機的主機名、namespace等
• 配置Pod使用外部DNS

dockerfile

tomcat 啟動

EXPOSE 8080
CMD ["catalina.sh", "run"]

nginx日志和啟動

# forward request and error logs to docker log collector
RUN ln -sf /dev/stdout /var/log/nginx/access.log \
	&& ln -sf /dev/stderr /var/log/nginx/error.log

EXPOSE 80

STOPSIGNAL SIGTERM

CMD ["nginx", "-g", "daemon off;"]

yaml 讓busybox當計數器

apiVersion: v1
kind: Pod
metadata:
  name: counter
spec:
  containers:
  - name: count
    image: busybox
    args: [/bin/sh, -c, 'i=0; while true; do echo "$i: $(date)"; i=$((i+1)); sleep 1; done']

相當於

docker run -d --name=b1 busybox  i=0; while true; do echo "$i: $(date)"; i=$((i+1)); sleep 1; done

docker run -d \
           --log-driver=fluentd \
           --log-opt fluentd-address=localhost:24224 \
           --log-opt tag="log-test-container-A" \
           busybox sh -c 'while true; do echo "This is a log message from container A"; sleep 10; done;'

yaml 讓busybox執行一個腳本

參考: https://kubernetes.io/docs/concepts/cluster-administration/logging/

apiVersion: v1
kind: Pod
metadata:
  name: counter
spec:
  containers:
  - name: count
    image: busybox
    args:
    - /bin/sh
    - -c
    - >
      i=0;
      while true;
      do
        echo "$i: $(date)" >> /var/log/1.log;
        echo "$(date) INFO $i" >> /var/log/2.log;
        i=$((i+1));
        sleep 1;
      done
    volumeMounts:
    - name: varlog
      mountPath: /var/log
  - name: count-log-1
    image: busybox
    args: [/bin/sh, -c, 'tail -n+1 -f /var/log/1.log']
    volumeMounts:
    - name: varlog
      mountPath: /var/log
  - name: count-log-2
    image: busybox
    args: [/bin/sh, -c, 'tail -n+1 -f /var/log/2.log']
    volumeMounts:
    - name: varlog
      mountPath: /var/log
  volumes:
  - name: varlog
    emptyDir: {}

yaml 讓busybox睡5min

參考: https://kubernetes.io/docs/concepts/services-networking/dns-pod-service/#dns-policy

apiVersion: v1
kind: Pod
metadata:
  name: busybox
  namespace: default
spec:
  containers:
  - image: busybox
    command:
      - sleep
      - "3600"
    imagePullPolicy: IfNotPresent
    name: busybox
  restartPolicy: Always

dockerfile拷貝目錄的一個坑

參考: https://stackoverflow.com/questions/26504846/copy-directory-to-other-directory-at-docker-using-add-command

## 拷貝文件不需要寫目標
FROM centos
COPY 2.txt /usr/local/

## 拷貝目錄則需要這樣寫,目標,不然拷貝不進去
FROM centos
COPY mysql /usr/local/mysql

設置容器的TZ另一種辦法

## override default time zone (Etc/UTC) if TZ variable is set
if [ ! -z "$TZ" ]; then
  ln -snf /usr/share/zoneinfo/$TZ /etc/localtime && echo $TZ > /etc/timezone
fi

k8s集群里pod高負載訪問apiserver的8080(api對外地址)

因為api對內地址是443
集群api如果有3台,則需要負載訪問,咋辦呢? 自定義svc,endpoint,實現對外負載(如果只有一個apiserver,則直連即可)

kind: Endpoints
apiVersion: v1
metadata:
  name: kube-apiserver-http
  namespace: kube-public
subsets:
- addresses:
  - ip: 192.168.x.132
  - ip: 192.168.x.133
  - ip: 192.168.x.134
  ports:
  - name: http
    port: 8080
    protocol: TCP

kind: Service
apiVersion: v1
metadata:
  labels:
    app-name: kube-apiserver-http
  name: kube-apiserver-http
  namespace: kube-public
spec:
  ports:
  - name: http
    port: 80
    targetPort: 8080
    protocol: TCP
  sessionAffinity: ClientIP

command寫法和容器privileged模式設置(cni部署網絡)/容器獲取宿主機(外界/或集群ns的名字)參數

參考: https://github.com/coreos/flannel/blob/master/Documentation/kube-flannel.yml
https://feisky.gitbooks.io/kubernetes/network/flannel/#cni集成
http://cizixs.com/2017/05/23/container-network-cni

        command: [ "/opt/bin/flanneld", "--ip-masq", "--kube-subnet-mgr" ]
        securityContext:
          privileged: true

        env:
        - name: POD_NAME
          valueFrom:
            fieldRef:
              fieldPath: metadata.name
        - name: POD_NAMESPACE
          valueFrom:
            fieldRef:
              fieldPath: metadata.namespace

參考

    env:
    - name: MY_POD_NAME
      valueFrom:
        fieldRef:
          fieldPath: metadata.name
    - name: MY_POD_NAMESPACE
      valueFrom:
        fieldRef:
          fieldPath: metadata.namespace
    - name: MY_POD_IP
      valueFrom:
        fieldRef:
          fieldPath: status.podIP