python-crawlab爬蟲管理平台-Docker安裝部署

1. 安裝 Docker

　　Windows 和 Mac下載Docker Desktop

　　下載地址: https://www.docker.com/products/docker-desktop

　　其他系統安裝請參考：https://docs.crawlab.cn/zh/Installation/Docker.html

　　軟件下載后雙擊安裝Docker

2. 下載Crawlab鏡像

　　cmd 執行命令：docker pull tikazyq/crawlab:latest

3. 安裝 Docker-Compose

pip install docker-compose

　　安裝好 docker-compose 后，請運行 docker-compose ps 來測試是否安裝正常。正常的應該是顯示如下內容。

Name   Command   State   Ports
------------------------------
--------------------------------
若顯示ERROR:**** 先操作第4步

4. 打開C:\Users\你的電腦名  目錄下新建文檔：docker-compose.yml  定義如下：

version: '3.3'
services:
  master: 
    image: tikazyq/crawlab:latest
    container_name: master
    environment:
      # CRAWLAB_API_ADDRESS: "https://<your_api_ip>:<your_api_port>"  # backend API address 后端 API 地址. 適用於 https 或者源碼部署
      CRAWLAB_SERVER_MASTER: "Y"  # whether to be master node 是否為主節點，主節點為 Y，工作節點為 N
      CRAWLAB_MONGO_HOST: "mongo"  # MongoDB host address MongoDB 的地址，在 docker compose 網絡中，直接引用服務名稱
      # CRAWLAB_MONGO_PORT: "27017"  # MongoDB port MongoDB 的端口
      # CRAWLAB_MONGO_DB: "crawlab_test"  # MongoDB database MongoDB 的數據庫
      # CRAWLAB_MONGO_USERNAME: "username"  # MongoDB username MongoDB 的用戶名
      # CRAWLAB_MONGO_PASSWORD: "password"  # MongoDB password MongoDB 的密碼
      # CRAWLAB_MONGO_AUTHSOURCE: "admin"  # MongoDB auth source MongoDB 的驗證源
      CRAWLAB_REDIS_ADDRESS: "redis"  # Redis host address Redis 的地址，在 docker compose 網絡中，直接引用服務名稱
      # CRAWLAB_REDIS_PORT: "6379"  # Redis port Redis 的端口
      # CRAWLAB_REDIS_DATABASE: "1"  # Redis database Redis 的數據庫
      # CRAWLAB_REDIS_PASSWORD: "password"  # Redis password Redis 的密碼
      # CRAWLAB_LOG_LEVEL: "info"  # log level 日志級別. 默認為 info
      # CRAWLAB_LOG_ISDELETEPERIODICALLY: "N"  # whether to periodically delete log files 是否周期性刪除日志文件. 默認不刪除
      # CRAWLAB_LOG_DELETEFREQUENCY: "@hourly"  # frequency of deleting log files 刪除日志文件的頻率. 默認為每小時
      # CRAWLAB_SERVER_REGISTER_TYPE: "mac"  # node register type 節點注冊方式. 默認為 mac 地址，也可設置為 ip（防止 mac 地址沖突）
      # CRAWLAB_SERVER_REGISTER_IP: "127.0.0.1"  # node register ip 節點注冊IP. 節點唯一識別號，只有當 CRAWLAB_SERVER_REGISTER_TYPE 為 "ip" 時才生效
      # CRAWLAB_TASK_WORKERS: 8  # number of task executors 任務執行器個數（並行執行任務數）
      # CRAWLAB_RPC_WORKERS: 16  # number of RPC workers RPC 工作協程個數
      # CRAWLAB_SERVER_LANG_NODE: "Y"  # whether to pre-install Node.js 預安裝 Node.js 語言環境
      # CRAWLAB_SERVER_LANG_JAVA: "Y"  # whether to pre-install Java 預安裝 Java 語言環境
      # CRAWLAB_SETTING_ALLOWREGISTER: "N"  # whether to allow user registration 是否允許用戶注冊
      # CRAWLAB_SETTING_ENABLETUTORIAL: "N"  # whether to enable tutorial 是否啟用教程
      # CRAWLAB_NOTIFICATION_MAIL_SERVER: smtp.exmaple.com  # STMP server address STMP 服務器地址
      # CRAWLAB_NOTIFICATION_MAIL_PORT: 465  # STMP server port STMP 服務器端口
      # CRAWLAB_NOTIFICATION_MAIL_SENDEREMAIL: [e-mail]  # sender email 發送者郵箱
      # CRAWLAB_NOTIFICATION_MAIL_SENDERIDENTITY: [e-mail]  # sender ID 發送者 ID
      # CRAWLAB_NOTIFICATION_MAIL_SMTP_USER: username  # SMTP username SMTP 用戶名
      # CRAWLAB_NOTIFICATION_MAIL_SMTP_PASSWORD: password  # SMTP password SMTP 密碼
    ports:    
      - "8080:8080" # frontend port mapping 前端端口映射
    depends_on:
      - mongo
      - redis
    # volumes:
    #   - "/var/crawlab/log:/var/logs/crawlab" # log persistent 日志持久化
  worker:
    image: tikazyq/crawlab:latest
    container_name: worker
    environment:
      CRAWLAB_SERVER_MASTER: "N"
      CRAWLAB_MONGO_HOST: "mongo"
      CRAWLAB_REDIS_ADDRESS: "redis"
    depends_on:
      - mongo
      - redis
    # environment:
    #   MONGO_INITDB_ROOT_USERNAME: username
    #   MONGO_INITDB_ROOT_PASSWORD: password
    # volumes:
    #   - "/var/crawlab/log:/var/logs/crawlab" # log persistent 日志持久化
  mongo:
    image: mongo:latest
    restart: always
    # volumes:
    #   - "/opt/crawlab/mongo/data/db:/data/db"  # make data persistent 持久化
    # ports:
    #   - "27017:27017"  # expose port to host machine 暴露接口到宿主機
  redis:
    image: redis:latest
    restart: always
    # command: redis-server --requirepass "password" # set redis password 設置 Redis 密碼
    # volumes:
    #   - "/opt/crawlab/redis/data:/data"  # make data persistent 持久化
    # ports:
    #   - "6379:6379"  # expose port to host machine 暴露接口到宿主機
  # splash:  # use Splash to run spiders on dynamic pages
  #   image: scrapinghub/splash
  #   container_name: splash
  #   ports:
  #     - "8050:8050"

5. 啟動 Crawlab

cmd下執行： docker-compose up -d  docker中如下即啟動成功，在瀏覽器中輸入 http://localhost:8080 就可以看到界面。

6. 更新/重啟 Crawlab

# 關閉並刪除 Docker 容器
docker-compose down

# 拉取最新鏡像
docker pull tikazyq/crawlab:latest

# 啟動 Docker 容器
docker-compose up -d