Centos7 搭建minio集群

一、准備機器

獲取最新更新以及文章用到的軟件包，請移步點擊：查看更新

　　1、准備四台機器，（minio集群最少四台）。

192.168.223.131 minio-1
192.168.223.128 minio-2
192.168.223.129 minio-3
192.168.223.130 minio-4

　　2、編輯hosts文件，將以上內容添加到hosts中

vim /etc/hosts

      

　　部署(所有機器均執行)
　　以下的操作都需要在四台機器上執行

　　3、創建掛載磁盤路徑

mkdir -p /data/minio_data/

　　4、掛載磁盤路徑到文件系統
　　注意：需要將新建的目錄掛在到對應的磁盤下,磁盤不掛載好，集群啟動會報錯，還需要注意的是掛載的文件系統至少要1G不然無法初始化導致集群報錯

　　文件系統 容量 已用 可用 已用% 掛載點

[root@minio-1 minio]# df -h
文件系統                                容量  已用  可用 已用% 掛載點
devtmpfs                                470M     0  470M    0% /dev
tmpfs                                   487M     0  487M    0% /dev/shm
tmpfs                                   487M  8.4M  478M    2% /run
tmpfs                                   487M     0  487M    0% /sys/fs/cgroup
/dev/mapper/centos_hadoop--master-root   47G   12G   36G   25% /
/dev/sda1                              1014M  240M  775M   24% /boot
tmpfs                                    98M     0   98M    0% /run/user/0
tmpfs                                    98M   12K   98M    1% /run/user/42
————————————————

　　5、將上面掛載磁盤路徑掛載到相應的文件系統上

mount /dev/sda1 /data/minio_data/

　　6、查看掛載信息

       

　　7、創建minio目錄

cd   /data/minio_data/

　　8、下載安裝包

wget http://dl.minio.org.cn/server/minio/release/linux-amd64/minio
wget https://dl.min.io/client/mc/release/linux-amd64/mc

　　9、賦執行權限（根據情況，這里賦全部權限）

chmod +x minio mc

　　10、創建啟動腳本，編輯run.sh文件

mkdir /data/minio_data && cd /data/minio_data
內容如下：
cat > run.sh <<EOF
#!/bin/bash
export MINIO_ACCESS_KEY=minio
export MINIO_SECRET_KEY=Leo825#20210423
/usr/local/bin/minio server --address=192.168.81.235:9000 http://192.168.81.235/data/minio_data/data1  http://192.168.81.236/data/minio_data/data1  http://192.168.81.237/data/minio_data/data1 http://192.168.81.234/data/minio_data/data1
EOF

　　11、賦執行權限（根據情況，這里賦全部權限）

chmod 777 /data/minio_data/run.sh

　　12、創建啟動服務，創建minio.service啟動腳本

vim /usr/lib/systemd/system/minio.service

內容如下：

[Unit]
Description=Minio service
Documentation=https://docs.minio.io/

[Service]
#安裝包路徑
WorkingDirectory=/data/minio_data
#啟動命令路徑
ExecStart=/data/minio_data/run.sh

Restart=on-failure
RestartSec=5

[Install]
WantedBy=multi-user.target

　　13、啟動測試（所有機器執行）

重新加載服務的配置文件
systemctl daemon-reload

啟動minio服務
systemctl start minio

查看minio狀態
systemctl status minio
[root@minio-2 ~]# systemctl status minio
● minio.service - Minio service
Loaded: loaded (/usr/lib/systemd/system/minio.service; disabled; vendor preset: disabled)
Active: active (running) since 日 2021-01-31 17:22:54 CST; 17s ago
Docs: https://docs.minio.io/
Main PID: 2036 (run.sh)
Tasks: 8
CGroup: /system.slice/minio.service
├─2036 /bin/bash /data/minio_data/run.sh
└─2039 /data/minio_data server http://192.168.223.232/data/minio_data/data1 http://192.168.223.233/data/minio_data/data1

關閉minio服務
systemctl stop minio

　　14、訪問地址

集群中的任何一台機器都可以訪問：

http://192.168.223.132:9000/
http://192.168.223.133:9000/
http://192.168.223.134:9000/
http://192.168.223.135:9000/

　　15、創建測試桶

        

　　16、上傳測試

       

　　17、主機上可以查看到上傳的文件

       

 二、nginx配置文件服務器訪問

　　1、執行命令

mc  alias set minio http://192.168.223.132:9000/ minio Leo825#20210423 --api S3v4
開啟匿名訪問
mc policy set public minio/sy01 

　　2、web頁面開啟匿名訪問

　　 

 

 　　

　　3、http訪問，sy01是桶名稱，方便瀏覽器訪問。

upstream minio-server{
　　server 192.168.6.124:9000 weight=25 max_fails=2 fail_timeout=30s;
　　server 192.168.6.125:9000 weight=25 max_fails=2 fail_timeout=30s;
　　server 192.168.6.126:9000 weight=25 max_fails=2 fail_timeout=30s;
　　server 192.168.6.128:9000 weight=25 max_fails=2 fail_timeout=30s;
}

server {
   listen 8888;
   server_name 192.168.6.120;

   #To allow special characters in headers
   ignore_invalid_headers off;
   # Allow any size file to be uploaded.
   # Set to a value such as 1000m; to restrict file size to a specific value
   client_max_body_size 0;
   # To disable buffering
   proxy_buffering off;

   location /sy01/ {
      proxy_set_header X-Real-IP $remote_addr;
      proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
      proxy_set_header X-Forwarded-Proto $scheme;
      proxy_set_header Host $http_host;

      proxy_connect_timeout 300;
      # Default is HTTP/1, keepalive is only enabled in HTTP/1.1
      proxy_http_version 1.1;
      proxy_set_header Connection "";
      chunked_transfer_encoding off;

      proxy_pass http://minio-server;
    }

    location / {
      proxy_set_header X-Real-IP $remote_addr;
      proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
      proxy_set_header X-Forwarded-Proto $scheme;
      proxy_set_header Host $http_host;

      proxy_connect_timeout 300;
      # Default is HTTP/1, keepalive is only enabled in HTTP/1.1
      proxy_http_version 1.1;
      proxy_set_header Connection "";
      chunked_transfer_encoding off;

      proxy_pass http://minio-server; # If you are using docker-compose this would be the hostname i.e. minio
      # Health Check endpoint might go here. See https://www.nginx.com/resources/wiki/modules/healthcheck/
      # /minio/health/live;
    }
}

　　4、https訪問，minio是負載minio服務，sy01是桶名稱，方便瀏覽器訪問。

upstream minio-server{
        server 192.168.6.124:9000 weight=25 max_fails=2 fail_timeout=30s;
        server 192.168.6.125:9000 weight=25 max_fails=2 fail_timeout=30s;
        server 192.168.6.126:9000 weight=25 max_fails=2 fail_timeout=30s;
        server 192.168.6.128:9000 weight=25 max_fails=2 fail_timeout=30s;
}

server {
     listen 443 ssl;
     server_name  192.168.6.120;

     ssl_certificate /etc/nginx/ssl/192.168.6.120.crt;
     ssl_certificate_key /etc/nginx/ssl/192.168.6.120.key;

     location /sy01/ {
          proxy_set_header X-Real-IP $remote_addr;
          proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
          proxy_set_header X-Forwarded-Proto $scheme;
          proxy_set_header Host $http_host;

          proxy_connect_timeout 300;
          proxy_http_version 1.1;
          proxy_set_header Connection "";
          chunked_transfer_encoding off;
          proxy_pass http://minio-server;
   }

   location /minio {
      proxy_set_header X-Real-IP $remote_addr;
      proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
      proxy_set_header X-Forwarded-Proto $scheme;
      proxy_set_header Host $http_host;

      proxy_connect_timeout 300;
      proxy_http_version 1.1;
      proxy_set_header Connection "";
      chunked_transfer_encoding off;
      proxy_pass http://minio-server; 
   }  
}

　　 或者換成tcp代理也可以實現，放在http層外，如果服務器在內網，訪問在外網，中間經過了代理，或者隧道之內的，必須要要tcp，否則會http頭部hred驗證會失敗。

upstream minio-server {
        server 192.168.178.40:9000     max_fails=3 fail_timeout=30s;
        server 192.168.178.41:9000     max_fails=3 fail_timeout=30s;
        server 192.168.178.42:9000     max_fails=3 fail_timeout=30s;
        server 192.168.178.43:9000     max_fails=3 fail_timeout=30s;
}
server {
        listen 9000;
        proxy_connect_timeout 2s;
        proxy_timeout 900s;
        proxy_pass minio-server;
}

        5、瀏覽器訪問效果：