minio開發需注意的事項(springboot)

spring-cloud-admin-minio

minio 分布式部署

minio下載

1 准備 4 台服務器

節點	應用目錄	文件存儲目錄	腳本目錄	日志存儲目錄
192.168.0.100	/usr/local/minio/bin	/usr/local/minio/upload	/usr/local/minio/run	/usr/local/minio/log
192.168.0.101	/usr/local/minio/bin	/usr/local/minio/upload	/usr/local/minio/run	/usr/local/minio/log
192.168.0.102	/usr/local/minio/bin	/usr/local/minio/upload	/usr/local/minio/run	/usr/local/minio/log
192.168.0.103	/usr/local/minio/bin	/usr/local/minio/upload	/usr/local/minio/run	/usr/local/minio/log

注: 在服務器數量少於 4 台的情況下，可以做偽集群部署，此時只需要修改端口號即可:

192.168.0.100:9000
192.168.0.100:9001
192.168.0.100:9002
192.168.0.100:9003

2 創建目錄

4 台服務器都執行以下命令:

# mkdir -p /usr/local/minio/{bin,upload,run,log}

3 下載 minio

把下載的 minio 分別存放到 4 台服務器的 /usr/local/minio/bin 目錄。

4 創建啟動腳本

4 台服務器都執行以下命令:

# vim /usr/local/minio/run/minio-run.sh

5 編輯啟動腳本

5.1 節點 192.168.0.100 的腳本內容

#!/bin/bash
export MINIO_ROOT_USER=admin
export MINIO_ROOT_PASSWORD=password
/usr/local/minio/bin/minio server --address "192.168.0.100:9000" --console-address ":7000" \
http://192.168.0.100:9000/usr/local/minio/upload \
http://192.168.0.101:9000/usr/local/minio/upload \
http://192.168.0.102:9000/usr/local/minio/upload \
http://192.168.0.103:9000/usr/local/minio/upload \
> /usr/local/minio/log/run.log

5.2 節點 192.168.0.101 的腳本內容

#!/bin/bash
export MINIO_ROOT_USER=admin
export MINIO_ROOT_PASSWORD=password
/usr/local/minio/bin/minio server --address "192.168.0.101:9000" --console-address ":7000" \
http://192.168.0.100:9000/usr/local/minio/upload \
http://192.168.0.101:9000/usr/local/minio/upload \
http://192.168.0.102:9000/usr/local/minio/upload \
http://192.168.0.103:9000/usr/local/minio/upload \
> /usr/local/minio/log/run.log

5.3 節點 192.168.0.102 的腳本內容

#!/bin/bash
export MINIO_ROOT_USER=admin
export MINIO_ROOT_PASSWORD=password
/usr/local/minio/bin/minio server --address "192.168.0.102:9000" --console-address ":7000" \
http://192.168.0.100:9000/usr/local/minio/upload \
http://192.168.0.101:9000/usr/local/minio/upload \
http://192.168.0.102:9000/usr/local/minio/upload \
http://192.168.0.103:9000/usr/local/minio/upload \
> /usr/local/minio/log/run.log

5.4 節點 192.168.0.103 的腳本內容

#!/bin/bash
export MINIO_ROOT_USER=admin
export MINIO_ROOT_PASSWORD=password
/usr/local/minio/bin/minio server --address "192.168.0.103:9000" --console-address ":7000" \
http://192.168.0.100:9000/usr/local/minio/upload \
http://192.168.0.101:9000/usr/local/minio/upload \
http://192.168.0.102:9000/usr/local/minio/upload \
http://192.168.0.103:9000/usr/local/minio/upload \
> /usr/local/minio/log/run.log

6 授權 minio 目錄

4 台服務器都執行以下命令:

# chmod +x -R /usr/local/minio

7 將 minio 加入系統服務

4 台服務器都執行以下命令:

# vim /usr/lib/systemd/system/minio.service
[Unit]
Description=Minio service
Documentation=https://docs.minio.io/

[Service]
WorkingDirectory=/usr/local/minio/run/
ExecStart=/usr/local/minio/run/minio-run.sh

Restart=on-failure
RestartSec=5

[Install]
WantedBy=multi-user.target

# chmod +x /usr/lib/systemd/system/minio.service

8 啟動 minio 服務

4 台服務器都執行以下命令:

# systemctl daemon-reload

# systemctl start minio

9 使用 nginx 負載均衡

在日志里增加 "$upstream_status" "$upstream_addr" 以便打印真實的服務端 IP 地址:

http {
    include       mime.types;
    default_type  application/octet-stream;

    log_format  main  '$remote_addr - $remote_user [$time_local] "$request" '
                      '$status $body_bytes_sent "$http_referer" '
                      '"$http_user_agent" "$http_x_forwarded_for"'
					  '"$upstream_status" "$upstream_addr"';
}

9.1 后台調用 endpoint

在 nginx.conf 文件里增加如下配置:

   upstream lb_minio {
        server 192.168.0.100:9000;
        server 192.168.0.101:9000;
        server 192.168.0.102:9000;
        server 192.168.0.103:9000;
    }

    server {
        listen 9000;
        server_name localhost;
		access_log  logs/minio.access.log  main;
		error_log  logs/minio.error.log;
		
        location / {
            proxy_set_header   X-Real-IP $remote_addr;
            proxy_set_header   X-Forwarded-Host  $host:$server_port;
            proxy_set_header   X-Forwarded-For $proxy_add_x_forwarded_for;
            proxy_set_header   X-Forwarded-Proto  $http_x_forwarded_proto;
            proxy_set_header   Host $http_host;
   
            proxy_pass http://lb_minio;
        }
    } 

此時，minio 對外的 ENDPOINT 為: http://${NGINX_HOST}:9000

9.2 前端訪問 console

在 nginx.conf 文件里增加如下配置:

    upstream lb_minio_console {
		server 192.168.0.100:7000;
		server 192.168.0.101:7000;
		server 192.168.0.102:7000;
		server 192.168.0.103:7000;
    }

    server {
        listen 7000;
        server_name localhost;
		access_log  logs/minio.access.log  main;
		error_log  logs/minio.error.log;
		
        location / {
            proxy_pass http://lb_minio_console;
        }
    }

此時，在瀏覽器的地址欄輸入 http://${NGINX_HOST}:7000 即可訪問 minio 的 Console 頁面。

FAQ

minio 服務啟動失敗

• 查看服務日志:

  # journalctl -u minio
  

• ERROR Unable to initialize backend: format.json file: expected format-type: fs, found: xl

  刪除文件存儲目錄里的 .minio.sys文件:

  # ls -la
  total 0
  drwxr-xr-x. 3 root root  24 Jan 12 11:53 .
  drwxr-xr-x. 6 root root  53 Jan 12 11:08 ..
  drwxr-xr-x. 9 root root 125 Jan 12 14:14 .minio.sys
  
  # rm -rf .minio.sys
  

minio 開發需注意的事項

<dependency>
    <groupId>io.minio</groupId>
    <artifactId>minio</artifactId>
    <version>8.3.4</version>
</dependency>

升級 okhttp3

okhttp3 的版本不能低於 4.8.1，升級的時候，只需要在父模塊的 properties 里配置 okhttp3 版本即可。

    <properties>
        <okhttp3.version>4.9.3</okhttp3.version>
    </properties>

源碼: .\io\minio\minio\8.3.4\minio-8.3.4.jar!\io\minio\S3Base.class

  static {
    try {
      RequestBody.create(new byte[0], (MediaType)null);
    } catch (NoSuchMethodError var1) {
      throw new RuntimeException("Unsupported OkHttp library found. Must use okhttp >= 4.8.1", var1);
    }

    DEFAULT_CONNECTION_TIMEOUT = TimeUnit.MINUTES.toMillis(5L);
    TRACE_QUERY_PARAMS = ImmutableSet.of("retention", "legal-hold", "tagging", "uploadId");
  }

命名 bucket 的名稱

bucket 的名稱必須滿足正則表達式: ^[a-z0-9][a-z0-9\\.\\-]+[a-z0-9]$

源碼: .\io\minio\minio\8.3.4\minio-8.3.4.jar!\io\minio\BucketArgs.class

  protected void validateBucketName(String name) {
    this.validateNotNull(name, "bucket name");
    if (name.length() >= 3 && name.length() <= 63) {
      String msg;
      if (name.contains("..")) {
        msg = "bucket name cannot contain successive periods. For more information refer http://docs.aws.amazon.com/AmazonS3/latest/dev/BucketRestrictions.html";
        throw new IllegalArgumentException(name + " : " + msg);
      } else if (!name.matches("^[a-z0-9][a-z0-9\\.\\-]+[a-z0-9]$")) {
        msg = "bucket name does not follow Amazon S3 standards. For more information refer http://docs.aws.amazon.com/AmazonS3/latest/dev/BucketRestrictions.html";
        throw new IllegalArgumentException(name + " : " + msg);
      }
    } else {
      throw new IllegalArgumentException(name + " : bucket name must be at least 3 and no more than 63 characters long");
    }
  }

正確的命名:

.bucket("example.minio")
.bucket("example-minio")
.bucket("example.minio-bucket")
.bucket("example-minio.bucket")

命名 object 的名稱

object 方法會根據層級自動創建目錄。

.object("example.jpg")
.object("/example.jpg")
.object("level1/level2/example.jpg")
.object("/level1/level2/example.jpg")

指定 ContentType

上傳文件的時候，盡量指定 ContentType，如果不指定的話，ContentType 默認為: application/octet-stream

源碼: .\io\minio\minio\8.3.4\minio-8.3.4.jar!\io\minio\PutObjectArgs.class

    public String contentType() throws IOException {
        String contentType = super.contentType();
        return contentType != null ? contentType : "application/octet-stream";
    }

• FilePart(webflux)

  List<String> contentTypeList = file.headers().get("Content-Type");
  .contentType((contentTypeList == null || contentTypeList.isEmpty()) ? null : contentTypeList.get(0))
  

• MultipartFile(web)

  .contentType(file.getContentType())
  

nginx 傳輸文件的限制

nginx 默認文件傳輸的大小是 1M，傳輸超過 1M 的文件會報異常: 413 Request Entity Too Large，此時可以在 http{ } 或 server{ } 或 location{ } 中添加以下配置:

	client_body_buffer_size 10m;
	client_max_body_size 10m;

• http{ }: 作用於全局 nginx
• server{ }: 作用於當前 server
• location{ }: 作用於當前路由