【原創】大叔經驗分享(4)Yarn ResourceManager頁面如何實現主被自動切換


hdfs、yarn、hbase這些組件的master支持多個,實現自動主備切換,其中hdfs、hbase無論訪問主master或者備master都可以正常訪問頁面,但是yarn比較特別,只有主master的頁面可以訪問,備master會返回Refresh,3s后重定向;

一種方式是提供兩個域名,分別對應兩個yarn的master,一旦有master切換,需要手工切換到另外一個,有沒有更好的方式?

 

訪問備master過程如下:

curl http://standby_ip:8088/cluster -v

* About to connect() to standby_ip port 8088 (#0)

*   Trying standby_ip...

* Connected to standby_ip (standby_ip) port 8088 (#0)

> GET /cluster HTTP/1.1

> User-Agent: curl/7.29.0

> Host: standby_ip:8088

> Accept: */*

>

< HTTP/1.1 200 OK

< Cache-Control: no-cache

< Expires: Tue, 25 Sep 2018 03:59:22 GMT

< Date: Tue, 25 Sep 2018 03:59:22 GMT

< Pragma: no-cache

< Expires: Tue, 25 Sep 2018 03:59:22 GMT

< Date: Tue, 25 Sep 2018 03:59:22 GMT

< Pragma: no-cache

< Content-Type: text/plain; charset=UTF-8

< Refresh: 3; url=http://active_ip:8088/cluster

< Content-Length: 103

< Server: Jetty(6.1.26)

<

This is standby RM. Redirecting to the current active RM: http://active_ip:8088/cluster

* Connection #0 to host standby_ip left intact

可見備master響應http status為200,包含Refresh頭,同時body為This is standby RM. Redirecting to the current active RM:***

有沒有可能在load balancer(比如nginx)上配置實現自動切換?這里非法響應需要判斷header包含‘Refresh’或者判斷body包含‘This is standby RM’

 

1)被動切換,支持http status以及timeout等判斷,不滿足

http://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_next_upstream

proxy_next_upstream error | timeout | invalid_header | http_500 | http_502 | http_503 | http_504 | http_403 | http_404 | http_429 | non_idempotent | off ...;

 

2)主動健康檢查,最常用的開源module,只支持http status級別的健康檢查,不滿足

https://github.com/yaoweibin/nginx_upstream_check_module

  check_http_expect_alive

    syntax: *check_http_expect_alive [ http_2xx | http_3xx | http_4xx |

    http_5xx ]*

    default: *http_2xx | http_3xx*

    context: *upstream*

    description: These status codes indicate the upstream server's http

    response is ok, the backend is alive.

 

3)nginx收費版本的ngx_http_upstream_hc_module,支持header和body的判斷,可以滿足

Dynamically configurable group with periodic health checks is available as part of our commercial subscription:

http://nginx.org/en/docs/http/ngx_http_upstream_hc_module.html

    match active {

        body !~ "This is standby RM";

    }

 


免責聲明!

本站轉載的文章為個人學習借鑒使用,本站對版權不負任何法律責任。如果侵犯了您的隱私權益,請聯系本站郵箱yoyou2525@163.com刪除。



 
粵ICP備18138465號   © 2018-2025 CODEPRJ.COM