八:The YARN Timeline Server


一、Overview 介紹
    yarn timeline server用於存儲和檢查應用程序過去和現在的信息(比如job history server)。有兩個功能:
1.Persisting Application Specific Information 保存應用程序特定的信息
    收集和檢索應用程序或者框架的特定信息。如裝mr任務的map\reduce數量通過TimeClient發布到Timeline Server中。然后用戶可以通過wetrest 來查詢這些參數。
2.Persisting Generic Information about Completed Applications 存儲已經執行完成的應用程序的通用信息
    mr任務有jobhistoryServer來保存歷史任務信息,但是jobhistoryServer只是yarn TimeLine server其中的一個應用。通用信息包括:
  • queue-name,   隊列名
  • user information and the like set in the ApplicationSubmissionContext,  用戶信息和applicationSubmissonContext的設置
  • a list of application-attempts that ran for an application   一系列應用程序而非一個應用
  • information about each application-attempt   每個應用程序的信息
  • the list of containers run under each application-attempt 每個應用程序使用的container列表 
  • information about each container. 每個container的信息


二、Current Status and Future Plans 當前狀況和遠景規划

Current status  當前狀況

  1. The core functionality of the timeline server has been completed.  核心功能已經完成
  2. It works in both secure and non secure clusters.  可以工作在安全和非安全模式下
  3. The generic history service is built on the timeline store.  測探歷史服務已經在timeline store中完成
  4. The history can be stored in memory or in a leveldb database store; the latter ensures the history is preserved over Timeline Server restarts. 歷史信息可以保存在內存或者數據庫中,即使timeline server重啟選樣可用
  5. The ability to install framework specific UIs in YARN is not supported.   框架中的特殊UIs不被支持(語法有問題)
  6. Application specific information is only available via RESTful APIs using JSON type content. 應用程序特殊的信息只支持RESTful APIs,格式是JSON
  7. The “Timeline Server v1” REST API has been declared one of the REST APIs whose compatibility will be maintained in future releases.  timeline server V1 的REST APIS是后向兼容的
  8. The single-server implementation of the Timeline Server places a limit on the scalability of the service; it also prevents the service being High-Availability component of the YARN infrastructure.  單節點的timeline server的擴展性問題,而且無法實現HA。

Future Plans 未來規划

  1. Future releases will introduce a next generation timeline service which is scalable and reliable, “Timeline Service v2”.  下一代的timeline server裝是可擴展和穩定的
  2. The expanded features of this service may not be available to applications using the Timeline Server v1 REST API. That includes extended data structures as well as the ability of the client to failover between Timeline Server instances.   使用timeline server v1 的rest api可能無法使用擴展特性,包括可擴展的數據結構和clinet failover

三、Timeline Structure
Timeline domain
    域,同namespace一個概念,將不同的應用和用戶隔開。域中定義和用戶信息、讀寫相關的ACL信息、時間信息等。每個域在yarn中都必須有獨立的ID。
entity
    實體可以是appliction\application attempt的元數據, 及用戶定義的信息等。可以定義實體主過濾器( Primary filters)
event
    實體中應用程序的事件。

四、配置
    默認是不開啟timeline server的。
具體配置請參考:

 














































免責聲明!

本站轉載的文章為個人學習借鑒使用,本站對版權不負任何法律責任。如果侵犯了您的隱私權益,請聯系本站郵箱yoyou2525@163.com刪除。



 
粵ICP備18138465號   © 2018-2025 CODEPRJ.COM