Hive日志(Hive Logging)--hive GettingStarted翻譯


Hive uses log4j for logging. By default logs are not emitted to the console by the CLI. The default logging level is WARN for Hive releases prior to 0.13.0. Starting with Hive 0.13.0, the default logging level is INFO.

Hive使用log4j記錄日志,默認日志不輸出到cli控制台。默認日志級別是warn在hive 0.13.0之前,從hive 0.13.0開始,默認日志輸出級別為info。

The logs are stored in the directory /tmp/<user.name>:

日志默認存儲在以下目錄中:

· /tmp/<user.name>/hive.log

比如:/tmp/root/hive.log

Note: In local mode, prior to Hive 0.13.0 the log file name was ".log" instead of "hive.log". This bug was fixed in release 0.13.0 (see HIVE-5528 and HIVE-5676).

這是一個Bug的說明。

To configure a different log location, set hive.log.dir in $HIVE_HOME/conf/hive-log4j.properties. Make sure the directory has the sticky bit set (chmod 1777 <dir>).

設置一個不同的hive日志位置,通過在$HIVE_HOME/conf/hive-log4j.properties文件中設置hive.log.dir參數。確定這個路徑是一個粘滯位權限。

· hive.log.dir=<other_location>

If the user wishes, the logs can be emitted to the console by adding the arguments shown below:

如果你願意,日志也可以通過如下命令顯示在控制台中:

· bin/hive --hiveconf hive.root.logger=INFO,console  //for HiveCLI (deprecated)

· bin/hiveserver2 --hiveconf hive.root.logger=INFO,console

Alternatively, the user can change the logging level only by using:

當然,也可以通過如下方式改變日志級別:

· bin/hive --hiveconf hive.root.logger=INFO,DRFA //for HiveCLI (deprecated)

· bin/hiveserver2 --hiveconf hive.root.logger=INFO,DRFA

Another option for logging is TimeBasedRollingPolicy (applicable for Hive 0.15.0 and above, HIVE-9001) by providing DAILY option as shown below:

· bin/hive --hiveconf hive.root.logger=INFO,DAILY //for HiveCLI (deprecated)

· bin/hiveserver2 --hiveconf hive.root.logger=INFO,DAILY

Note that setting hive.root.logger via the 'set' command does not change logging properties since they are determined at initialization time.

注意通過set命令設置hive.root.logger屬性不能改變日志記錄屬性,因為它是在初始化時確定的。

Hive also stores query logs on a per Hive session basis in /tmp/<user.name>/, but can be configured in hive-site.xml with the hive.querylog.location property.

Hive也存儲查詢日志在每一個hive session記錄在/tmp/<user.name>/,但是可以在hive-site.xml文件中配置屬性ive.querylog.location。

Logging during Hive execution on a Hadoop cluster is controlled by Hadoop configuration. Usually Hadoop will produce one log file per map and reduce task stored on the cluster machine(s) where the task was executed. The log files can be obtained by clicking through to the Task Details page from the Hadoop JobTracker Web UI.

當hive在hadoop集群執行期間產生日志是由hadoop配置屬性控制的。實際上hadoop會為每一個map或reduce任務產生一個日志文件並且存儲在集群上任務執行的那個計算機上。這些日志信息在hadoop web UI界面通過點擊任務詳細信息看到。

When using local mode (using mapreduce.framework.name=local), Hadoop/Hive execution logs are produced on the client machine itself. Starting with release 0.6 – Hive uses the hive-exec-log4j.properties (falling back to hive-log4j.properties only if it's missing) to determine where these logs are delivered by default. The default configuration file produces one log file per query executed in local mode and stores it under /tmp/<user.name>. The intent of providing a separate configuration file is to enable administrators to centralize execution log capture if desired (on a NFS file server for example). Execution logs are invaluable for debugging run-time errors.

當運行在本地模式時(using mapreduce.framework.name=local),hadoop/hive運行日志產生在自己機器的客戶端產生。從0.6版本開始,hive使用hive-exec-log4j.properties配置文件決定日志保存位置。默認的配置將產生一個log文件記錄在本地模式運行的每一個執行動作,並且存儲log在/tmp/<user.name>下。提供一個單獨的配置文件的目的是如果管理員需要,可以集中捕獲執行日志。執行日志對於提供運行時debug錯誤來說是非常重要的。

For information about WebHCat errors and logging, see Error Codes and Responses and Log Files in the WebHCat manual.

Error logs are very useful to debug problems. Please send them with any bugs (of which there are many!) to hive-dev@hadoop.apache.org.

From Hive 2.1.0 onwards (with HIVE-13027), Hive uses Log4j2's asynchronous logger by default. Setting hive.async.log.enabled to false will disable asynchronous logging and fallback to synchronous logging. Asynchronous logging can give significant performance improvement as logging will be handled in a separate thread that uses the LMAX disruptor queue for buffering log messages. Refer to https://logging.apache.org/log4j/2.x/manual/async.html for benefits and drawbacks.

從hive 2.1.0開始,hive使用log4j2方法實現異步日志方法。設置hive.async.log.enabled為false將會關閉異步日志功能並且回退到同步日志模式。使用異步日志功能可以極大改善性能,日志處理使用一個應用了LMAX disruptor queue的單獨線程緩存消息隊列。

HiveServer2 Logs

HiveServer2 operation logs are available to clients starting in Hive 0.14. See HiveServer2 Logging for configuration.

Hiveserver2操作日志從hive0.14開始提供給用戶。

Audit Logs

Audit logs are logged from the Hive metastore server for every metastore API invocation.

審計日志記錄hive元數據的訪問和每一個api針對元數據的調用。

An audit log has the function and some of the relevant function arguments logged in the metastore log file. It is logged at the INFO level of log4j, so you need to make sure that the logging at the INFO level is enabled (see HIVE-3505). The name of the log entry is "HiveMetaStore.audit".

Audit logs were added in Hive 0.7 for secure client connections (HIVE-1948) and in Hive 0.10 for non-secure connections (HIVE-3277; also see HIVE-2797).

Perf Logger

In order to obtain the performance metrics via the PerfLogger, you need to set DEBUG level logging for the PerfLogger class (HIVE-12675). This can be achieved by setting the following in the log4j properties file.

log4j.logger.org.apache.hadoop.hive.ql.log.PerfLogger=DEBUG

If the logger level has already been set to DEBUG at root via hive.root.logger, the above setting is not required to see the performance logs.


免責聲明!

本站轉載的文章為個人學習借鑒使用,本站對版權不負任何法律責任。如果侵犯了您的隱私權益,請聯系本站郵箱yoyou2525@163.com刪除。



 
粵ICP備18138465號   © 2018-2025 CODEPRJ.COM