Window中調試HBase問題小結

　　1.好久沒用log4j了，轉到logback好多年了，hbase程序運行時，報缺少log4j配置，那么，就轉去logback吧（以下的XXX表示版本號）。

　　原先lib包里面有log4j-XXX.jar、slf4j-api-XXX.jar、slf4j-log4j12-XXX.jar，干掉log4j-XXX.jar和slf4j-log4j12-XXX.jar，加入jcl-over-slf4j-XXX.jar，log4j-over-slf4j-XXX.jar、logback-classic-XXX.jar、logback-core-XXX.jar，至於原先的slf4j-api這個包，可以順手替換成新的，然后，加入一個logback.xml到classpath下

<?xml version="1.0" encoding="UTF-8"?>

<!-- For assistance related to logback-translator or configuration  -->
<!-- files in general, please contact the logback user mailing list -->
<!-- at http://www.qos.ch/mailman/listinfo/logback-user             -->
<!--                                                                -->
<!-- For professional support please see                            -->
<!--    http://www.qos.ch/shop/products/professionalSupport         -->
<!--                                                                -->
<configuration scan="true" scanPeriod="120 seconds" debug="false">
  <appender name="FileApp" class="ch.qos.logback.core.rolling.RollingFileAppender">
    <!--See also http://logback.qos.ch/manual/appenders.html#RollingFileAppender-->
    <filter class="ch.qos.logback.classic.filter.ThresholdFilter"> 
      <level>WARN</level> 
    </filter>
    <File>log/HBaseClient.log</File>
    <encoder>
      <pattern>%d [%t] %-5p %c - %m%n</pattern>
    </encoder>
    <rollingPolicy class="ch.qos.logback.core.rolling.FixedWindowRollingPolicy">
      <maxIndex>10</maxIndex>
      <FileNamePattern>log/HBaseClient.log.%i</FileNamePattern>
    </rollingPolicy>
    <triggeringPolicy class="ch.qos.logback.core.rolling.SizeBasedTriggeringPolicy">
      <MaxFileSize>1024KB</MaxFileSize>
    </triggeringPolicy>
  </appender>
  <appender name="ConApp" class="ch.qos.logback.core.ConsoleAppender">
    <encoder>
      <pattern>%d [%t] %-5p %c - %m%n</pattern>
    </encoder>
  </appender> 
  <root level="ERROR">
      <appender-ref ref="ConApp"/>
    <appender-ref ref="FileApp"/>
  </root>
</configuration>

　　這樣，log4j和系統的日志系統就被logback接管了，至於為啥不用log4j，用logback，這個。。。自己去看吧，有空我可能會寫點啥，logback有點小bug，就是有的linux系統下會找不到配置文件，以前修復過，不知道新版本有處理掉沒。

　　2.Could not locate executable null\bin\winutils.exe in the Hadoop binaries

　　老掉牙的問題了，系統變量設置HADOOP_HOME，我還是不想去設置環境變量，還是一行代碼來得快

System.setProperty("hadoop.home.dir", "G:/hadoop/hadoop-2.4.1");

　　3.Unable to load native-hadoop library for your platform... using builtin-java classes where applicable

　　本地庫是為了提高性能和包含一些java不支持的實現，比如集群中設置支持gzip lzo壓縮后，在對壓縮文件進行讀取或者對輸入文件壓縮的時候要使用到hadoop的本地庫。

　　找不到本地庫，linux下可以自己編譯一下，官方文檔中也有介紹http://hadoop.apache.org/docs/r2.4.1/hadoop-project-dist/hadoop-common/NativeLibraries.html。

　　windows下。。。。我以前是在https://github.com/srccodes/hadoop-common-2.2.0-bin/tree/master/bin下載的，但是我win7 64位，就用不了，注冊dll文件的時候，就報錯了。官方文檔中寫着“The native hadoop library is supported on *nix platforms only.”。。。

　　找到一個windows下編譯hadoop本地庫的地址http://www.srccodes.com/p/article/38/build-install-configure-run-apache-hadoop-2.2.0-microsoft-windows-os

　　看了一遍，想想不划算，算了，到時候肯定是丟去linux系統下跑的，折騰這個干嘛啊，linux下又沒有問題。

　　4.程序運行正常，但是當log級別改為Info時，突然發現了這么一條

2014-08-04 16:19:22,942 [main-SendThread(Master:2222)] INFO  org.apache.zookeeper.ClientCnxn - Opening socket connection to server Master/192.168.117.128:2222. Will not attempt to authenticate using SASL (java.lang.SecurityException: 無法定位登錄配置)

　　看到個Exception，什么情況？覺得哪里不對，但是程序又是一切正常的。

　　搜索了一通，發現好多一樣的問題，但是貌似和我的沒啥關系，別人是程序跑不通，我是程序正常得很。

　　有說是zookeeper版本不一致的，我看了下hadoop和hbase中的確實有區別，一個zookeeper-3.4.5.jar，一個zookeeper-3.4.6.jar，但是我總覺得不對，新舊怎么可能會不兼容啊。那就直接統一了試試，連客戶端程序的都統一了，結果那個info信息還是存在，就和不換是一樣的，果然不是這里的問題！

　　有說是配置和hosts的，我這里都沒這問題的，都配置過了的。加參數？我在配置中都寫好的，直接打印一看就知道有了。

　　這只是一個info信息，很奇怪為啥是個info，照說info級別的都是不影響啥的信息，程序確實是很正常的跑，但是看到這個信息就是不舒服。

　　自己查代碼去算了，根據日志，是在ClientCnxn.java中出現的信息，好吧，查看源碼

private void logStartConnect(InetSocketAddress addr) {
            String msg = "Opening socket connection to server " + addr;
            if (zooKeeperSaslClient != null) {
              msg += ". " + zooKeeperSaslClient.getConfigStatus();
            }
            LOG.info(msg);
        }

　　汗，還得換去查ZooKeeperSaslClient.java，繼續吧

public ZooKeeperSaslClient(final String serverPrincipal)
            throws LoginException {
        /**
         * ZOOKEEPER-1373: allow system property to specify the JAAS
         * configuration section that the zookeeper client should use.
         * Default to "Client".
         */
        String clientSection = System.getProperty(ZooKeeperSaslClient.LOGIN_CONTEXT_NAME_KEY, "Client");
        // Note that 'Configuration' here refers to javax.security.auth.login.Configuration.
        AppConfigurationEntry entries[] = null; RuntimeException runtimeException = null; try { entries = Configuration.getConfiguration().getAppConfigurationEntry(clientSection); } catch (SecurityException e) { // handle below: might be harmless if the user doesn't intend to use JAAS authentication.
            runtimeException = e; } catch (IllegalArgumentException e) { // third party customized getAppConfigurationEntry could throw IllegalArgumentException when JAAS // configuration isn't set. We can reevaluate whether to catch RuntimeException instead when more // different types of RuntimeException found
            runtimeException = e; } if (entries != null) { this.configStatus = "Will attempt to SASL-authenticate using Login Context section '" + clientSection + "'"; this.saslClient = createSaslClient(serverPrincipal, clientSection); } else { // Handle situation of clientSection's being null: it might simply because the client does not intend to // use SASL, so not necessarily an error.
            saslState = SaslState.FAILED; String explicitClientSection = System.getProperty(ZooKeeperSaslClient.LOGIN_CONTEXT_NAME_KEY); if (explicitClientSection != null) { // If the user explicitly overrides the default Login Context, they probably expected SASL to // succeed. But if we got here, SASL failed.
                if (runtimeException != null) { throw new LoginException("Zookeeper client cannot authenticate using the " + explicitClientSection +
                            " section of the supplied JAAS configuration: '" + System.getProperty(Environment.JAAS_CONF_KEY) + "' because of a " +
                            "RuntimeException: " + runtimeException); } else { throw new LoginException("Client cannot SASL-authenticate because the specified JAAS configuration " +
                            "section '" + explicitClientSection + "' could not be found."); } } else { // The user did not override the default context. It might be that they just don't intend to use SASL, // so log at INFO, not WARN, since they don't expect any SASL-related information.
                String msg = "Will not attempt to authenticate using SASL "; if (runtimeException != null) {  msg += "(" + runtimeException + ")"; } else { msg += "(unknown error)"; } this.configStatus = msg; this.isSASLConfigured = false; } if (System.getProperty(Environment.JAAS_CONF_KEY)  != null) { // Again, the user explicitly set something SASL-related, so they probably expected SASL to succeed.
                if (runtimeException != null) { throw new LoginException("Zookeeper client cannot authenticate using the '" + System.getProperty(ZooKeeperSaslClient.LOGIN_CONTEXT_NAME_KEY, "Client") +
                            "' section of the supplied JAAS configuration: '" + System.getProperty(Environment.JAAS_CONF_KEY) + "' because of a " +
                            "RuntimeException: " + runtimeException); } else { throw new LoginException("No JAAS configuration section named '" + System.getProperty(ZooKeeperSaslClient.LOGIN_CONTEXT_NAME_KEY, "Client") +
                            "' was found in specified JAAS configuration file: '" + System.getProperty(Environment.JAAS_CONF_KEY) + "'."); } } } }

　　根據上面的代碼，可知問題在於紅色部分，再注意一下上面的注釋，我靠，果然只是一個普通的說明文字，但是，尼瑪能不能不在info信息里面搞個Exception提示出來？？？

　　好吧，那確實是一個info信息，是無關緊要的info信息，其實只是說明了。。。。。由於客戶端沒用SASL，所以不使用SASL進行驗證。。。

　　能不能換個好點的說明文字？我這些操作都是多余的了！人家就那樣顯示的，唉。看得不爽也沒辦法了，難道為了這點東西還去改源碼不成？夠汗的。　　

　　解決辦法，客戶端添加SASL信息，然后服務端也改掉，添加驗證，這個官方文檔http://hbase.apache.org/book.html#security有說明了，直接看去就是了，這里不說明了。

　　唉，處女座追求完美的心，果然是不變的。。。加上，看不到煩心的文字，舒坦了。。