solr常見錯誤


  這兩天給客戶部署獨立的solr服務器遇到的錯誤,現在總結一下。

1. org.apache.solr.client.solrj.SolrServerException: Error executing query

具體的錯誤信息如下:

org.apache.solr.client.solrj.SolrServerException: Error executing query
    at org.apache.solr.client.solrj.request.QueryRequest.process(QueryRequest.java:100)
    at org.apache.solr.client.solrj.SolrServer.query(SolrServer.java:301)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:606)
    at ognl.OgnlRuntime.invokeMethod(OgnlRuntime.java:897)
    at ognl.OgnlRuntime.callAppropriateMethod(OgnlRuntime.java:1299)
    at ognl.ObjectMethodAccessor.callMethod(ObjectMethodAccessor.java:68)
    at com.opensymphony.xwork2.ognl.accessor.XWorkMethodAccessor.callMethodWithDebugInfo(XWorkMethodAccessor.java:117)
    at com.opensymphony.xwork2.ognl.accessor.XWorkMethodAccessor.callMethod(XWorkMethodAccessor.java:108)
    at ognl.OgnlRuntime.callMethod(OgnlRuntime.java:1375)
    at ognl.ASTMethod.getValueBody(ASTMethod.java:91)
    at ognl.SimpleNode.evaluateGetValueBody(SimpleNode.java:212)
    at ognl.SimpleNode.getValue(SimpleNode.java:258)
    at ognl.Ognl.getValue(Ognl.java:470)
    at ognl.Ognl.getValue(Ognl.java:434)
    at com.opensymphony.xwork2.ognl.OgnlUtil$3.execute(OgnlUtil.java:398)
    at com.opensymphony.xwork2.ognl.OgnlUtil.compileAndExecuteMethod(OgnlUtil.java:450)
    at com.opensymphony.xwork2.ognl.OgnlUtil.callMethod(OgnlUtil.java:396)
    at com.opensymphony.xwork2.DefaultActionInvocation.invokeAction(DefaultActionInvocation.java:430)
    at com.opensymphony.xwork2.DefaultActionInvocation.invokeActionOnly(DefaultActionInvocation.java:290)
    at com.opensymphony.xwork2.DefaultActionInvocation.invoke(DefaultActionInvocation.java:251)
    at com.opensymphony.xwork2.interceptor.annotations.AnnotationWorkflowInterceptor.intercept(AnnotationWorkflowInterceptor.java:135)
    at com.opensymphony.xwork2.DefaultActionInvocation.invoke(DefaultActionInvocation.java:245)
    at com.opensymphony.xwork2.interceptor.ConversionErrorInterceptor.intercept(ConversionErrorInterceptor.java:138)
    at com.opensymphony.xwork2.DefaultActionInvocation.invoke(DefaultActionInvocation.java:245)
    at com.opensymphony.xwork2.interceptor.ParametersInterceptor.doIntercept(ParametersInterceptor.java:229)
    at com.opensymphony.xwork2.interceptor.MethodFilterInterceptor.intercept(MethodFilterInterceptor.java:98)
    at com.opensymphony.xwork2.DefaultActionInvocation.invoke(DefaultActionInvocation.java:245)
    at org.apache.struts2.interceptor.ProfilingActivationInterceptor.intercept(ProfilingActivationInterceptor.java:104)
    at com.opensymphony.xwork2.DefaultActionInvocation.invoke(DefaultActionInvocation.java:245)
    at com.opensymphony.xwork2.interceptor.PrepareInterceptor.doIntercept(PrepareInterceptor.java:171)
    at com.opensymphony.xwork2.interceptor.MethodFilterInterceptor.intercept(MethodFilterInterceptor.java:98)
    at com.opensymphony.xwork2.DefaultActionInvocation.invoke(DefaultActionInvocation.java:245)
    at org.apache.struts2.interceptor.ServletConfigInterceptor.intercept(ServletConfigInterceptor.java:164)
    at com.opensymphony.xwork2.DefaultActionInvocation.invoke(DefaultActionInvocation.java:245)
    at com.opensymphony.xwork2.DefaultActionInvocation.invoke(DefaultActionInvocation.java:245)
    at com.opensymphony.xwork2.DefaultActionInvocation.invoke(DefaultActionInvocation.java:245)
    at org.apache.struts2.impl.StrutsActionProxy.execute(StrutsActionProxy.java:54)
    at org.apache.struts2.dispatcher.Dispatcher.serviceAction(Dispatcher.java:575)
    at org.apache.struts2.dispatcher.ng.ExecuteOperations.executeAction(ExecuteOperations.java:81)
    at org.apache.struts2.dispatcher.ng.filter.StrutsExecuteFilter.doFilter(StrutsExecuteFilter.java:93)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
    at org.apache.struts2.dispatcher.ng.filter.StrutsPrepareFilter.doFilter(StrutsPrepareFilter.java:91)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
    at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:76)
    at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:237)
    at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:167)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
    at org.tuckey.web.filters.urlrewrite.RuleChain.handleRewrite(RuleChain.java:176)
    at org.tuckey.web.filters.urlrewrite.RuleChain.doRules(RuleChain.java:145)
    at org.tuckey.web.filters.urlrewrite.UrlRewriter.processRequest(UrlRewriter.java:92)
    at org.tuckey.web.filters.urlrewrite.UrlRewriteFilter.doFilter(UrlRewriteFilter.java:394)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
    at org.springframework.orm.hibernate3.support.OpenSessionInViewFilter.doFilterInternal(OpenSessionInViewFilter.java:198)
    at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:76)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
    at org.springframework.web.filter.CharacterEncodingFilter.doFilterInternal(CharacterEncodingFilter.java:88)
    at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:76)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
    at com.svorx.core.web.filter.NoCacheFilter.doFilterInternal(NoCacheFilter.java:48)
    at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:76)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
    at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:202)
    at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:122)
    at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:505)
    at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:169)
    at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:103)
    at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:116)
    at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:442)
    at org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1083)
    at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:640)
    at org.apache.tomcat.util.net.JIoEndpoint$SocketProcessor.run(JIoEndpoint.java:318)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
    at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
    at java.lang.Thread.run(Thread.java:745)
Caused by: org.apache.http.ParseException: Invalid content type: 
    at org.apache.http.entity.ContentType.parse(ContentType.java:233)
    at org.apache.solr.client.solrj.impl.HttpSolrServer.executeMethod(HttpSolrServer.java:496)
    at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:210)
    at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:206)
    at org.apache.solr.client.solrj.request.QueryRequest.process(QueryRequest.java:91)
    ... 90 more

 

  原因:GET向solr傳遞參數的時候參數過長導致的。如下方法,solrServer是 org.apache.solr.client.solrj.impl.HttpSolrServer 實例

        // 執行查詢
        QueryResponse response = solrServer.query(solrQuery);

 

  解決辦法:改為POST提交

        SolrServer solrServer = new HttpSolrServer("xx");
        SolrQuery solrQuery = new SolrQuery();
        solrServer.query(solrQuery, METHOD.POST);

 

2.  拼接的條件過多,導致出現    too many boolean clauses

解決辦法: 修改solr配置
配置文件位置:SOLRHOME/CORE_NAME/conf/solrconfig.xml

找到如下配置:

<maxBooleanClauses>1024</maxBooleanClauses>

1024 改為10240

其中,SOLRHOME為你的solr主目錄,CORENAME為你的core的名稱

 

3. solr增加索引之后沒有立即生效

可能的原因是:雖然代碼中調用server.add(doc)了,但是沒有進行commit,所以solr不會立即更新索引

解決辦法:增加完索引之后,進行commit。不要add一個就commit一次,太耗性能了,最好是批量add完成之后一次性commit。

solrServer.commit();

  這種提交叫硬提交(hard commit), 使用這種提交會把文檔立即持久化到磁盤,並可以讓你能立馬查詢到它,因為它會開啟一個新的searcher,但是它缺點很明顯,就是很耗性能,並會阻塞到提交任務完成,使用它是非常昂貴的操作。 查看solrj的源碼發現commit()方法的實現是commit(true,true);及waitFlush=true 和 waitSearcher=true。

源碼如下:

  /** 
   * Performs an explicit commit, causing pending documents to be committed for indexing
   * <p>
   * waitFlush=true and waitSearcher=true to be inline with the defaults for plain HTTP access
   * @throws IOException If there is a low-level I/O error.
   */
  public UpdateResponse commit( ) throws SolrServerException, IOException {
    return commit(true, true);
  }

 

  上面手動commit太耗性能了,也可以用solr的配置自動提交,自動提交程序只需要server.add(doc),無需commit()。

  一般也是自動提交結合手動commit來實現,比如你想刪除索引批量重建就可以刪除之后手動commit,然后批量建立索引之后再手動commit;如果是單個文檔添加索引就可以使用solr的自動提交。

 

  在collection/conf/solrconfig.xml配置文件的updateHandler節點,提供了autoCommit和autoSoftCommit兩種提交索引的方式。

    <!-- AutoCommit

         Perform a hard commit automatically under certain conditions.
         Instead of enabling autoCommit, consider using "commitWithin"
         when adding documents. 

         http://wiki.apache.org/solr/UpdateXmlMessages

         maxDocs - Maximum number of documents to add since the last
                   commit before automatically triggering a new commit.

         maxTime - Maximum amount of time in ms that is allowed to pass
                   since a document was added before automatically
                   triggering a new commit. 
         openSearcher - if false, the commit causes recent index changes
           to be flushed to stable storage, but does not cause a new
           searcher to be opened to make those changes visible.

         If the updateLog is enabled, then it's highly recommended to
         have some sort of hard autoCommit to limit the log size.
      -->
     <autoCommit> 
       <maxTime>${solr.autoCommit.maxTime:15000}</maxTime> 
       <openSearcher>false</openSearcher> 
     </autoCommit>

    <!-- softAutoCommit is like autoCommit except it causes a
         'soft' commit which only ensures that changes are visible
         but does not ensure that data is synced to disk.  This is
         faster and more near-realtime friendly than a hard commit.
      -->

     <autoSoftCommit> 
       <maxTime>${solr.autoSoftCommit.maxTime:-1}</maxTime> 
     </autoSoftCommit>

解釋:

autoCommit 硬提交
  maxDocs:add到solr緩沖區的文檔數量超過這個值時,自動執行commit把結果寫入到磁盤索引文件中
  maxTime:最長多久執行一次commit,即如果文檔一直沒有通過commit沉淀到磁盤上,那么持久性就可能出問題,所以要隔一段時間就執行一次commit
  openSearcher:要不要新開一個searcher,如果要新開,那么目前已有的cache就會失效,具體代碼可以看SolrIndexSearcher,它里面有幾個Cache成員變量,如果你新建立一個searcher那么cache失效也是正常的了,除非你有autoWarming機制。在hard commit的時候設置openSearcher=false,那么執行完hard commit之后,磁盤文件雖然改變了,但是searcher沒有新開,需要reload或者重啟,才能看到索引中的新文檔。

一般硬提交配置如下:

     <autoCommit> 
       <maxTime>${solr.autoCommit.maxTime:60000}</maxTime> 
       <openSearcher>true</openSearcher> 
     </autoCommit>

 

autoSoftCommit 軟提交
  自動軟提交只確保改變的索引是可見的,但不保證數據會同步到磁盤。該提交方式比應提交來說更快,更接近實時,就是所謂的支持near real-time (NRT) searching近實時查詢。如果不想執行autoSoftCommit(autoSoftCommit的maxTime設置成-1)。

  軟提交可以根據需求決定是否開啟。

 


免責聲明!

本站轉載的文章為個人學習借鑒使用,本站對版權不負任何法律責任。如果侵犯了您的隱私權益,請聯系本站郵箱yoyou2525@163.com刪除。



 
粵ICP備18138465號   © 2018-2025 CODEPRJ.COM