三、solr與mysql數據庫結合之批量導入、實時更新數據

一、在自己配置的sorlhome中的core1文件下solrconfig.xml配置

 

 vim solrconfig.xml添加如下配置

<requestHandler name="/dataimport" class="solr.DataImportHandler"> 
<lst name="defaults"> 
<str name="config">solr-data-config.xml</str> 
</lst> 
</requestHandler>

 

二、把solr-dataimporthandler-extras-8.4.0.jar、solr-dataimporthandler-8.4.0.jar、還有mysql連接包mysql-connector-java-5.1.38.jar（這個mysql驅動包要自己下載）

扔到tomcat目錄下的solr：/usr/local/solr/tomcat/apache-tomcat-9.0.30/webapps/solr/WEB-INF/lib

 

 

 

 

三、編輯，solrhome下面的：/usr/local/solr/solrhome/core1/conf中的solr-data-config.xml

<?xml version="1.0" encoding="UTF-8" ?>
<!--
  Licensed to the Apache Software Foundation (ASF) under one or more
  contributor license agreements.  See the NOTICE file distributed with
  this work for additional information regarding copyright ownership.
  The ASF licenses this file to You under the Apache License, Version 2.0
  (the "License"); you may not use this file except in compliance with
  the License.  You may obtain a copy of the License at

      http://www.apache.org/licenses/LICENSE-2.0

  Unless required by applicable law or agreed to in writing, software
  distributed under the License is distributed on an "AS IS" BASIS,
  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
  See the License for the specific language governing permissions and
  limitations under the License.
  -->

<dataConfig>
  <!-- 數據庫信息 -->
  <dataSource type="JdbcDataSource" 
        driver="com.mysql.jdbc.Driver" 
        url="jdbc:mysql://localhost:3306/renren" 
        user="root" password="123456"/>
        
  <!-- document實體 name為數據庫表；pk為managed-schema文件中定義的<uniqueKey>id</uniqueKey> -->    
  <document>
    <!-- document實體 name為數據庫表；pk為managed-schema文件中定義的<uniqueKey>id</uniqueKey> -->
    <entity name="sys_platform" pk="id" 
            query="SELECT `id` as id,
                  `parent_id` as parentId,
                `platform_name` as platformName,
                `longitude` as longitude,
                 `latitude` as latitude,
                  `en_name` as enName,
                `gmt_create` as gmtCreate,
                `gmt_modified` as gmtModified,
                `is_deleted` as isDeleted,
                  concat( longitude,' ',latitude) as GEO
                  FROM sys_platform where is_deleted=0"
                
            deltaQuery="select id from sys_platform where gmt_modified > '${dataimporter.last_index_time}'"     
            deletedPkQuery="select id from sys_platform where is_deleted=0"
            deltaImportQuery="select `id` as id,
                  `parent_id` as parentId,
                `platform_name` as platformName,
                `longitude` as longitude,
                 `latitude` as latitude,
                  `en_name` as enName,
                `gmt_create` as gmtCreate,
                `gmt_modified` as gmtModified,
                `is_deleted` as isDeleted,
                  concat( longitude,' ',latitude) as GEO 
                from sys_platform where id='${dataimporter.delta.id}'"    
    >
    
            <field column="id" name="id"/>
            <field column="parent_id" name="parentId"/>
            <field column="platform_name" name="platformName"/>
            <field column="longitude" name="longitude"/>
            <field column="latitude" name="latitude"/>
            <field column="en_name" name="enName"/>
            <field column="gmt_create" name="gmtCreate"/>
            <field column="gmt_modified" name="gmtModified"/>
            <field column="is_deleted" name="isDeleted"/>        
      </entity>
  </document>
 
</dataConfig>

solr-data-config.xml各配置參數的作用，請參考：https://www.cnblogs.com/davidwang456/p/4744415.html

至於為什么要用concat( longitude,' ',latitude) as GEO，那是因為我要用到空間搜索

 其中以下的域要在managed-schema文件中自定義配置的一致

 

managed-schema文件

 

 

其中solr中的managed-schema默認配置了id的域，並且指定是唯一的，所以我們不用自己去配置了

 

四、配置成功后，即可在管理頁面進行數據導入，如下就意味着配置成功了，可以導入了mysql數據庫中的數據了

 

 

五、如果要實時更新mysql數據庫到solr中，那怎么做呢？接下來，我們開始配置實時數據更新到solr庫

1、打開tomcat下面solr的web.xml配置文件，路徑：/usr/local/solr/tomcat/apache-tomcat-9.0.30/webapps/solr/WEB-INF，添加如下內容

<!-- solr數據導入監聽器 -->
  <listener>
        <listener-class>
       　　org.apache.solr.handler.dataimport.scheduler.ApplicationListener
        </listener-class>
  </listener>

2、導包，在網上找到pache-solr-dataimportscheduler.jar，我的包是重新編譯過了，名稱是apache-solr-dataimportscheduler-recompile.jar，名稱不重要。

然后扔到tomcat目錄下面：/usr/local/solr/tomcat/apache-tomcat-9.0.30/webapps/solr/WEB-INF/lib

3、在solrhome目錄下，新建配置文件: mkdir conf文件夾，並在conf中新建mkdir dataimport.properties文件，注意，一定要在solrhome目錄下，以solr實例core1要在同級目錄，並在dataimport.properties添加以下內容

#################################################
#                                               #
#       dataimport scheduler properties         #
#                                               #
#################################################
 
#  to sync or not to sync 是否同步功能
#  1 - active; anything else - inactive 1 - 開啟; 否則不開啟
syncEnabled=1
#  which cores to schedule
#  in a multi-core environment you can decide which cores you want syncronized
#  leave empty or comment it out if using single-core deployment
#syncCores=game,resource 需要同步的solr core
syncCores=appDbDisasterShelter,appDbMaterialAddress,appDbProtectionobject,appDbRisk,appDbTeam
#  solr server name or IP address solr server 名稱或IP地址
#  [defaults to localhost if empty] 默認為localhost
server=localhost
#  solr server port solr server端口
#  [defaults to 80 if empty] 默認為80
port=8999
#  調度區間
#  默認為30分鍾
interval=1
#  application name/context
#  [defaults to current ServletContextListener's context (app) name]
webapp=solr
#  URL params [mandatory]
#  remainder of URL
params=/dataimport?command=delta-import&clean=false&commit=true
#  schedule interval
#  number of minutes between two runs
#  [defaults to 30 if empty]
#  重做索引的時間間隔，單位分鍾，默認7200，即5天; 
#  為空,為0,或者注釋掉:表示永不重做索引
reBuildIndexInterval=1
 
#  重做索引的參數
reBuildIndexParams=/dataimport?command=full-import&clean=true&commit=true
 
#  重做索引時間間隔的計時開始時間，第一次真正執行的時間=reBuildIndexBeginTime+reBuildIndexInterval*60*1000；
#  兩種格式：2012-04-11 03:10:00 或者  03:10:00，后一種會自動補全日期部分為服務啟動時的日期
reBuildIndexBeginTime=03:10:00

 

六、重啟tomcat，然后再數據庫添加一條數據，就可以在solr后台查詢到添加的結果了

我配置成功所用的solr工具包與要點：https://download.csdn.net/download/caohanren/12116984