ES-密碼設置及JAVA應用

一、開啟密碼驗證

（一）開啟密碼驗證

　　1、安裝x-pack

　　在ES6.3之前添加用戶密碼驗證需要安裝x-pack插件，在6.3之后被去掉，可以直接設置。

　　在es的安裝目錄中，執行：

./bin/elasticsearch-plugin install x-pack

　　2、設置密碼

　　在安裝x-pack后，es的bin目錄中會產生一個x-pack目錄，進入該目錄，進行密碼設置

./setup-passwords interactive

　　該命令會為es、logstash、kibana同時設置密碼，需要手動輸入。

　　　　　　

 　　3、設置es的配置文件

http.cors.enabled: true
http.cors.allow-origin: '*'
http.cors.allow-headers: Authorization,X-Requested-With,Content-Length,Content-Type

　　4、測試

　　不使用用戶名密碼訪問：curl 172.20.10.6:9200，返回信息狀態為401，無權限操作。

　　使用用戶名密碼訪問：curl 172.20.10.6:9200 -u elastic:espass，訪問狀態為成功

　　5、修改用戶密碼：curl -H "Content-Type:application/json" -XPOST -u elastic 'http://127.0.0.1:9200/_xpack/security/user/elastic/_password' -d '{ "password" : "123456" }'

　　6、添加用戶

　　添加用戶接口為：POST/_xpack/security/user/,下面以添加一個test用戶並添加至admin角色為例:

　　　　

 

 　　7、重置超級管理員密碼

　　先停止es，然后使用下述命令創建一個本地認證的超級用戶：

bin/x-pack/users useradd my_admin -p my_password -r superuser

　　然后通過api設置

curl -u my_admin -XPUT 'http://localhost:9200/_xpack/security/user/elastic/_password?pretty' -H 'Content-Type: application/json' -d'
{
  "password" : "new_password"
}

　　驗證：curl -u elastic:123456 'http://172.20.10.6:9200/_xpack/security/_authenticate?pretty'

（二）ES-HEAD和Kibana使用密碼訪問

　　上面對ES做了密碼設置后，訪問ES-HEAD時，就需要加上用戶名密碼進行訪問：http://172.20.10.6:9100/?auth_user=elastic&auth_password=123456

　　對於kibana的設置，需要修改kibana的配置文件，添加訪問es的用戶名密碼

elasticsearch.username: "kibana"
elasticsearch.password: "123456"

二、JAVA API

　　ES的JAVA API很多，這里介紹一個Java High Level REST Client，每個版本變化較大，這里以6.3.2來演示。

（一）客戶端配置

　　pom文件：

<dependency>
            <groupId>org.elasticsearch</groupId>
            <artifactId>elasticsearch</artifactId>
            <version>${elasticsearch.version}</version>
        </dependency>
        <dependency>
            <groupId>org.elasticsearch.client</groupId>
            <artifactId>transport</artifactId>
            <version>${elasticsearch.version}</version>
        </dependency>
        <dependency>
            <groupId>org.elasticsearch.client</groupId>
            <artifactId>elasticsearch-rest-client</artifactId>
            <version>${elasticsearch.version}</version>
        </dependency>
        <dependency>
            <groupId>org.elasticsearch.plugin</groupId>
            <artifactId>transport-netty4-client</artifactId>
            <version>${elasticsearch.version}</version>
        </dependency>
        <dependency>
            <groupId>org.springframework.data</groupId>
            <artifactId>spring-data-elasticsearch</artifactId>
            <version>${spring.data.elasticsearch.version}</version>
        </dependency>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-data-elasticsearch</artifactId>
        </dependency>

        <!--  es升級需要依賴的 -->
        <dependency>
            <groupId>org.elasticsearch.client</groupId>
            <artifactId>elasticsearch-rest-high-level-client</artifactId>
            <version>${elasticsearch.version}</version>
        </dependency>

　　配置文件

elasticsearch:
  clientIps: 172.20.10.6 # ES網關地址，實際地址在ES實例概覽中查看，注意至少配置兩個網關地址
  httpPort: 9200  # ES網關端口
  username: elastic
  password: 123456

　　配置客戶端：下面提供了使用用戶名密碼和不使用用戶名密碼兩個Bean，可以任選其一

@Configuration
public class ElasticSearchClientConfig {
    @Value("${elasticsearch.clientIps}")
    private String clientIps;

    @Value("${elasticsearch.httpPort}")
    private int httpPort;

    @Value("${elasticsearch.username}")
    private String username;

    @Value("${elasticsearch.password}")
    private String password;

    private HttpHost[] getHttpHosts(String clientIps, int esHttpPort) {
        String[] clientIpList = clientIps.split(",");
        HttpHost[] httpHosts = new HttpHost[clientIpList.length];
        for (int i = 0; i < clientIpList.length; i++) {
            httpHosts[i] = new HttpHost(clientIpList[i], esHttpPort, "http");
        }
        return httpHosts;
    }

    /**
     * 創建帶HTTP Basic Auth認證rest客戶端
     */
    @Bean
    public RestHighLevelClient restHighLevelClient() {
        CredentialsProvider credentialsProvider = new BasicCredentialsProvider();
        credentialsProvider.setCredentials(AuthScope.ANY, new UsernamePasswordCredentials(username, password));
        return new RestHighLevelClient(RestClient.builder(getHttpHosts(clientIps, httpPort)).setHttpClientConfigCallback((HttpAsyncClientBuilder httpAsyncClientBuilder) -> httpAsyncClientBuilder.setDefaultCredentialsProvider(credentialsProvider)));
    }

    //不帶用戶名密碼驗證
    //@Bean
    public RestHighLevelClient restClient() {
        return new RestHighLevelClient(RestClient.builder(getHttpHosts(clientIps, httpPort)));
    }
}

（二）索引操作

　　添加索引

@Test
    public void creatIndex() throws Exception{

        Map<String, Object> properties = new HashMap<>();
        Map<String, Object> orderIdIndex = new HashMap<>();
        orderIdIndex.put("type", "long");
        orderIdIndex.put("store", true);
        orderIdIndex.put("index", true);
        properties.put("orderId", orderIdIndex);

        Map<String, Object> orderNameIndex = new HashMap<>();
        orderNameIndex.put("type", "text");
        orderNameIndex.put("store", true);
        orderNameIndex.put("index", true);
        properties.put("orderName", orderNameIndex);

        Map<String, Object> content = new HashMap<>();
        content.put("type", "text");
        content.put("store", true);
        content.put("index", true);
        properties.put("content", content);

        Map<String, Object> mapping = new HashMap<>();
        mapping.put("properties", properties);

        CreateIndexRequest request = new CreateIndexRequest(INDEX);
        request.settings(Settings.builder().put("index.number_of_shards", 3).put("index.number_of_replicas", 2).build());
        request.mapping(TYPE, new Gson().toJson(mapping), XContentType.JSON);

        //request.mapping(mapping);
        CreateIndexResponse response = restHighLevelClient.indices().create(request);
        log.info("response====================={}", new Gson().toJson(response));
    }

　　刪除索引

    @Test
    public void deleteIndex() throws Exception{
        DeleteIndexRequest request = new DeleteIndexRequest(INDEX);
        DeleteIndexResponse response = restHighLevelClient.indices().delete(request);
        log.info("response====================={}", new Gson().toJson(response));
    }

（三）數據操作

　　1、添加數據，添加索引可以使用Json格式、Map格式、XContentBuilder格式等，具體如下所示：

@Test
    public void saveDataByJson() throws Exception{
        OrderInfo orderInfo = OrderInfo.builder().orderId(12345).orderName("測試12345的訂單").content("的撒出手大方【平時都i法第四u發").build();
        IndexRequest request = new IndexRequest(INDEX, TYPE, String.valueOf(orderInfo.getOrderId()));
        request.source(new Gson().toJson(orderInfo), XContentType.JSON);
        this.save(request);
    }

    @Test
    public void saveDataBMap() throws Exception{
        Map<String, Object> jsonMap = new HashMap<>();
        jsonMap.put("orderId", 11111);
        jsonMap.put("orderName", "1111的訂單");
        jsonMap.put("content", "破欸俗人偉大撒打發的");
        IndexRequest request = new IndexRequest(INDEX,TYPE, jsonMap.get("orderId").toString()).source(jsonMap);
        this.save(request);
    }

    @Test
    public void saveDataByContentBuild() throws Exception{
        XContentBuilder builder = XContentFactory.jsonBuilder();
        builder.startObject();
        builder.field("orderId",2222);
        builder.field("orderName", "2222的訂單");
        builder.field("content", "送iu惡意然后我i和拉票；薩拉寬度");
        builder.endObject();
        IndexRequest request = new IndexRequest(INDEX, TYPE, "2222").source(builder);
        this.save(request);
    }

    @Test
    public void saveDataByObject() throws Exception{
        XContentBuilder builder = XContentFactory.jsonBuilder();
        builder.startObject();
        builder.field("orderId",2222);
        builder.field("orderName", "2222的訂單");
        builder.field("content", "送iu惡意然后我i和拉票；薩拉寬度");
        builder.endObject();
        IndexRequest request = new IndexRequest(INDEX, TYPE, "3333").source(
                "orderId",3333,"orderName", "3333的訂單","content", "laisuyeiluqwgkjhgkjsgd"
        );
        this.save(request);
    }
    private boolean save(IndexRequest request) throws Exception{
        IndexResponse response = restHighLevelClient.index(request);
        log.info("response====================={}", new Gson().toJson(response));
        if (response.status().getStatus() == 200) {
            return true;
        }
        return false;
    }

　　2、刪除數據

    @Test
    public void deleteData() throws Exception{
        DeleteRequest request = new DeleteRequest(INDEX,TYPE,"2222");
        DeleteResponse response = restHighLevelClient.delete(request);
        RestStatus restStatus = response.status();
    }

　　3、獲取數據

    @Test
    public void getIncludeData() throws Exception{
        GetRequest request = new GetRequest(INDEX, TYPE, "11111");
        String[] includes = new String[]{"content", "修改"};
        String[] excludes = Strings.EMPTY_ARRAY;
        FetchSourceContext fetchSourceContext = new FetchSourceContext(true, includes, excludes);
        request.fetchSourceContext(fetchSourceContext);
        this.getData(request);
    }

    private void getData(GetRequest request) throws Exception{
        GetResponse response = restHighLevelClient.get(request);
        log.info("response====================={}", new Gson().toJson(response));
        Map<String, Object> source = response.getSource();
        log.info("source====================={}", new Gson().toJson(source));
    }

　　4、更新數據

    @Test
    public void updateByMapping() throws Exception{
        Map<String, Object> jsonMap = new HashMap<>();
        jsonMap.put("orderId", 11111);
        jsonMap.put("orderName", "修改========1111的訂單");
        jsonMap.put("content", "修改==============破欸俗人偉大撒打發的");
        UpdateRequest request = new UpdateRequest(INDEX,TYPE, jsonMap.get("orderId").toString()).doc(jsonMap);
        this.update(request);
    }


    private boolean update(UpdateRequest request) throws Exception{
        UpdateResponse response = restHighLevelClient.update(request);
        RestStatus restStatus = response.status();
        log.info("response====================={}", new Gson().toJson(response));
        if (response.status().getStatus() == 200) {
            return true;
        }
        return false;
    }

　　5、批量添加

    @Test
    public void batch() throws Exception{
        BulkRequest request = new BulkRequest();
        request.add(new IndexRequest(INDEX, TYPE, "5678").source(
                new Gson().toJson(OrderInfo.builder().orderId(5678).orderName("5678的訂單").content("我餓請問一日").build()),
                XContentType.JSON));
        request.add(new IndexRequest(INDEX, TYPE, "56781").source(
                new Gson().toJson(OrderInfo.builder().orderId(56781).orderName("56781的訂單").content("我餓請問一日").build()),
                XContentType.JSON));
        request.add(new IndexRequest(INDEX, TYPE, "56782").source(
                new Gson().toJson(OrderInfo.builder().orderId(56782).orderName("56782的訂單").content("我餓請問一日").build()),
                XContentType.JSON));
        request.add(new IndexRequest(INDEX, TYPE, "56783").source(
                new Gson().toJson(OrderInfo.builder().orderId(56783).orderName("56783的訂單").content("我餓請問一日").build()),
                XContentType.JSON));

        BulkResponse response = restHighLevelClient.bulk(request);
        log.info("response====================={}", new Gson().toJson(response));
    }

　　6、數據查詢

@Test
    public void searchAll() throws Exception{
        SearchRequest request = new SearchRequest(INDEX);
        request.types(TYPE);
        //SearchRequest request = new SearchRequest();
        SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder();
        searchSourceBuilder.query(QueryBuilders.matchAllQuery());
        request.source(searchSourceBuilder);
        this.search(request);
    }

    @Test
    public void searchPage() throws Exception{
        SearchSourceBuilder sourceBuilder = new SearchSourceBuilder();
        sourceBuilder.query(QueryBuilders.termQuery("orderId", "5678"));
        //sourceBuilder.from(2);
        //sourceBuilder.size(5);
        SearchRequest request = new SearchRequest(INDEX);
        request.source(sourceBuilder);
        this.search(request);
    }


    @Test
    public void matchQuery() throws Exception{
        /*QueryBuilder matchQueryBuilder = QueryBuilders.matchQuery("orderId", "5678")
                .fuzziness(Fuzziness.AUTO)
                .prefixLength(3)
                .maxExpansions(10);*/
        SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder();
        //searchSourceBuilder.query(matchQueryBuilder);
        //排序
        //searchSourceBuilder.sort(new ScoreSortBuilder().order(SortOrder.DESC));
        searchSourceBuilder.sort(new FieldSortBuilder("orderId").order(SortOrder.ASC));

        //過濾字段
        String[] includeFields = new String[] {"orderId", "orderName", "content"};
        String[] excludeFields = new String[] {"_type"};
        searchSourceBuilder.fetchSource(includeFields, excludeFields);

        //添加高亮
        highlightBuilder = new HighlightBuilder();
        HighlightBuilder.Field highligthName = new HighlightBuilder.Field("orderName"); //設置高亮字段
        highligthName.highlighterType("unified");
        highlightBuilder.field(highligthName);

        HighlightBuilder.Field highligthContent = new HighlightBuilder.Field("content"); //設置高亮字段
        highligthName.highlighterType("unified");
        highlightBuilder.field(highligthContent);
        SearchRequest request = new SearchRequest(INDEX);
        request.types(TYPE);
        request.source(searchSourceBuilder);
        this.search(request);
    }



    private void search(SearchRequest request) throws Exception{
        SearchResponse response = restHighLevelClient.search(request);
        log.info("response====================={}", new Gson().toJson(response));
        //關於請求執行本身的有用信息，如HTTP狀態代碼、執行時間或請求是否提前終止或超時：
        RestStatus status = response.status();
        TimeValue took = response.getTook();
        Boolean terminatedEarly = response.isTerminatedEarly();
        boolean timedOut = response.isTimedOut();
        log.info("status:{}=======took:{}====terminatedEarly:{}=========timedOut:{}",status,took,terminatedEarly,timedOut);
        //有關碎片級執行的信息，提供了有關受搜索影響的碎片總數以及成功碎片和不成功碎片的統計信息
        int totalShards = response.getTotalShards();
        int successfulShards = response.getSuccessfulShards();
        int failedShards = response.getFailedShards();
        log.info("totalShards:{}=======successfulShards:{}====failedShards:{}",totalShards,successfulShards,failedShards);
        for (ShardSearchFailure failure : response.getShardFailures()) {
            log.info("failure==============={}", new Gson().toJson(failure));
        }
        //將返回的數據返回到SearchHits對象中
        SearchHits hits = response.getHits();
        //SearchHits包含了總點擊次數、最大得分等信息：
        long totalHits = hits.getTotalHits();
        float maxScore = hits.getMaxScore();
        log.info("totalHits:{}=======maxScore:{}",totalHits,maxScore);
        //對SearchHits進行遍歷得到單個的SearchHit對象，SearchHit包含索引、類型、數據id和數據的得分情況
        SearchHit[] searchHits = hits.getHits();
        for (SearchHit hit : searchHits) {
            String index = hit.getIndex();
            String type = hit.getType();
            String id = hit.getId();
            float score = hit.getScore();
            log.info("index:{}=======type:{}=======id:{}=======score:{}",index,type,id,score);
            //可以將數據JSON字符串或鍵/值對映射的形式返回。
            String sourceAsString = hit.getSourceAsString();
            Map<String, Object> sourceAsMap = hit.getSourceAsMap();
            String documentTitle = (String) sourceAsMap.get("title");
            List<Object> users = (List<Object>) sourceAsMap.get("name");
            Map<String, Object> innerObject = (Map<String, Object>) sourceAsMap.get("innerObject");
            log.info("sourceAsString:{}=======documentTitle:{}=======users:{}",sourceAsString,documentTitle,new Gson().toJson(users));
            //設置高亮
            Map<String, HighlightField> highlightFields = hit.getHighlightFields();
            //獲取title高亮顯示
            HighlightField highlight = highlightFields.get("title");
            if(highlight != null){
                //獲取高亮顯示的字段
                Text[] fragments = highlight.fragments();
                String fragmentString = fragments[0].string();
                log.info("fragments:{}=======fragmentString:{}",fragments,fragmentString);
            }
        }
    }