前情提要:
web服務往華為雲上遷移
================內網的好環境，相關配置===================
1.web服務關於ES的集群配置如下：
elasticAddress1=192.168.6.16
elasticAddress2=192.168.6.17
elasticPort1=9300
elasticPort2=9301
elasticClusterName=elasticsearch-crm
是在內網兩台服務器上啟動的ES集群，分別配置的IP和端口如上.
ES的配置文件內容如下：
#192.168.6.16 ES配置文件
cluster.name: elasticsearch-crm
node.name: "node-1"
node.master: true
node.data: true
network.host : 192.168.6.16
http.port : 9200
transport.tcp.port: 9300
discovery.zen.ping.unicast.hosts: ["192.168.6.17:9301"]
discovery.zen.fd.connect_on_network_disconnect : true
discovery.zen.initial_ping_timeout : 10s
discovery.zen.fd.ping_interval : 2s
discovery.zen.fd.ping_retries : 5
#192.168.6.17 ES配置文件
cluster.name: elasticsearch-crm
node.name: "node-2"
node.master: true
node.data: true
network.host : 192.168.6.17
http.port : 9201
transport.tcp.port: 9301
discovery.zen.ping.unicast.hosts: ["192.168.6.16:9300"]
discovery.zen.fd.connect_on_network_disconnect : true
discovery.zen.initial_ping_timeout : 10s
discovery.zen.fd.ping_interval : 2s
discovery.zen.fd.ping_retries : 5
==================遷移的環境，一開始的配置========================
2.華為雲上ES啟動單節點，和web服務在同一服務器上，因此配置如下：
web服務配置如下：
elasticAddress1=127.0.0.1
elasticAddress2=127.0.0.1
elasticPort1=9200
elasticPort2=9200
elasticClusterName=elasticsearch-crm
同和web服務在一台服務器上的ES的配置文件如下[docker啟動]：[只用了單節點，沒有啟動多節點的集群]
#集群名
cluster.name: elasticsearch-crm
node.master: true
node.data: true
network.host : 127.0.0.1
http.port : 9200
transport.tcp.port: 9300
docker啟動命令如下：[docker鏡像使用版本為6.5.4]
docker run -itd --name es1 -p 9200:9200 -p 9300:9300 --restart=always -v /mnt/apps/es/elasticsearch.yml:/usr/share/elasticsearch/config/elasticsearch.yml -v /mnt/apps/es/data:/usr/share/elasticsearch/data --net=host elasticsearch:6.5.4
報錯開始：
報錯1：
Exception in thread "main" NoNodeAvailableException[None of the configured nodes are available: [{#transport#-1}{GRphuiRLRgOVTP0707mGFQ}{127.0.0.1}{127.0.0.1:9200}]]
at org.elasticsearch.client.transport.TransportClientNodesService.ensureNodesAreAvailable(TransportClientNodesService.java:347)
at org.elasticsearch.client.transport.TransportClientNodesService.execute(TransportClientNodesService.java:245)
at org.elasticsearch.client.transport.TransportProxyClient.execute(TransportProxyClient.java:60)
at org.elasticsearch.client.transport.TransportClient.doExecute(TransportClient.java:378)
at org.elasticsearch.client.support.AbstractClient.execute(AbstractClient.java:405)
at org.elasticsearch.client.support.AbstractClient.execute(AbstractClient.java:394)
at org.elasticsearch.action.ActionRequestBuilder.execute(ActionRequestBuilder.java:46)
at org.elasticsearch.action.ActionRequestBuilder.get(ActionRequestBuilder.java:53)
at com.es.test.EsTest.addIndex1(EsTest.java:70)
at com.es.test.EsTest.main(EsTest.java:89)
錯誤原因：
ES提供的端口號
9200作為Http協議，主要用於外部通訊
9300作為Tcp協議，jar之間就是通過tcp協議通訊
ES集群之間是通過9300進行通訊，web服務連接調用ES，也是使用9300端口
修改位置：
將web服務的連接ES配置更改為如下：
elasticAddress1=127.0.0.1
elasticAddress2=127.0.0.1
elasticPort1=9300
elasticPort2=9300
elasticClusterName=elasticsearch-crm
報錯2：
Exception in thread "main" NoNodeAvailableException[None of the configured nodes are available: [{#transport#-1}{GRphuiRLRgOVTP0707mGFQ}{127.0.0.1}{127.0.0.1:9300}]]
at org.elasticsearch.client.transport.TransportClientNodesService.ensureNodesAreAvailable(TransportClientNodesService.java:347)
at org.elasticsearch.client.transport.TransportClientNodesService.execute(TransportClientNodesService.java:245)
at org.elasticsearch.client.transport.TransportProxyClient.execute(TransportProxyClient.java:60)
at org.elasticsearch.client.transport.TransportClient.doExecute(TransportClient.java:378)
at org.elasticsearch.client.support.AbstractClient.execute(AbstractClient.java:405)
at org.elasticsearch.client.support.AbstractClient.execute(AbstractClient.java:394)
at org.elasticsearch.action.ActionRequestBuilder.execute(ActionRequestBuilder.java:46)
at org.elasticsearch.action.ActionRequestBuilder.get(ActionRequestBuilder.java:53)
at com.es.test.EsTest.addIndex1(EsTest.java:70)
at com.es.test.EsTest.main(EsTest.java:89)
錯誤原因：
web服務中配置連接使用127.0.0.1不可以。
應該使用本機IP配置才行。
本機IP，應該web服務和ES服務的配置文件都更改為使用本機真實IP
修改位置：
將web服務連接ES配置更改為：
elasticAddress1=10.10.10.60
elasticAddress2=10.10.10.60
elasticPort1=9300
elasticPort2=9300
elasticClusterName=elasticsearch-crm
將ES配置文件中的配置內容改為：
#集群名
cluster.name: elasticsearch-crm
node.master: true
node.data: true
network.host : 10.10.10.60
http.port : 9200
transport.tcp.port: 9300
==============本類問題總結========================
java程序連接ES服務，報錯NoNodeAvailableException[None of the configured nodes are available，解決的思路應該是：
第一、配置文件有問題，web服務和ES配置文件中，雙方都應該
1.IP應該用本機真實IP
2.ES這邊注意啟動的端口是多少
3.web服務這邊應該使用9300或者ES啟用的另外的TCP端口，而不能使用9200端口
4.注意雙方的集群名稱 ，一定要一致
第二、這里確認一下，原環境中ES集群是兩個ES節點，新環境中是單個ES節點
配置依舊寫成：
elasticAddress1=10.10.10.60
elasticAddress2=10.10.10.60
elasticPort1=9300
elasticPort2=9300
elasticClusterName=elasticsearch-crm
一度懷疑，java程序識別不到可用節點，是不是因為我雙節點的配置是重復的，這個問題引起的。
經過測試，這樣重復的配置雖然不合適，但是完全不影響使用。
所以這一點也排除！
第三、經過上述兩點，確認配置確實無誤之后，需要排查一下，ES節點是不是真正的正常啟動起來了？
如果你的ES雖然啟動了，但是並沒有啟動成功，那肯定連不上的呀。
【至於因為網絡問題，防火牆問題，端口未開放的問題，這些都是可排查的原因，但是這里我的WEB和ES是在同一台服務器上，所以排除這個問題】
=================新一類問題開啟==========================
經過上面的問題解決和排查，配置方面沒有問題了，現在看ES這邊到底是什么鬼。。。。
這邊啟動ES，查看日志報錯
報錯3：
[2018-06-15T09:56:53,019][INFO ][o.e.n.Node ] [node1] initializing ...
[2018-06-15T09:56:53,141][INFO ][o.e.e.NodeEnvironment ] [node1] using [1] data paths, mounts [[/data (/dev/mapper/data-data)]], net usable_space [179.8gb], net total_space [179.9gb], spins? [possibly], types [xfs]
[2018-06-15T09:56:53,142][INFO ][o.e.e.NodeEnvironment ] [node1] heap size [1.9gb], compressed ordinary object pointers [true]
[2018-06-15T09:56:53,143][INFO ][o.e.n.Node ] [node1] node name [node1], node ID [F0PzQ9qSRPWq8YNcwjj0vg]
[2018-06-15T09:56:53,144][INFO ][o.e.n.Node ] [node1] version[5.6.0], pid[27627], build[781a835/2017-09-07T03:09:58.087Z], OS[Linux/3.10.0-514.el7.x86_64/amd64], JVM[Oracle Corporation/OpenJDK 64-Bit Server VM/1.8.0_102/25.102-b14]
[2018-06-15T09:56:53,144][INFO ][o.e.n.Node ] [node1] JVM arguments [-Xms2g, -Xmx2g, -XX:+UseConcMarkSweepGC, -XX:CMSInitiatingOccupancyFraction=75, -XX:+UseCMSInitiatingOccupancyOnly, -XX:+AlwaysPreTouch, -Xss1m, -Djava.awt.headless=true, -Dfile.encoding=UTF-8, -Djna.nosys=true, -Djdk.io.permissionsUseCanonicalPath=true, -Dio.netty.noUnsafe=true, -Dio.netty.noKeySetOptimization=true, -Dio.netty.recycler.maxCapacityPerThread=0, -Dlog4j.shutdownHookEnabled=false, -Dlog4j2.disable.jmx=true, -Dlog4j.skipJansi=true, -XX:+HeapDumpOnOutOfMemoryError, -Des.path.home=/usr/local/elasticsearch-5.6.0]
[2018-06-15T09:56:54,161][INFO ][o.e.p.PluginsService ] [node1] loaded module [aggs-matrix-stats]
[2018-06-15T09:56:54,162][INFO ][o.e.p.PluginsService ] [node1] loaded module [ingest-common]
[2018-06-15T09:56:54,162][INFO ][o.e.p.PluginsService ] [node1] loaded module [lang-expression]
[2018-06-15T09:56:54,162][INFO ][o.e.p.PluginsService ] [node1] loaded module [lang-groovy]
[2018-06-15T09:56:54,162][INFO ][o.e.p.PluginsService ] [node1] loaded module [lang-mustache]
[2018-06-15T09:56:54,162][INFO ][o.e.p.PluginsService ] [node1] loaded module [lang-painless]
[2018-06-15T09:56:54,163][INFO ][o.e.p.PluginsService ] [node1] loaded module [parent-join]
[2018-06-15T09:56:54,163][INFO ][o.e.p.PluginsService ] [node1] loaded module [percolator]
[2018-06-15T09:56:54,163][INFO ][o.e.p.PluginsService ] [node1] loaded module [reindex]
[2018-06-15T09:56:54,163][INFO ][o.e.p.PluginsService ] [node1] loaded module [transport-netty3]
[2018-06-15T09:56:54,163][INFO ][o.e.p.PluginsService ] [node1] loaded module [transport-netty4]
[2018-06-15T09:56:54,164][INFO ][o.e.p.PluginsService ] [node1] no plugins loaded
[2018-06-15T09:56:55,941][INFO ][o.e.d.DiscoveryModule ] [node1] using discovery type [zen]
[2018-06-15T09:56:56,770][INFO ][o.e.n.Node ] [node1] initialized
[2018-06-15T09:56:56,770][INFO ][o.e.n.Node ] [node1] starting ...
[2018-06-15T09:56:57,059][WARN ][o.e.b.ElasticsearchUncaughtExceptionHandler] [node1] uncaught exception in thread [main]
org.elasticsearch.bootstrap.StartupException: BindTransportException[Failed to bind to [9300-9400]]; nested: BindException[Cannot assign requested address];
at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:136) ~[elasticsearch-5.6.0.jar:5.6.0]
at org.elasticsearch.bootstrap.Elasticsearch.execute(Elasticsearch.java:123) ~[elasticsearch-5.6.0.jar:5.6.0]
at org.elasticsearch.cli.EnvironmentAwareCommand.execute(EnvironmentAwareCommand.java:67) ~[elasticsearch-5.6.0.jar:5.6.0]
at org.elasticsearch.cli.Command.mainWithoutErrorHandling(Command.java:134) ~[elasticsearch-5.6.0.jar:5.6.0]
at org.elasticsearch.cli.Command.main(Command.java:90) ~[elasticsearch-5.6.0.jar:5.6.0]
at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:91) ~[elasticsearch-5.6.0.jar:5.6.0]
at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:84) ~[elasticsearch-5.6.0.jar:5.6.0]
Caused by: org.elasticsearch.transport.BindTransportException: Failed to bind to [9300-9400]
at org.elasticsearch.transport.TcpTransport.bindToPort(TcpTransport.java:771) ~[elasticsearch-5.6.0.jar:5.6.0]
at org.elasticsearch.transport.TcpTransport.bindServer(TcpTransport.java:736) ~[elasticsearch-5.6.0.jar:5.6.0]
at org.elasticsearch.transport.netty4.Netty4Transport.doStart(Netty4Transport.java:173) ~[?:?]
at org.elasticsearch.common.component.AbstractLifecycleComponent.start(AbstractLifecycleComponent.java:69) ~[elasticsearch-5.6.0.jar:5.6.0]
at org.elasticsearch.transport.TransportService.doStart(TransportService.java:209) ~[elasticsearch-5.6.0.jar:5.6.0]
at org.elasticsearch.common.component.AbstractLifecycleComponent.start(AbstractLifecycleComponent.java:69) ~[elasticsearch-5.6.0.jar:5.6.0]
at org.elasticsearch.node.Node.start(Node.java:694) ~[elasticsearch-5.6.0.jar:5.6.0]
at org.elasticsearch.bootstrap.Bootstrap.start(Bootstrap.java:278) ~[elasticsearch-5.6.0.jar:5.6.0]
at org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:351) ~[elasticsearch-5.6.0.jar:5.6.0]
at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:132) ~[elasticsearch-5.6.0.jar:5.6.0]
... 6 more
Caused by: java.net.BindException: Cannot assign requested address
at sun.nio.ch.Net.bind0(Native Method) ~[?:?]
at sun.nio.ch.Net.bind(Net.java:433) ~[?:?]
at sun.nio.ch.Net.bind(Net.java:425) ~[?:?]
at sun.nio.ch.ServerSocketChannelImpl.bind(ServerSocketChannelImpl.java:223) ~[?:?]
at io.netty.channel.socket.nio.NioServerSocketChannel.doBind(NioServerSocketChannel.java:128) ~[?:?]
at io.netty.channel.AbstractChannel$AbstractUnsafe.bind(AbstractChannel.java:554) ~[?:?]
at io.netty.channel.DefaultChannelPipeline$HeadContext.bind(DefaultChannelPipeline.java:1258) ~[?:?]
at io.netty.channel.AbstractChannelHandlerContext.invokeBind(AbstractChannelHandlerContext.java:501) ~[?:?]
at io.netty.channel.AbstractChannelHandlerContext.bind(AbstractChannelHandlerContext.java:486) ~[?:?]
at io.netty.channel.DefaultChannelPipeline.bind(DefaultChannelPipeline.java:980) ~[?:?]
at io.netty.channel.AbstractChannel.bind(AbstractChannel.java:250) ~[?:?]
at io.netty.bootstrap.AbstractBootstrap$2.run(AbstractBootstrap.java:365) ~[?:?]
at io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:163) ~[?:?]
at io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:403) ~[?:?]
at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:462) ~[?:?]
at io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:858) ~[?:?]
at java.lang.Thread.run(Thread.java:745) [?:1.8.0_102]
[2018-06-15T09:56:57,822][INFO ][o.e.n.Node ] [node1] stopping ...
[2018-06-15T09:56:57,826][INFO ][o.e.n.Node ] [node1] stopped
[2018-06-15T09:56:57,826][INFO ][o.e.n.Node ] [node1] closing ...
[2018-06-15T09:56:57,839][INFO ][o.e.n.Node ] [node1] closed
錯誤原因：
此時的ES配置文件中，network.host : 10.10.10.60 配置的是本機的真實IP。[對於本機的web服務，就是內網IP就可以了，不用外網IP]
但是就是綁定不起，ES啟動不起來。到底為什么？
經過反復對比和排查，發現可能是因為ES版本的問題。
解決方案：
所以，docker pull一個更低版本的ES，重新使用命令重啟docker容器，就使用咱們的ES配置文件中的配置
docker命令：
docker run -itd --name es1 -p 9200:9200 -p 9300:9300 --restart=always -v /mnt/apps/es/elasticsearch.yml:/usr/share/elasticsearch/config/elasticsearch.yml -v /mnt/apps/es/data:/usr/share/elasticsearch/data --net=host elasticsearch:5.5.0
此時的ES配置文件內容依舊是：
elasticAddress1=10.10.10.60
elasticAddress2=10.10.10.60
elasticPort1=9300
elasticPort2=9300
elasticClusterName=elasticsearch-crm
發現，這次ES節點成功啟動了，本機真實IP綁定成功啦。
這下應該web服務這邊調用，就可以通了吧應該，結果一調用還是報錯：
報錯4：
[2019-06-27T07:07:36,721][WARN ][o.e.t.n.Netty4Transport ] [8J1aQeT] exception caught on transport layer [[id: 0x00206b72, L:/10.10.10.60:9300 - R:/10.10.10.60:60672]], closing connection
java.lang.IllegalStateException: Received message from unsupported version: [2.0.0] minimal compatible version is: [5.0.0]
at org.elasticsearch.transport.TcpTransport.messageReceived(TcpTransport.java:1379) ~[elasticsearch-5.5.0.jar:5.5.0]
at org.elasticsearch.transport.netty4.Netty4MessageChannelHandler.channelRead(Netty4MessageChannelHandler.java:74) ~[transport-netty4-5.5.0.jar:5.5.0]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362) [netty-transport-4.1.11.Final.jar:4.1.11.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348) [netty-transport-4.1.11.Final.jar:4.1.11.Final]
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340) [netty-transport-4.1.11.Final.jar:4.1.11.Final]
at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:310) [netty-codec-4.1.11.Final.jar:4.1.11.Final]
at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:297) [netty-codec-4.1.11.Final.jar:4.1.11.Final]
at io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:413) [netty-codec-4.1.11.Final.jar:4.1.11.Final]
at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:265) [netty-codec-4.1.11.Final.jar:4.1.11.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362) [netty-transport-4.1.11.Final.jar:4.1.11.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348) [netty-transport-4.1.11.Final.jar:4.1.11.Final]
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340) [netty-transport-4.1.11.Final.jar:4.1.11.Final]
at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:86) [netty-transport-4.1.11.Final.jar:4.1.11.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362) [netty-transport-4.1.11.Final.jar:4.1.11.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348) [netty-transport-4.1.11.Final.jar:4.1.11.Final]
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340) [netty-transport-4.1.11.Final.jar:4.1.11.Final]
at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1334) [netty-transport-4.1.11.Final.jar:4.1.11.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362) [netty-transport-4.1.11.Final.jar:4.1.11.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348) [netty-transport-4.1.11.Final.jar:4.1.11.Final]
at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:926) [netty-transport-4.1.11.Final.jar:4.1.11.Final]
at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:134) [netty-transport-4.1.11.Final.jar:4.1.11.Final]
at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:644) [netty-transport-4.1.11.Final.jar:4.1.11.Final]
at io.netty.channel.nio.NioEventLoop.processSelectedKeysPlain(NioEventLoop.java:544) [netty-transport-4.1.11.Final.jar:4.1.11.Final]
at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:498) [netty-transport-4.1.11.Final.jar:4.1.11.Final]
錯誤原因：
注意上面報錯中，標紅的部分，提示 你正在用2.X的jar包調用5.X的ES服務！！
解決方法：
因為是原服務的遷移，所以去查看一下原環境ES的版本，我上面更換低版本不就是去查看了原環境的ES版本了么？應該不是版本的問題呀，然后再次確認一下
原環境版本
新環境版本：
這下錯誤原因真正的找到了，原本查看的5.5.0 是lucene_version。而實際原環境的ES版本是2.3.3
好了，現在再docker pull2.3.3版本的ES，docker啟動命令依舊如上面：
docker run -itd --name es1 -p 9200:9200 -p 9300:9300 --restart=always -v /mnt/apps/es/elasticsearch.yml:/usr/share/elasticsearch/config/elasticsearch.yml -v /mnt/apps/es/data:/usr/share/elasticsearch/data --net=host elasticsearch:2.3.3
最終，ES成功啟動，web服務成功調通。
=======================================
最后，竟然是因為ES各個版本不兼容，導致的一系列的問題！！！