先簡單的介紹下關於LVS負載均衡
LVS（Linux Virtual Server）Linux服務器集群系統
在lvs環境中，需要設定以下的參數
| echo "1" > /proc/sys/net/ipv4/conf/all/arp_ignore echo "1" > /proc/sys/net/ipv4/conf/lo/arp_ignore echo "2" > /proc/sys/net/ipv4/conf/lo/arp_announce echo "2" > /proc/sys/net/ipv4/conf/all/arp_announce |
先來看看關於arp_ignore和arp_announce的有關介紹
arp_ignore - INTEGER
Define different modes for sending replies in response to
received ARP requests that resolve local target IP addresses:
0 - (default): reply for any local target IP address, configured
on any interface
1 - reply only if the target IP address is local address
configured on the incoming interface
2 - reply only if the target IP address is local address
configured on the incoming interface and both with the
sender's IP address are part from same subnet on this interface
3 - do not reply for local addresses configured with scope host,
only resolutions for global and link addresses are replied
4-7 - reserved
8 - do not reply for all local addresses
The max value from conf/{all,interface}/arp_ignore is used
when ARP request is received on the {interface} |
arp_ignore:定義對目標地址為本地IP的ARP詢問不同的應答模式0
0 - (默認值): 回應任何網絡接口上對任何本地IP地址的arp查詢請求
1 - 只回答目標IP地址是來訪網絡接口本地地址的ARP查詢請求
2 -只回答目標IP地址是來訪網絡接口本地地址的ARP查詢請求,且來訪IP必須在該網絡接口的子網段內
3 - 不回應該網絡界面的arp請求，而只對設置的唯一和連接地址做出回應
4-7 - 保留未使用
8 -不回應所有（本地地址）的arp查詢
有關arp_announce的相關介紹：
arp_announce - INTEGER
Define different restriction levels for announcing the local
source IP address from IP packets in ARP requests sent on
interface:
0 - (default) Use any local address, configured on any interface
1 - Try to avoid local addresses that are not in the target's
subnet for this interface. This mode is useful when target
hosts reachable via this interface require the source IP
address in ARP requests to be part of their logical network
configured on the receiving interface. When we generate the
request we will check all our subnets that include the
target IP and will preserve the source address if it is from
such subnet. If there is no such subnet we select source
address according to the rules for level 2.
2 - Always use the best local address for this target.
In this mode we ignore the source address in the IP packet
and try to select local address that we prefer for talks with
the target host. Such local address is selected by looking
for primary IP addresses on all our subnets on the outgoing
interface that include the target IP address. If no suitable
local address is found we select the first local address
we have on the outgoing interface or on all other interfaces,
with the hope we will receive reply for our request and
even sometimes no matter the source IP address we announce.
The max value from conf/{all,interface}/arp_announce is used.
Increasing the restriction level gives more chance for
receiving answer from the resolved target while decreasing
the level announces more valid sender's information. |
arp_announce:對網絡接口上，本地IP地址的發出的，ARP回應，作出相應級別的限制: 確定不同程度的限制,宣布對來自本地源IP地址發出Arp請求的接口
0 - (默認) 在任意網絡接口（eth0,eth1，lo）上的任何本地地址
1 -盡量避免不在該網絡接口子網段的本地地址做出arp回應. 當發起ARP請求的源IP地址是被設置應該經由路由達到此網絡接口的時候很有用.此時會檢查來訪IP是否為所有接口上的子網段內ip之一.如果改來訪IP不屬於各個網絡接口上的子網段內,那么將采用級別2的方式來進行處理.
2 - 對查詢目標使用最適當的本地地址.在此模式下將忽略這個IP數據包的源地址並嘗試選擇與能與該地址通信的本地地址.首要是選擇所有的網絡接口的子網中外出訪問子網中包含該目標IP地址的本地地址. 如果沒有合適的地址被發現,將選擇當前的發送網絡接口或其他的有可能接受到該ARP回應的網絡接口來進行發送.
關於對arp_announce 理解的一點補充
| Assume that a linux box X has three interfaces - eth0, eth1 and eth2. Each interface has an IP address IP0, IP1 and IP2. When a local application tries to send an IP packet with IP0 through the eth2. Unfortunately, the target node’s mac address is not resolved. Thelinux box X will send the ARP request to know the mac address of the target(or the gateway). In this case what is the IP source address of the “ARP request message”? The IP0- the IP source address of the transmitting IP or IP2 - the outgoing interface? Until now(actually just 3 hours before) ARP request uses the IP address assigned to the outgoing interface(IP2 in the above example) However the linux’s behavior is a little bit different. Actually the selection of source address in ARP request is totally configurable bythe proc variable “arp_announce” If we want to use the IP2 not the IP0 in the ARP request, we should change the value to 1 or 2. The default value is 0 - allow IP0 is used for ARP request. |
其實就是路由器的問題，因為路由器一般是動態學習ARP包的（一般動態配置DHCP的話），當內網的機器要發送一個到外部的ip包，那么它就會請求 路由器的Mac地址，發送一個arp請求，這個arp請求里面包括了自己的ip地址和Mac地址，而linux默認是使用ip的源ip地址作為arp里面 的源ip地址，而不是使用發送設備上面的 ，這樣在lvs這樣的架構下，所有發送包都是同一個VIP地址，那么arp請求就會包括VIP地址和設備 Mac，而路由器收到這個arp請求就會更新自己的arp緩存，這樣就會造成ip欺騙了，VIP被搶奪，所以就會有問題。
arp緩存為什么會更新了，什么時候會更新呢，為了減少arp請求的次數，當主機接收到詢問自己的arp請求的時候，就會把源ip和源Mac放入自 己的arp表里面，方便接下來的通訊。如果收到不是詢問自己的包（arp是廣播的，所有人都收到），就會丟掉，這樣不會造成arp表里面無用數據太多導致 有用的記錄被刪除。