異常現象：
上海到深圳機房，走專線網絡(10MB)，出現深圳行情數據接收延遲缺失
在行情的接收端抓包得到的信息：
分析：TCP segment of a reassembled PDU說明服務端發送的是一個大數據幀，並且經過了分割，以每個1448字節大小的tcp段發送給客戶端，當客戶端收到服務端SMPP消息后，會把屬於同一個PDU的包重組。
[Reassembly error, protocol TCP: New fragment overlaps old data (retransmission?)] 說明有部分的tcp段出現了重傳。
統計-捕獲文件屬性：平均400KB/s
stevens:60s內Sequence number連續增加
根據以上的信息無法確定問題原因，經過baidu看到有一種說法是建議關閉服務器網卡的LSO，LSO是利用網卡分割大數據包，釋放CPU資源的一種機制，但是這種機制很多時候會嚴重的降低網絡性能
以下內容來自網絡：
示例1：
Large Send Offload
TCP 分段通常是由協議棧完成。啟用 Large Send Offload 屬性時，TCP 分段可由網絡適配器完成。
Disable： 禁用 Large Send Offload。
Enable （默認值）： 啟用 Large Send Offload。
Large Send Offload是網絡適配器的高級功能之一，其目的是在網絡適配器端進行TCP的分段工作，以此來降低CPU以及其他相關設備的壓力；但隨着多核CPU的廣泛應用，網絡適配器的處理能力相較於CPU弱了很多，因此當大量並發請求導致數據頻繁更新或大數據量傳送時，開啟Large Send Offload將嚴重影響性能；
示例2：
Large Send Offload and Network Performance
One issue that I continually see reported by customers is slow network performance. Although there are literally a ton of issues that can effect how fast data moves to and from a server, there is one fix I've found that will resolve this 99% of time — disable Large Send Offload on the Ethernet adapter.
So what is Large Send Offload (also known as Large Segmetation Offload, and LSO for short)? It's a feature on modern Ethernet adapters that allows the TCP\IP network stack to build a large TCP message of up to 64KB in length before sending to the Ethernet adapter. Then the hardware on the Ethernet adapter — what I'll call the LSO engine — segments it into smaller data packets (known as "frames" in Ethernet terminology) that can be sent over the wire. This is up to 1500 bytes for standard Ethernet frames and up to 9000 bytes for jumbo Ethernet frames. In return, this frees up the server CPU from having to handle segmenting large TCP messages into smaller packets that will fit inside the supported frame size. Which means better overall server performance. Sounds like a good deal. What could possibly go wrong?
Quite a lot, as it turns out. In order for this to work, the other network devices — the Ethernet switches through which all traffic flows — all have to agree on the frame size. The server cannot send frames that are larger than the Maximum Transmission Unit (MTU) supported by the switches. And this is where everything can, and often does, fall apart.
The server can discover the MTU by asking the switch for the frame size, but there is no way for the server to pass this along to the Ethernet adapter. The LSO engine doesn't have ability to use a dynamic frame size. It simply uses the default standard value of 1500 bytes,or if jumbo frames are enabled, the size of the jumbo frame configured for the adapter. (Because the maximum size of a jumbo frame can vary between different switches, most adapters allow you to set or select a value.) So what happens if the LSO engine sends a frame larger than the switch supports? The switch silently drops the frame. And this is where a performance enhancement feature becomes a performance degredation nightmare.
To understand why this hits network performance so hard, let's follow a typical large TCP message as it traverses the network between two hosts.
- With LSO enabled, the TCP/IP network stack on the server builds a large TCP message.
- The server sends the large TCP message to the Ethernet adapter to be segmented by its LSO engine for the network. Because the LSO engine cannot discover the MTU supported by the switch, it uses a standard default value.
- The LSO engine sends each of the frame segments that make up the large TCP message to the switch.
- The switch receives the frame segments, but because LSO sent frames larger than the MTU, they are silently discarded.
- On the server that is waiting to receive the TCP message, the timeout clock reaches zero when no data is received and it sends back a request to retransmit the data. Although the timeout is very short in human terms, it rather long in computer terms.
- The sending server receives the retransmission request and rebuilds the TCP message. But because this is a retransmission request, the server does not send the TCP message to the Ethernet adapter to be segmented. Instead, it handles the segmentation process itself. This appears to be designed to overcome failures caused by the offloading hardware on the adapter.
- The switch receives the retransmission frames from the server, which are the proper size because the server is able to discover the MTU, and forwards them on to the router.
- The other server finally receives the TCP message intact.
This can basicly be summed up as offload data, segment data, discard data, wait for timeout, request retransmission, segment retransmission data, resend data. The big delay is waiting for the timeout clock on the receiving server to reach zero. And the whole process is repeated the very next time a large TCP message is sent. So is it any wonder that this can cause severe network performance issues.
This is by no means an issue that effects only Peer 1. Google is littered with artices by major vendors of both hardware and software telling their customers to turn off Large Send Offload. Nor is it specific to one operating system. It effects both Linux and Windows.
I've found that Intel adapters are by far the worst offenders with Large Send Offload, but Broadcom also has problems with this as well. And, naturally, this is a feature that is enabled by default on the adapters, meaning that you have to explicitly turn it off in the Ethernet driver (preferred) or server's TCP/IP network stack.