Python3學習之路~8.5 SocketServer實現多並發

前面幾節我們寫的socket都只能實現服務端與一個客戶端通信，並不能實現服務端與多客戶端同時通信。接下來我們就來學習一下如何實現服務端同時與多個客戶端通信，即並發。

Socket Server

socketserver就是對socket的一個再封裝，主要功能就是實現並發。

socketserver模塊簡化了編寫網絡服務器的任務。

socketserver一共有以下4種類型：

class socketserver.TCPServer（server_address，RequestHandlerClass，bind_and_activate = True）

它使用Internet TCP協議，該協議在客戶端和服務器之間提供連續的數據流。

class socketserver.UDPServer（server_address，RequestHandlerClass，bind_and_activate = True）

它使用數據報，這些數據報是可能無序到達或在傳輸過程中丟失的離散信息包。參數與TCPServer相同。

class socketserver.UnixStreamServer（server_address，RequestHandlerClass，bind_and_activate = True）
class socketserver.UnixDatagramServer（server_address，RequestHandlerClass，bind_and_activate = True）

這些是不經常使用的 類似於TCP和UDP類 的類，但使用Unix域套接字。它們不適用於非Unix平台，參數與TCPServer相同。

如下繼承圖中有五個類，並且分別顯示了他們的繼承關系，其中四個代表四種類型的同步服務器：

+------------+ | BaseServer | +------------+ | v +-----------+ +------------------+ | TCPServer |------->| UnixStreamServer | +-----------+ +------------------+ | v +-----------+ +--------------------+ | UDPServer |------->| UnixDatagramServer | +-----------+ +--------------------+

注意：它們都繼承了同一個基類：BaseServer。

 

創建一個 socketserver 至少分以下幾步：

1.自己創建一個請求處理類，並且這個類要繼承BaseRequestHandler類，並且還要重寫父親類里的handle()方法，此方法將用來處理傳入的請求，即跟客戶端所有的交互都是在handle()里完成的。

2.實例化一個SocketServer類（4種類型選其1，比如TCPServer），並且傳遞server address和 你上面創建的請求處理類 給這個SocketServer。

3.調用SocketServer對象的handle_request()或者serve_forever()方法來處理一個或多個請求。
server.handle_request() # 只能處理一個請求，因為處理完一個就退出了，一般不用它
server.serve_forever() # 可處理多個請求，因為永遠執行

4.調用server_close()來關閉套接字。

 

實例1：基本的socketserver代碼

import socketserver

class MyTCPHandler(socketserver.BaseRequestHandler):
    """
    The request handler class for our server.

    It is instantiated once per connection to the server, and must
    override the handle() method to implement communication to the
    client.
    """

    def handle(self):
        # self.request is the TCP socket connected to the client
        self.data = self.request.recv(1024).strip()
        print("{} wrote:".format(self.client_address[0]))
        print(self.data)
        # just send back the same data, but upper-cased
        self.request.send(self.data.upper())

if __name__ == "__main__":
    HOST, PORT = "localhost", 9999

    # Create the server, binding to localhost on port 9999
    server = socketserver.TCPServer((HOST, PORT), MyTCPHandler)

    # Activate the server; this will keep running until you
    # interrupt the program with Ctrl-C
    server.serve_forever()

上面是一個最簡單的socketserver代碼，只能處理一個請求，比如運行如下客戶端代碼，通過結果你就會發現，處理第二個請求就會失敗。

#Author:Zheng Na

# 客戶端

import socket


client = socket.socket()
client.connect(('localhost',9999))

while True:
    msg = input(">>: ").strip()
    client.send(msg.encode("UTF-8"))

    data = client.recv(1024) # 接收1024字節
    print("recv from server: ",data)

client.close()

D:\software\Python3.6.5\python.exe D:/python-study/s14/Day08/socketclient.py
>>: aa
recv from server:  b'AA'
>>: bb
Traceback (most recent call last):
  File "D:/python-study/s14/Day08/socketclient.py", line 15, in <module>
    data = client.recv(1024) # 接收1024字節
ConnectionAbortedError: [WinError 10053] 您的主機中的軟件中止了一個已建立的連接。

Process finished with exit code 1

 

實例2：處理多個請求

如果想要它處理多個請求，那么就要自己在handle()方法中加循環，跟前面我們學的socket實例一樣。

#Author:Zheng Na

import socketserver

class MyTCPHandler(socketserver.BaseRequestHandler):

    def handle(self):
        while True:
            self.data = self.request.recv(1024).strip()
            print("{} wrote:".format(self.client_address[0]))
            print(self.data)
            # 下面3行代碼防止服務器端隨着客戶端的斷開而斷開
            # 經過測試發現，在linux有效，Windows無效。
            if not self.data:
                print(self.client_address,"斷開了")
                break
            self.request.send(self.data.upper())

if __name__ == "__main__":
    HOST, PORT = "localhost", 9999

    # Create the server, binding to localhost on port 9999
    server = socketserver.TCPServer((HOST, PORT), MyTCPHandler)

    # Activate the server; this will keep running until you
    # interrupt the program with Ctrl-C
    server.serve_forever()

上段代碼你在linux運行是沒問題的，但是在Windows上面運行的話，一旦你關閉了客戶端，服務端就會報錯ConnectionResetError

D:\software\Python3.6.5\python.exe D:/python-study/s14/Day08/socketclient.py
>>: aa
recv from server:  b'AA'
>>: bb
recv from server:  b'BB'
>>: 
Process finished with exit code -1

D:\software\Python3.6.5\python.exe D:/python-study/s14/Day08/socketserver2.py
127.0.0.1 wrote:
b'aa'
127.0.0.1 wrote:
b'bb'
----------------------------------------
Exception happened during processing of request from ('127.0.0.1', 53532)
Traceback (most recent call last):
  File "D:\software\Python3.6.5\lib\socketserver.py", line 317, in _handle_request_noblock
    self.process_request(request, client_address)
  File "D:\software\Python3.6.5\lib\socketserver.py", line 348, in process_request
    self.finish_request(request, client_address)
  File "D:\software\Python3.6.5\lib\socketserver.py", line 361, in finish_request
    self.RequestHandlerClass(request, client_address, self)
  File "D:\software\Python3.6.5\lib\socketserver.py", line 696, in __init__
    self.handle()
  File "D:/python-study/s14/Day08/socketserver2.py", line 9, in handle
    self.data = self.request.recv(1024).strip()
ConnectionResetError: [WinError 10054] 遠程主機強迫關閉了一個現有的連接。

 

實例3：

如果想要在Windows上運行不報錯，我們可以嘗試主動抓住這個錯誤。

#Author:Zheng Na

import socketserver

class MyTCPHandler(socketserver.BaseRequestHandler):

    def handle(self):
        while True:
            try:
                self.data = self.request.recv(1024).strip()
                print("{} wrote:".format(self.client_address[0]))
                print(self.data)
                # 下面3行代碼防止服務器端隨着客戶端的斷開而斷開
                # 經過測試發現，在linux有效，Windows無效。
                if not self.data:
                    print(self.client_address, "斷開了")
                    break
                self.request.send(self.data.upper())
            except ConnectionResetError as e:
                print('出現錯誤',e)
                break

if __name__ == "__main__":
   
    HOST, PORT = "localhost", 9999
   
    server = socketserver.TCPServer((HOST, PORT), MyTCPHandler)
   
    server.serve_forever()

D:\software\Python3.6.5\python.exe D:/python-study/s14/Day08/socketclient.py
>>: aa
recv from server:  b'AA'
>>: bb
recv from server:  b'BB'
>>: 
Process finished with exit code -1

D:\software\Python3.6.5\python.exe D:/python-study/s14/Day08/socketserver3.py
127.0.0.1 wrote:
b'aa'
127.0.0.1 wrote:
b'bb'
出現錯誤 [WinError 10054] 遠程主機強迫關閉了一個現有的連接。

此時，你就可以實現關閉一個客戶端后，繼續重新打開另一個客戶端向服務器發送數據。

 

實例4：ThreadingTCPServer 實現並發

但你發現，上面的代碼，依然不能同時處理多個連接，哎？那我搞這個干嘛？別急，不是不能處理多並發，如果你想，你還要啟用多線程，多線程我們現在還沒學，但你大體知道，有了多線程，就能同時讓cpu干多件事了。

讓你的socketserver並發起來， 必須選擇使用以下一個多並發的類

class socketserver.ForkingTCPServer # 多進程

class socketserver.ForkingUDPServer

class socketserver.ThreadingTCPServer # 多線程

class socketserver.ThreadingUDPServer

so 只需要把下面這句

server = socketserver.TCPServer((HOST, PORT), MyTCPHandler)

換成下面這個，就可以多並發了，這樣，客戶端每連進一個來，服務器端就會分配一個新的線程來處理這個客戶端的請求

server = socketserver.ThreadingTCPServer((HOST, PORT), MyTCPHandler)

運行代碼

#Author:Zheng Na

import socketserver

class MyTCPHandler(socketserver.BaseRequestHandler):

    def handle(self):
        while True:
            try:
                self.data = self.request.recv(1024).strip()
                print("{} wrote:".format(self.client_address[0]))
                print(self.data)
                # 下面3行代碼防止服務器端隨着客戶端的斷開而斷開
                # 經過測試發現，在linux有效，Windows無效。
                if not self.data:
                    print(self.client_address, "斷開了")
                    break
                self.request.send(self.data.upper())
            except ConnectionResetError as e:
                print('出現錯誤',e)
                break

if __name__ == "__main__":

    HOST, PORT = "localhost", 9999

    server = socketserver.ThreadingTCPServer((HOST, PORT), MyTCPHandler)

    server.serve_forever()

輸出結果

D:\software\Python3.6.5\python.exe D:/python-study/s14/Day08/socketserver4_thread_concurrence.py
127.0.0.1 wrote:
b'from client1'
127.0.0.1 wrote:
b'from client2'
127.0.0.1 wrote:
b'from client3'

D:\software\Python3.6.5\python.exe D:/python-study/s14/Day08/socketclient.py
>>: from client1
recv from server:  b'FROM CLIENT1'
>>: 

D:\software\Python3.6.5\python.exe D:/python-study/s14/Day08/socketclient.py
>>: from client2
recv from server:  b'FROM CLIENT2'
>>: 

D:\software\Python3.6.5\python.exe D:/python-study/s14/Day08/socketclient.py
>>: from client3
recv from server:  b'FROM CLIENT3'
>>: 

[root@hadoop my-test-files]# python3 socketserver4_thread_concurrence.py 
127.0.0.1 wrote:
b'from client1'
127.0.0.1 wrote:
b'from client2'
127.0.0.1 wrote:
b'from client3'

[root@hadoop my-test-files]# python3 socketclient.py 
>>: from client1
recv from server:  b'FROM CLIENT1'
>>: 

[root@hadoop my-test-files]# python3 socketclient.py 
>>: from client2    
recv from server:  b'FROM CLIENT2'
>>: 

[root@hadoop my-test-files]# python3 socketclient.py 
>>: from client3    
recv from server:  b'FROM CLIENT3'
>>: 

 

實例5：ForkingTCPServer 實現並發

同實例4類似，只是這次將下面這句

server = socketserver.TCPServer((HOST, PORT), MyTCPHandler)

換成了下面這個

server = socketserver.ForkingTCPServer((HOST, PORT), MyTCPHandler)

運行代碼

#Author:Zheng Na

import socketserver

class MyTCPHandler(socketserver.BaseRequestHandler):

    def handle(self):
        while True:
            try:
                self.data = self.request.recv(1024).strip()
                print("{} wrote:".format(self.client_address[0]))
                print(self.data)
                # 下面3行代碼防止服務器端隨着客戶端的斷開而斷開
                # 經過測試發現，在linux有效，Windows無效。
                if not self.data:
                    print(self.client_address, "斷開了")
                    break
                self.request.send(self.data.upper())
            except ConnectionResetError as e:
                print('出現錯誤',e)
                break

if __name__ == "__main__":

    HOST, PORT = "localhost", 9999

    server = socketserver.ForkingTCPServer((HOST, PORT), MyTCPHandler)

    server.serve_forever()

輸出結果

# 我的運行結果（Windows Python3.6.5）：一運行就報錯
D:\software\Python3.6.5\python.exe D:/python-study/s14/Day08/socketserver5_fork_concurrence.py
Traceback (most recent call last):
  File "D:/python-study/s14/Day08/socketserver5_fork_concurrence.py", line 27, in <module>
    server = socketserver.ForkingTCPServer((HOST, PORT), MyTCPHandler)
AttributeError: module 'socketserver' has no attribute 'ForkingTCPServer'

Process finished with exit code 1

老師的運行結果：啟動3個客戶端后才報錯
AttributeError: module 'os' has no attribute 'fork'

總之一句話，ForkingTCPServer在Windows上不好使

[root@hadoop my-test-files]# python3 socketserver5_fork_concurrence.py 
127.0.0.1 wrote:
b'from client1'
127.0.0.1 wrote:
b'from client2'
127.0.0.1 wrote:
b'from client3'

[root@hadoop my-test-files]# python3 socketclient.py 
>>: from client1
recv from server:  b'FROM CLIENT1'
>>: 

[root@hadoop my-test-files]# python3 socketclient.py 
>>: from client2    
recv from server:  b'FROM CLIENT2'
>>: 

[root@hadoop my-test-files]# python3 socketclient.py 
>>: from client3    
recv from server:  b'FROM CLIENT3'
>>: 

注意：ForkingTCPServer在Windows上不好使，在linux上百分比好使（效果與ThreadingTCPServer一模一樣）。

 

總結：TCPServer VS ThreadingTCPServer VS ForkingTCPServer

TCPServer是接收到請求后執行handle方法，如果前一個的handle沒有結束，那么其他的請求將不會受理，新的客戶端也無法加入。

而ThreadingTCPServer和ForkingTCPServer則允許前一連接的handle未結束也可受理新的請求和連接新的客戶端，區別在於前者用建立新線程的方法運行handle，后者用新進程的方法運行handle。

 

class socketserver.BaseServer(server_address, RequestHandlerClass) 主要有以下方法

class socketserver.BaseServer(server_address, RequestHandlerClass)
This is the superclass of all Server objects in the module. It defines the interface, given below, but does not implement most of the methods, which is done in subclasses. The two parameters are stored in the respective server_address and RequestHandlerClass attributes.

fileno() # 返回文件描述符，這個一般是系統內部調用時用到的，我們一般用不到，知道即可
Return an integer file descriptor for the socket on which the server is listening. This function is most commonly passed to selectors, to allow monitoring multiple servers in the same process.

handle_request() # 處理單個請求，我們一般也不用
Process a single request. This function calls the following methods in order: get_request(), verify_request(), and process_request(). If the user-provided handle() method of the handler class raises an exception, the server’s handle_error() method will be called. If no request is received within timeout seconds, handle_timeout() will be called and handle_request() will return.
 
serve_forever(poll_interval=0.5) # 一直處理請求，直到收到一個明確的shutdown()請求。每0.5秒檢查一下是否有程序給我發了shutdown的信號。
Handle requests until an explicit shutdown() request. Poll for shutdown every poll_interval seconds. Ignores the timeout attribute. It also calls service_actions(), which may be used by a subclass or mixin to provide actions specific to a given service. For example, the ForkingMixIn class uses service_actions() to clean up zombie child processes.

Changed in version 3.3: Added service_actions call to the serve_forever method.

service_actions() # Python3.3引入，被serve_forever()調用。
This is called in the serve_forever() loop. This method can be overridden by subclasses or mixin classes to perform actions specific to a given service, such as cleanup actions.

New in version 3.3.

shutdown() # 告訴serve_forever()停止處理請求
Tell the serve_forever() loop to stop and wait until it does.

server_close() # 關閉
Clean up the server. May be overridden.

address_family # 地址簇
The family of protocols to which the server’s socket belongs. Common examples are socket.AF_INET and socket.AF_UNIX.

RequestHandlerClass # 請求處理類
The user-provided request handler class; an instance of this class is created for each request.

server_address # 地址
The address on which the server is listening. The format of addresses varies depending on the protocol family; see the documentation for the socket module for details. For Internet protocols, this is a tuple containing a string giving the address, and an integer port number: ('127.0.0.1', 80), for example.

socket # 套接字
The socket object on which the server will listen for incoming requests.

The server classes support the following class variables:

allow_reuse_address # 允許重用地址
Whether the server will allow the reuse of an address. This defaults to False, and can be set in subclasses to change the policy.

request_queue_size # 暫時不用管它
The size of the request queue. If it takes a long time to process a single request, any requests that arrive while the server is busy are placed into a queue, up to request_queue_size requests. Once the queue is full, further requests from clients will get a “Connection denied” error. The default value is usually 5, but this can be overridden by subclasses.

socket_type # 協議類型
The type of socket used by the server; socket.SOCK_STREAM and socket.SOCK_DGRAM are two common values.

timeout # 超時時間，在handle_request()中使用，由於我們不同handle_request()，所以不用管它
Timeout duration, measured in seconds, or None if no timeout is desired. If handle_request() receives no incoming requests within the timeout period, the handle_timeout() method is called.

There are various server methods that can be overridden by subclasses of base server classes like TCPServer; these methods aren’t useful to external users of the server object.

finish_request()
Actually processes the request by instantiating RequestHandlerClass and calling its handle() method.

get_request()
Must accept a request from the socket, and return a 2-tuple containing the new socket object to be used to communicate with the client, and the client’s address.

handle_error(request, client_address)
This function is called if the handle() method of a RequestHandlerClass instance raises an exception. The default action is to print the traceback to standard output and continue handling further requests.

handle_timeout()
This function is called when the timeout attribute has been set to a value other than None and the timeout period has passed with no requests being received. The default action for forking servers is to collect the status of any child processes that have exited, while in threading servers this method does nothing.

process_request(request, client_address)
Calls finish_request() to create an instance of the RequestHandlerClass. If desired, this function can create a new process or thread to handle the request; the ForkingMixIn and ThreadingMixIn classes do this.

server_activate()
Called by the server’s constructor to activate the server. The default behavior for a TCP server just invokes listen() on the server’s socket. May be overridden.

server_bind()
Called by the server’s constructor to bind the socket to the desired address. May be overridden.

verify_request(request, client_address)
Must return a Boolean value; if the value is True, the request will be processed, and if it’s False, the request will be denied. This function can be overridden to implement access controls for a server. The default implementation always returns True.

 

 

 

ThreadingTCPServer