問題描述
aiohttp 的 getting started 入門案例是這樣寫的
import aiohttp
import asyncio
async def main():
async with aiohttp.ClientSession() as session:
async with session.get('https://python.org') as response:
print("Status:", response.status)
print("Content-type:", response.headers['content-type'])
html = await response.text()
print("Body:", html[:15], "...")
loop = asyncio.get_event_loop()
loop.run_until_complete(main())
運行結果為
Status: 200
Content-type: text/html; charset=utf-8
Body: <!doctype html> ...
看上去沒問題，但是在 Python3.7 后對 asyncio 進行了改進，可以直接調用 asyncio.run() 執行協程程序，而不需管底層 API 如事件循環 loop 的操作，所以上述代碼的
loop = asyncio.get_event_loop()
loop.run_until_complete(main())
可以直接替換為
asyncio.run()
Linux 和 Mac 上這樣運行是沒問題的，但是在 Windows 上運行會報如下錯誤
Exception ignored in: <function _ProactorBasePipeTransport.__del__ at 0x000001B1FFE978B0>
Traceback (most recent call last):
File "D:\DevTools\Python\lib\asyncio\proactor_events.py", line 116, in __del__
self.close()
File "D:\DevTools\Python\lib\asyncio\proactor_events.py", line 108, in close
self._loop.call_soon(self._call_connection_lost, None)
File "D:\DevTools\Python\lib\asyncio\base_events.py", line 746, in call_soon
self._check_closed()
File "D:\DevTools\Python\lib\asyncio\base_events.py", line 510, in _check_closed
raise RuntimeError('Event loop is closed')
RuntimeError: Event loop is closed
原因分析
像 aiohttp 這類第三方協程庫都是依賴於標准庫 asyncio 的，而 asyncio 對 Windows 的支持本來就不好。Python3.8 后默認 Windows 系統上的事件循環采用 ProactorEventLoop （僅用於 Windows ）這篇文檔描述了其在 Windows 下的缺陷：https://docs.python.org/zh-cn/3/library/asyncio-platforms.html#windows 👈
引發異常的函數是 _ProactorBasePipeTransport.__del__ ，所以 aiohttp 鐵定使用了 _ProactorBasePipeTransport，並且在程序退出釋放內存時自動調用了其__del__ 方法
就是上述一串連環反應最終拋出了 RuntimeError: Event loop is closed
一般的協程程序是不會使用 _ProactorBasePipeTransport 的，所以下面的代碼還是可以正常的運行
import asyncio
async def main():
print('Hello...')
await asyncio.sleep(1)
print('...World')
asyncio.run(main())
我特意寫了個裝飾器來驗證這一點：
import asyncio
from asyncio.proactor_events import _ProactorBasePipeTransport
from functools import wraps
def explicit_call(func):
@wraps(func)
def wrappers(self, *args, **kwargs):
print('call _ProactorBasePipeTransport.__del__')
return func(self, *args, **kwargs)
return wrappers
_ProactorBasePipeTransport.__del__ = explicit_call(_ProactorBasePipeTransport.__del__)
async def main():
print('Hello...')
await asyncio.sleep(1)
print('...World')
asyncio.run(main())
正常執行，沒有使用 _ProactorBasePipeTransport
Hello...
...World
import asyncio
from asyncio.proactor_events import _ProactorBasePipeTransport
from functools import wraps
import aiohttp
async def main():
async with aiohttp.ClientSession() as session:
async with session.get('https://www.baidu.com') as response:
print("Status:", response.status)
print("Content-type:", response.headers['content-type'])
html = await response.text()
print("Body:", html[:15], "...")
def explicit_call(func):
@wraps(func)
def wrappers(self, *args, **kwargs):
print('call _ProactorBasePipeTransport.__del__')
return func(self, *args, **kwargs)
return wrappers
_ProactorBasePipeTransport.__del__ = explicit_call(_ProactorBasePipeTransport.__del__)
asyncio.run(main())
先打印 call _ProactorBasePipeTransport.__del__ 然后報錯，說明使用了 _ProactorBasePipeTransport
Status: 200
Content-type: text/html
Body: <html>
...
call _ProactorBasePipeTransport.__del__
Exception ignored in: <function _ProactorBasePipeTransport.__del__ at 0x000001B11653EF70>
Traceback (most recent call last):
File "D:\Codes\SpiderProject\BingImageSpider\demo.py", line 16, in wrappers
return func(self, *args, **kwargs)
File "D:\DevTools\Python\lib\asyncio\proactor_events.py", line 116, in __del__
self.close()
File "D:\DevTools\Python\lib\asyncio\proactor_events.py", line 108, in close
self._loop.call_soon(self._call_connection_lost, None)
File "D:\DevTools\Python\lib\asyncio\base_events.py", line 746, in call_soon
self._check_closed()
File "D:\DevTools\Python\lib\asyncio\base_events.py", line 510, in _check_closed
raise RuntimeError('Event loop is closed')
RuntimeError: Event loop is closed
進程已結束，退出代碼為 0
解決方案
如果執意要在 Windows 下繼續開發，有這幾個方案可以選擇
1. 不要使用 run 函數
既然 _ProactorBasePipeTransport 會在程序結束后自動關閉事件循環，那就不要用 run 函數了，用官網的例子，乖乖使用 loop 吧
loop = asyncio.get_event_loop()
loop.run_until_complete(main())
2. 替換事件循環
在調用 run 函數前，替換默認的 ProactorEventLoop 為 SelectorEventLoop
asyncio.set_event_loop_policy(asyncio.WindowsSelectorEventLoopPolicy())
asyncio.run(main())
但是 SelectorEventLoop 是有一些缺點的，比如不支持子進程等
3. 忽略異常
這是 Github 上一個外國大佬的方法，在不改變源碼的前提下，使用裝飾器忽略掉異常
import asyncio
from asyncio.proactor_events import _ProactorBasePipeTransport
from functools import wraps
import aiohttp
async def main():
async with aiohttp.ClientSession() as session:
async with session.get('https://www.baidu.com') as response:
print("Status:", response.status)
print("Content-type:", response.headers['content-type'])
html = await response.text()
print("Body:", html[:15], "...")
def silence_event_loop_closed(func):
@wraps(func)
def wrapper(self, *args, **kwargs):
try:
return func(self, *args, **kwargs)
except RuntimeError as e:
if str(e) != 'Event loop is closed':
raise
return wrapper
_ProactorBasePipeTransport.__del__ = silence_event_loop_closed(_ProactorBasePipeTransport.__del__)
asyncio.run(main())
更詳細的信息可以在這個 issue 上找到：https://github.com/aio-libs/aiohttp/issues/4324 👈
相關鏈接：
喜歡我的文章的話，歡迎關注👇點贊👇評論👇收藏👇 謝謝支持！！！