python 爬蟲 使用requests設置代理

免費代理的網站: http://www.xicidaili.com/nn/

代碼部分：

import requests

proxy='124.243.226.18:8888'

#如果代理需要驗證，只需要在前面加上用戶名密碼，如下所示

# proxy='username:password@124.243.226.18:8888'
proxies={
    'http':'http://'+proxy,
    'https':'https://'+proxy,
}
try:
    response=requests.get('http://httpbin.org/get',proxies=proxies)
    print(response.text)
except requests.exceptions.ConnectionError as e:
    print("Error",e.args)
 

輸出：

{
  "args": {},
  "headers": {
    "Accept": "*/*",
    "Accept-Encoding": "gzip, deflate",
    "Connection": "close",
    "Host": "httpbin.org",
    "User-Agent": "python-requests/2.20.0"
  },
  "origin": "124.243.226.18",
  "url": "http://httpbin.org/get"
}

 

 

基於 selenium的代理設置：

from selenium import webdriver

proxy='124.243.226.18:8888'

option=webdriver.ChromeOptions()

option.add_argument('--proxy-server=http://'+proxy)

driver = webdriver.Chrome(options=option)

driver.get('http://httpbin.org/get')