股票交易數據采集+數據可視化（一個朴素無華的Python爬蟲+可視化案例，附代全部代碼）

前言

我國股票投資者數量為15975.24萬戶, 如此多的股民熱衷於炒股,首先拋開炒股技術不說, 那么多股票數據是不是非常難找,找到之后是不是看着密密麻麻的數據是不是頭都大了?

今天帶大家爬取雪球平台的股票數據

對於本篇文章有疑問的同學可以加【資料白嫖、解答交流群：910981974】

開發環境

• 解釋器版本: python 3.8
• 代碼編輯器: pycharm 2021.2

第三方模塊

• requests: pip install requests
• csv

爬蟲案例的步驟:

1.確定url地址(鏈接地址)
2.發送網絡請求
3.數據解析(篩選數據)
4.數據的保存(數據庫(mysql\mongodb\redis), 本地文件)

爬蟲程序全部代碼

分析網頁

打開開發者工具，搜索關鍵字，找到正確url

導入模塊

import requests     # 發送網絡請求
import csv

 

請求數據

url = f'https://xueqiu.com/service/v5/stock/screener/quote/list?page=1&size=30&order=desc&order_by=amount&exchange=CN&market=CN&type=sha&_=1637908787379'
# 偽裝
headers = {
    # 瀏覽器偽裝
    'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/96.0.4664.45 Safari/537.36'
}
response = requests.get(url, headers=headers)
json_data = response.json()

 

解析數據

data_list = json_data['data']['list']
for data in data_list:
    data1 = data['symbol']
    data2 = data['name']
    data3 = data['current']
    data4 = data['chg']
    data5 = data['percent']
    data6 = data['current_year_percent']
    data7 = data['volume']
    data8 = data['amount']
    data9 = data['turnover_rate']
    data10 = data['pe_ttm']
    data11 = data['dividend_yield']
    data12 = data['market_capital']
    print(data1, data2, data3, data4, data5, data6, data7, data8, data9, data10, data11, data12)
    data_dict = {
        '股票代碼': data1,
        '股票名稱': data2,
        '當前價': data3,
        '漲跌額': data4,
        '漲跌幅': data5,
        '年初至今': data6,
        '成交量': data7,
        '成交額': data8,
        '換手率': data9,
        '市盈率(TTM)': data10,
        '股息率': data11,
        '市值': data12,
    }
    csv_write.writerow(data_dict)

 

翻頁

對比1、2、3頁數據url，找到規律

for page in range(1, 56): url = f'https://xueqiu.com/service/v5/stock/screener/quote/list?page={page}&size=30&order=desc&order_by=amount&exchange=CN&market=CN&type=sha&_=1637908787379' 

保存數據

file = open('data2.csv', mode='a', encoding='utf-8', newline='')
csv_write = csv.DictWriter(file, fieldnames=['股票代碼','股票名稱','當前價','漲跌額','漲跌幅','年初至今','成交量','成交額','換手率','市盈率(TTM)','股息率','市值'])
csv_write.writeheader()
file.close()

 

運行代碼，實現效果

數據可視化全部代碼

導入數據

import pandas as pd
from pyecharts import options as opts
from pyecharts.charts import Bar

 

讀取數據

data_df = pd.read_csv('data2.csv')
df = data_df.dropna()
df1 = df[['股票名稱', '成交量']]
df2 = df1.iloc[:20]
print(df2['股票名稱'].values)
print(df2['成交量'].values)

 

可視化圖表

c = (
    Bar()
        .add_xaxis(list(df2['股票名稱']))
        .add_yaxis("股票成交量情況", list(df2['成交量']))
        .set_global_opts(
        title_opts=opts.TitleOpts(title="成交量圖表 - Volume chart"),
        datazoom_opts=opts.DataZoomOpts(),
    )
        .render("data.html")
)

print('數據可視化結果完成,請在當前目錄下查找打開 data.html 文件!')

 

運行代碼，效果展示