Python 數據分析：Pandas 缺省值的判斷

背景

我們從數據庫中取出數據存入 Pandas None 轉換成 NaN 或 NaT。但是，我們將 Pandas 數據寫入數據庫時又需要轉換成 None，不然就會報錯。因此，我們就需要處理 Pandas 的缺省值。

樣本數據

   id         name  password  sn  sex  age  amount  content  remark  login_date login_at    created_at  
0   1  123456789.0       NaN NaN  NaN   20     NaN      NaN     NaN  NaN        NaT         2019-08-10 10:00:00  
1   2          NaN       NaN NaN  NaN   20     NaN      NaN     NaN  NaN        NaT         2019-08-10 10:00:00 

判斷缺省值

如果 column 是缺省值，則統一處理為 None。

def judge_null(column):
    if pd.isnull(column):
        return None
    return column

處理缺省值

按列處理缺省值。

df['id'] = df.apply(lambda row: judge_null(row['id']), axis=1)
df['name'] = df.apply(lambda row: judge_null(row['name']), axis=1)
df['password'] = df.apply(lambda row: judge_null(row['password']), axis=1)
df['sn'] = df.apply(lambda row: judge_null(row['sn']), axis=1)
df['sex'] = df.apply(lambda row: judge_null(row['sex']), axis=1)
df['age'] = df.apply(lambda row: judge_null(row['age']), axis=1)
df['amount'] = df.apply(lambda row: judge_null(row['amount']), axis=1)
df['content'] = df.apply(lambda row: judge_null(row['content']), axis=1)
df['remark'] = df.apply(lambda row: judge_null(row['remark']), axis=1)
df['login_date'] = df.apply(lambda row: judge_null(row['login_date']), axis=1)
df['login_at'] = df.apply(lambda row: judge_null(row['login_at']), axis=1)
df['created_at'] = df.apply(lambda row: judge_null(row['created_at']), axis=1)

處理完成之后的數據

   id         name  password  sn    sex    age   amount    content  remark  login_date  login_at  created_at  
0   1  123456789.0      None  None  None   20    None      None     None    None        None      2019-08-10 10:00:00  
1   2         None      None  None  None   20    None      None     None    None        None      2019-08-10 10:00:00 

補充

設置顯示所有的行、列及值得長度。

# 顯示所有列
pd.set_option('display.max_columns', None)
# 顯示所有行
pd.set_option('display.max_rows', None)
# 設置value的顯示長度為100，默認為50
pd.set_option('max_colwidth', 100)

對應的數據庫建表語句

create table test
(
  id         int(10)        not null primary key,
  name       varchar(32)    null,
  password   char(10)       null,
  sn         bigint         null,
  sex        tinyint(1)     null,
  age        int(5)         null,
  amount     decimal(10, 2) null,
  content    text           null,
  remark     json           null,
  login_date date           null,
  login_at   datetime       null,
  created_at timestamp      null
);