python數據分析——pandas的拼接操作

pandas的拼接操作

pandas的拼接分為兩種：

• 級聯：pd.concat, pd.append
• 合並：pd.merge, pd.join

1. 使用pd.concat()級聯

pandas使用pd.concat函數，與np.concatenate函數類似，只是多了一些參數：

objs
axis=0
keys
join='outer' / 'inner':表示的是級聯的方式，outer會將所有的項進行級聯（忽略匹配和不匹配），而inner只會將匹配的項級聯到一起，不匹配的不級聯
ignore_index=False

1)匹配級聯

In [1]:

import numpy as np import pandas as pd from pandas import Series,DataFrame 

In [2]:

df1 = DataFrame(data=np.random.randint(0,100,size=(3,3)),index=['a','b','c'],columns=['A','B','C']) df2 = DataFrame(data=np.random.randint(0,100,size=(3,3)),index=['a','d','c'],columns=['A','d','C']) 

In [7]:

pd.concat((df1,df1),axis=0,join='inner') 

Out[7]:

	A	B	C
a	59	40	89
b	71	5	76
c	29	34	87
a	59	40	89
b	71	5	76
c	29	34	87

2) 不匹配級聯

不匹配指的是級聯的維度的索引不一致。例如縱向級聯時列索引不一致，橫向級聯時行索引不一致

有2種連接方式：

• 外連接：補NaN（默認模式）

• 內連接：只連接匹配的項

In [11]:

pd.concat((df1,df2),axis=1,join='outer')

Out[11]:

	A	B	C	A	d	C
a	59.0	40.0	89.0	50.0	26.0	45.0
b	71.0	5.0	76.0	NaN	NaN	NaN
c	29.0	34.0	87.0	31.0	82.0	35.0
d	NaN	NaN	NaN	23.0	95.0	94.0

3) 使用df.append()函數添加

由於在后面級聯的使用非常普遍，因此有一個函數append專門用於在后面添加

2. 使用pd.merge()合並

merge與concat的區別在於，merge需要依據某一共同的列來進行合並

使用pd.merge()合並時，會自動根據兩者相同column名稱的那一列，作為key來進行合並。

注意每一列元素的順序不要求一致

參數：

• how：out取並集 inner取交集

• on：當有多列相同的時候，可以使用on來指定使用那一列進行合並，on的值為一個列表

1) 一對一合並

In [12]:

df1 = DataFrame({'employee':['Bob','Jake','Lisa'], 'group':['Accounting','Engineering','Engineering'], }) df1 

Out[12]:

	employee	group
0	Bob	Accounting
1	Jake	Engineering
2	Lisa	Engineering

In [13]:

df2 = DataFrame({'employee':['Lisa','Bob','Jake'], 'hire_date':[2004,2008,2012], }) df2 

Out[13]:

	employee	hire_date
0	Lisa	2004
1	Bob	2008
2	Jake	2012

In [14]:

pd.merge(df1,df2,how='outer') 

Out[14]:

	employee	group	hire_date
0	Bob	Accounting	2008
1	Jake	Engineering	2012
2	Lisa	Engineering	2004

2) 多對一合並

In [15]:

df3 = DataFrame({ 'employee':['Lisa','Jake'], 'group':['Accounting','Engineering'], 'hire_date':[2004,2016]}) df3 

Out[15]:

	employee	group	hire_date
0	Lisa	Accounting	2004
1	Jake	Engineering	2016

In [16]:

df4 = DataFrame({'group':['Accounting','Engineering','Engineering'], 'supervisor':['Carly','Guido','Steve'] }) df4 

Out[16]:

	group	supervisor
0	Accounting	Carly
1	Engineering	Guido
2	Engineering	Steve

In [17]:

pd.merge(df3,df4) 

Out[17]:

	employee	group	hire_date	supervisor
0	Lisa	Accounting	2004	Carly
1	Jake	Engineering	2016	Guido
2	Jake	Engineering	2016	Steve

3) 多對多合並

In [18]:

df1 = DataFrame({'employee':['Bob','Jake','Lisa'], 'group':['Accounting','Engineering','Engineering']}) df1 

Out[18]:

	employee	group
0	Bob	Accounting
1	Jake	Engineering
2	Lisa	Engineering

In [19]:

df5 = DataFrame({'group':['Engineering','Engineering','HR'], 'supervisor':['Carly','Guido','Steve'] }) df5 

Out[19]:

	group	supervisor
0	Engineering	Carly
1	Engineering	Guido
2	HR	Steve

In [21]:

pd.merge(df1,df5,how='outer') 

Out[21]:

	employee	group	supervisor
0	Bob	Accounting	NaN
1	Jake	Engineering	Carly
2	Jake	Engineering	Guido
3	Lisa	Engineering	Carly
4	Lisa	Engineering	Guido
5	NaN	HR	Steve

• 加載excl數據:pd.read_excel('excl_path',sheetname=1)

4) key的規范化

• 當列沖突時，即有多個列名稱相同時，需要使用on=來指定哪一個列作為key，配合suffixes指定沖突列名

In [10]:

df1 = DataFrame({'employee':['Jack',"Summer","Steve"], 'group':['Accounting','Finance','Marketing']}) 

In [11]:

df2 = DataFrame({'employee':['Jack','Bob',"Jake"], 'hire_date':[2003,2009,2012], 'group':['Accounting','sell','ceo']}) 

In [22]:

display(df1,df2) 

 

	employee	group
0	Bob	Accounting
1	Jake	Engineering
2	Lisa	Engineering

 

	employee	hire_date
0	Lisa	2004
1	Bob	2008
2	Jake	2012

• 當兩張表沒有可進行連接的列時，可使用left_on和right_on手動指定merge中左右兩邊的哪一列列作為連接的列

In [12]:

df1 = DataFrame({'employee':['Bobs','Linda','Bill'], 'group':['Accounting','Product','Marketing'], 'hire_date':[1998,2017,2018]}) 

In [13]:

df5 = DataFrame({'name':['Lisa','Bobs','Bill'], 'hire_dates':[1998,2016,2007]}) 

In [23]:

display(df1,df5) 

 

	employee	group
0	Bob	Accounting
1	Jake	Engineering
2	Lisa	Engineering

 

	group	supervisor
0	Engineering	Carly
1	Engineering	Guido
2	HR	Steve

5) 內合並與外合並:out取並集 inner取交集

• 內合並：只保留兩者都有的key（默認模式）

In [25]:

df6 = DataFrame({'name':['Peter','Paul','Mary'], 'food':['fish','beans','bread']} ) df7 = DataFrame({'name':['Mary','Joseph'], 'drink':['wine','beer']}) 

In [26]:

display(df6,df7) 

 

	name	food
0	Peter	fish
1	Paul	beans
2	Mary	bread

 

	name	drink
0	Mary	wine
1	Joseph	beer

• 外合並 how='outer'：補NaN

In [27]:

df6 = DataFrame({'name':['Peter','Paul','Mary'], 'food':['fish','beans','bread']} ) df7 = DataFrame({'name':['Mary','Joseph'], 'drink':['wine','beer']}) display(df6,df7) pd.merge() 

 

	name	food
0	Peter	fish
1	Paul	beans
2	Mary	bread

 

	name	drink
0	Mary	wine
1	Joseph	beer