為CK+人臉情緒識別數據集所做的數據庫設計

就我們組所做的項目（人臉情緒識別）來說，本是可以不需要建立任何數據庫的。但是本着追求進一步效率的想法，考慮到用於神經網絡學習的CK+數據集的本身結構較為復雜，擁有比較復雜的文件結構，且體積也較大（接近2GB），不太方便輸入到神經網絡中進行訓練，因此對此做了一個簡單的數據庫，用以將這個數據集的各個關鍵信息記錄，從而在獲取訓練數據時，只要通過對數據庫的SELECT操作即可獲取，而不需要復雜的文件操作以及獲取label的字符串處理等操作。

CK+數據集簡介

In 2000, the Cohn-Kanade (CK) database was released for the purpose of promoting research into automatically detecting individual facial expressions. Since then, the CK database has become one of the most widely used test-beds for algorithm development and evaluation. During this period, three limitations have become apparent: 1) While AU codes are well validated, emotion labels are not, as they refer to what was requested rather than what was actually performed, 2) The lack of a common performance metric against which to evaluate new algorithms, and 3) Standard protocols for common databases have not emerged. As a consequence, the CK database has been used for both AU and emotion detection (even though labels for the latter havenotbeenvalidated),comparisonwithbenchmarkalgorithmsismissing, anduseofrandomsubsetsoftheoriginal database makes meta-analyses difficult. To address these andother concerns, we presenttheExtended Cohn-Kanade (CK+) database. The number of sequences is increased by 22%andthenumberofsubjectsby27%. Thetargetexpression for each sequence is fully FACS coded and emotion labels have been revised and validated. In addition to this, non-posed sequences for several types of smiles and their associated metadata have been added. 

參照論文《The Extended Cohn-Kanade Dataset(CK+):  A complete dataset for action unit and emotion-specified expression》

 

CK+數據集結構

分為四個文件夾：

1) The Images (cohn-kanade-images.zip) - there are 593 sequences across 123 subjects which are FACS coded at the peak frame. All sequences are from the neutral face to the peak expression.

2) The Landmarks (Landmarks.zip) - All sequences are AAM tracked with 68points landmarks for each image.

3) The FACS coded files (FACS_labels.zip) - for each sequence (593) there is only 1 FACS file, which is the last frame (the peak frame). Each line of the file corresponds to a specific AU and then the intensity. An example is given below.

4) The Emotion coded files (Emotion_labels.zip) - ONLY 327 of the 593 sequences have emotion sequences. This is because these are the only ones the fit the prototypic definition. Like the FACS files, there is only 1 Emotion file for each sequence which is the last frame (the peak frame). There should be only one entry and the number will range from 0-7 (i.e. 0=neutral, 1=anger, 2=contempt, 3=disgust, 4=fear, 5=happy, 6=sadness, 7=surprise). N.B there is only 327 files- IF THERE IS NO FILE IT MEANS THAT THERE IS NO EMOTION LABEL (sorry to be explicit but this will avoid confusion).

在目前階段，我們暫時只考慮第一個和第四個文件夾。

對於第一個文件夾，包含123個文件夾，對應123個主體（Subject），即受試者，文件夾命名為S+主體編號。

對每個主體的文件夾，又包含若干個文件夾，每個文件夾對應一個表情序列，文件夾命名為序列編號，從001開始。

對於每個表情序列文件夾，包含若干張圖片，展示主體從平靜狀態到表情峰值的變化過程中的圖片序列，命名為S+主體編號_序列編號_圖片編號，圖片編號從00000001開始。

對於第二個文件夾，與第一個文件夾結構基本一致，但是到最里層的文件目錄中不是圖片，而是一個txt文件，命名為指定目錄下對應的序列的最后一張圖片的名稱+emotion，該txt文件中包含一個值，從0到7不等，其中0=neutral, 1=anger, 2=contempt, 3=disgust, 4=fear, 5=happy, 6=sadness, 7=surprise。

數據庫設計

數據庫分為兩個表Groups和Imgs。前者每個元組對應一個主體所做的一個表情序列的總和，后者每個元組對應一張圖片。由於一個表情序列有多張圖片，所以顯然這是一個一對多的關系。

兩個表的設計如下：

imgs

字段名	類型	注釋	鍵
subject	smallint(6)	主體	主鍵1，外鍵
group	tinyint(4)	組號	主鍵2，外鍵
no	tinyint(4)	圖片編號	主鍵3
addr	varchar(255)	圖片路徑

 

 

 

 

 

 

 

 

groups

字段名	類型	注釋	鍵
subject	smallint(6)	主體	主鍵1
group	tinyint(4)	組號	主鍵2
emotion	tinyint(4)	情緒編號
num	tinyint(4)	組內圖片總數

 

 

 

 

 

 

 

 

數據庫導入

 數據庫建立完了，但是對於這超過一萬張的圖片，全部手動輸入到數據庫中是絕對不可能的，因此我寫了一個Python的小腳本程序來代替我們執行數據庫導入的操作。

import pymysql
import sys
import os

db = pymysql.connect('localhost', 'root', '123456', 'train')
cursor = db.cursor()
addr = 'H:\CK\cohn-kanade-images'
addr_e = 'H:\CK\Emotion'
for folder in os.listdir(addr):
    addr2 = os.path.join(addr, folder)
    for folder2 in os.listdir(addr2):
        if not str(folder2)[0] == '.':
            addr3 = os.path.join(addr2, folder2)
            imgs = os.listdir(addr3)
            num = len(imgs)
            i = num - 1
            while i != -1:
                img = imgs[i]
                img_name = img[: -4]
                if img_name[0] == '.':
                    i -= 1
                    continue
                print(img_name)
                subject, group, no = (j for j in img_name.split('_'))
                addr_p = os.path.join(addr3, img)
                if i == num - 1:
                    addr_ans = os.path.join(addr_e, subject, group, subject
                                            + '_' + group + '_' + no + '_emotion.txt')
                    try:
                        with open(addr_ans, 'r') as f:
                            ans = f.read().split('.')[0]
                            sql = """
                                INSERT INTO `groups` VALUES(%s, %s, %s, %s)
                            """ % (subject[1:], group, ans, num)
                            print(subject[1:], group, ans, num)
                            try:
                                cursor.execute(sql)
                                db.commit()
                            except Exception as e:
                                print(e)
                                db.rollback()
                    except:
                        ans = '0'
                        sql = """
                            INSERT INTO `groups` VALUES(%s, %s, %s, %s)
                        """ % (subject[1:], group, ans, num)
                        print(subject[1:], group, ans, num)
                        try:
                            cursor.execute(sql)
                            db.commit()
                        except Exception as e:
                            print(e)
                            db.rollback()

                sql = """
                        INSERT INTO imgs VALUES(%s, %s, %s, "%s");
                        """ % (subject[1:], group, no, addr_p.replace('\\', '/'))
                print(subject[1:], group, no, addr_p.replace('\\', '/'))
                try:
                    cursor.execute(sql)
                    db.commit()
                except Exception as e:
                    print(e)
                    db.rollback()
                i -= 1

通過Python下的一系列文件操作和對文件名的字符串處理，再通過與MySQL的連接即可將這超過一萬張圖片的數據全部導入到指定數據庫中。

導入結果如下：

 

 

 

導入完成后，我們要獲取一張圖片的地址以及其情緒類型，只需要通過以下語句：

db = pymysql.connect('localhost', 'root', '123456', 'train')
cursor = db.cursor()
sql = """select i.addr, g.emotion 
    from imgs i, `groups` g 
    where i.subject = g.subject and i.group = g.group;"""
cursor.execute(sql)
res = cursor.fetchone()

這樣相比利用復雜的文件操作通過找文件的方式來獲取訓練數據要簡化了相當多。