Android打開相機進行人臉識別，使用虹軟人臉識別引擎

上一張效果圖，渣畫質，能看就好

 

功能說明：

人臉識別使用的是虹軟的FreeSDK，包含人臉追蹤，人臉檢測，人臉識別，年齡、性別檢測功能，其中本demo只使用了FT和FR（人臉追蹤和人臉識別），封裝了開啟相機和人臉追蹤、識別功能在FaceCameraHelper中。

實現邏輯：

打開相機，監聽預覽數據回調進行人臉追蹤，且為每個檢測到的人臉都分配一個trackID（上下幀位置變化不大的人臉框可認為是同一個人臉，具體實現的邏輯可見代碼），同時，為了人臉搜索，為每個trackID都分配一個狀態（識別中，識別失敗，識別通過）、姓名，識別通過則在人臉框上顯示姓名，否則只顯示trackID（本demo沒配服務端，只做了模擬操作）。流程說明見下圖。

 

FaceCameraHelper包含的接口：

 

public interface FaceTrackListener {

/**
* 回傳相機預覽數據和人臉框位置
*
* @param nv21 相機預覽數據
* @param ftFaceList 待處理的人臉列表
* @param trackIdList 人臉追蹤ID列表
*/
void onPreviewData(byte[] nv21, List<AFT_FSDKFace> ftFaceList, List<Integer> trackIdList);


/**
* 當出現異常時執行
*
* @param e 異常信息
*/
void onFail(Exception e);


/**
* 當相機打開時執行
*
* @param camera 相機實例
*/
void onCameraOpened(Camera camera);

/**
* 根據自己的需要可以刪除部分人臉，比如指定區域、留下最大人臉等
*
* @param ftFaceList 人臉列表
* @param trackIdList 人臉追蹤ID列表
*/
void adjustFaceRectList(List<AFT_FSDKFace> ftFaceList, List<Integer> trackIdList);

/**
* 請求人臉特征后的回調
*
* @param frFace 人臉特征數據
* @param requestId 請求碼
*/
void onFaceFeatureInfoGet(@Nullable AFR_FSDKFace frFace, Integer requestId);
}

```
FT人臉框繪制並回調數據：

　　

 

 

@Override
public void onPreviewFrame(byte[] nv21, Camera camera) {
if (faceTrackListener != null) {
ftFaceList.clear();
int ftCode = ftEngine.AFT_FSDK_FaceFeatureDetect(nv21, previewSize.width, previewSize.height, AFT_FSDKEngine.CP_PAF_NV21, ftFaceList).getCode();
if (ftCode != 0) {
faceTrackListener.onFail(new Exception("ft failed,code is " + ftCode));
}
refreshTrackId(ftFaceList);
faceTrackListener.adjustFaceRectList(ftFaceList, currentTrackIdList);
if (surfaceViewRect != null) {
Canvas canvas = surfaceViewRect.getHolder().lockCanvas();
if (canvas == null) {
faceTrackListener.onFail(new Exception("can not get canvas of surfaceViewRect"));
return;
}
canvas.drawColor(0, PorterDuff.Mode.CLEAR);
if (ftFaceList.size() > 0) {
for (int i = 0; i < ftFaceList.size(); i++) {
Rect adjustedRect = TrackUtil.adjustRect(new Rect(ftFaceList.get(i).getRect()), previewSize.width, previewSize.height, surfaceWidth, surfaceHeight, cameraOrientation, mCameraId);
TrackUtil.drawFaceRect(canvas, adjustedRect, faceRectColor, faceRectThickness, currentTrackIdList.get(i), nameMap.get(currentTrackIdList.get(i)));
}
}
surfaceViewRect.getHolder().unlockCanvasAndPost(canvas);
}

faceTrackListener.onPreviewData(nv21, ftFaceList, currentTrackIdList);
}
}

　　

大多數設備相機預覽數據圖像的朝向在橫屏時為0度。其他情況按逆時針依次增加90度，因此人臉框的繪制需要做同步轉化。CameraID為0時，也就是后置攝像頭情況，相機預覽數據的顯示為原畫面，而CameraID為1時，也就是前置攝像頭情況，相機的預覽畫面顯示為鏡像畫面，適配的代碼：

/**
     * @param rect          FT人臉框
     * @param previewWidth  相機預覽的寬度
     * @param previewHeight 相機預覽高度
     * @param canvasWidth   畫布的寬度
     * @param canvasHeight  畫布的高度
     * @param cameraOri     相機預覽方向
     * @param mCameraId     相機ID
     * @return
     */
    static Rect adjustRect(Rect rect, int previewWidth, int previewHeight, int canvasWidth, int canvasHeight, int cameraOri, int mCameraId) {
        if (rect == null) {
            return null;
        }
        if (canvasWidth < canvasHeight) {
            int t = previewHeight;
            previewHeight = previewWidth;
            previewWidth = t;
        }
 
        float horizontalRatio;
        float verticalRatio;
        if (cameraOri == 0 || cameraOri == 180) {
            horizontalRatio = (float) canvasWidth / (float) previewWidth;
            verticalRatio = (float) canvasHeight / (float) previewHeight;
        } else {
            horizontalRatio = (float) canvasHeight / (float) previewHeight;
            verticalRatio = (float) canvasWidth / (float) previewWidth;
        }
        rect.left *= horizontalRatio;
        rect.right *= horizontalRatio;
        rect.top *= verticalRatio;
        rect.bottom *= verticalRatio;
 
        Rect newRect = new Rect();
 
        switch (cameraOri) {
            case 0:
                if (mCameraId == Camera.CameraInfo.CAMERA_FACING_FRONT) {
                    newRect.left = canvasWidth - rect.right;
                    newRect.right = canvasWidth - rect.left;
                } else {
                    newRect.left = rect.left;
                    newRect.right = rect.right;
                }
                newRect.top = rect.top;
                newRect.bottom = rect.bottom;
                break;
            case 90:
                newRect.right = canvasWidth - rect.top;
                newRect.left = canvasWidth - rect.bottom;
                if (mCameraId == Camera.CameraInfo.CAMERA_FACING_FRONT) {
                    newRect.top = canvasHeight - rect.right;
                    newRect.bottom = canvasHeight - rect.left;
                } else {
                    newRect.top = rect.left;
                    newRect.bottom = rect.right;
                }
                break;
            case 180:
                newRect.top = canvasHeight - rect.bottom;
                newRect.bottom = canvasHeight - rect.top;
                if (mCameraId == Camera.CameraInfo.CAMERA_FACING_FRONT) {
                    newRect.left = rect.left;
                    newRect.right = rect.right;
                } else {
                    newRect.left = canvasWidth - rect.right;
                    newRect.right = canvasWidth - rect.left;
                }
                break;
            case 270:
                newRect.left = rect.top;
                newRect.right = rect.bottom;
                if (mCameraId == Camera.CameraInfo.CAMERA_FACING_FRONT) {
                    newRect.top = rect.left;
                    newRect.bottom = rect.right;
                } else {
                    newRect.top = canvasHeight - rect.right;
                    newRect.bottom = canvasHeight - rect.left;
                }
                break;
            default:
                break;
        }
        return newRect;
    }

　　

由於FR引擎不支持多線程調用，因此只能串行執行，若需要更高效的實現，可創建多個FREngine實例進行任務分配。

FR線程隊列：

private LinkedBlockingQueue<FaceRecognizeRunnable> faceRecognizeRunnables = new LinkedBlockingQueue<FaceRecognizeRunnable>(MAX_FRTHREAD_COUNT);

　　

FR線程：

public class FaceRecognizeRunnable implements Runnable {
private Rect faceRect;
private int width;
private int height;
private int format;
private int ori;
private Integer requestId;
private byte[]nv21Data;
public FaceRecognizeRunnable(byte[]nv21Data,Rect faceRect, int width, int height, int format, int ori, Integer requestId) {
if (nv21Data==null) {
return;
}
this.nv21Data = new byte[nv21Data.length];
System.arraycopy(nv21Data,0,this.nv21Data,0,nv21Data.length);
this.faceRect = new Rect(faceRect);
this.width = width;
this.height = height;
this.format = format;
this.ori = ori;
this.requestId = requestId;
}

@Override
public void run() {
if (faceTrackListener!=null && nv21Data!=null) {
if (frEngine != null) {
AFR_FSDKFace frFace = new AFR_FSDKFace();
int frCode = frEngine.AFR_FSDK_ExtractFRFeature(nv21Data, width, height, format, faceRect, ori, frFace).getCode();
if (frCode == 0) {
faceTrackListener.onFaceFeatureInfoGet(frFace, requestId);
} else {
faceTrackListener.onFaceFeatureInfoGet(null, requestId);
faceTrackListener.onFail(new Exception("fr failed errorCode is " + frCode));
}
nv21Data = null;
}else {
faceTrackListener.onFaceFeatureInfoGet(null, requestId);
faceTrackListener.onFail(new Exception("fr failed ,frEngine is null" ));
}
if (faceRecognizeRunnables.size()>0){
executor.execute(faceRecognizeRunnables.poll());
}
}
}
}

　　

上下幀是否為相同人臉的判斷（trackID刷新）：

 

/**
* 刷新trackId
*
* @param ftFaceList 傳入的人臉列表
*/
public void refreshTrackId(List<AFT_FSDKFace> ftFaceList) {
currentTrackIdList.clear();
//每項預先填充-1
for (int i = 0; i < ftFaceList.size(); i++) {
currentTrackIdList.add(-1);
}
//前一次無人臉現在有人臉，填充新增TrackId
if (formerTrackIdList.size() == 0) {
for (int i = 0; i < ftFaceList.size(); i++) {
currentTrackIdList.set(i, ++currentTrackId);
}
} else {
//前后都有人臉,對於每一個人臉框
for (int i = 0; i < ftFaceList.size(); i++) {
//遍歷上一次人臉框
for (int j = 0; j < formerFaceRectList.size(); j++) {
//若是同一張人臉
if (TrackUtil.isSameFace(SIMILARITY_RECT, formerFaceRectList.get(j), ftFaceList.get(i).getRect())) {
//記錄ID
currentTrackIdList.set(i, formerTrackIdList.get(j));
break;
}
}
}
}
//上一次人臉框不存在此人臉
for (int i = 0; i < currentTrackIdList.size(); i++) {
if (currentTrackIdList.get(i) == -1) {
currentTrackIdList.set(i, ++currentTrackId);
}
}
formerTrackIdList.clear();
formerFaceRectList.clear();
for (int i = 0; i < ftFaceList.size(); i++) {
formerFaceRectList.add(new Rect(ftFaceList.get(i).getRect()));
formerTrackIdList.add(currentTrackIdList.get(i));
}
}

　　

項目地址：https://github.com/wangshengyang1996/FaceTrackDemo

若有不當的地方望指出。