Camera Calibration 相機標定：Opencv應用方法

本文轉載自查看原文 2015-10-26 17:32 2145

本系列文章由 @YhL_Leo 出品，轉載請注明出處。
文章鏈接： http://blog.csdn.net/yhl_leo/article/details/49427383

Opencv中Camera Calibration and 3D Reconstruction中使用的是Z. Zhang(PAMI, 2000). A Flexible New Technique for Camera Calibration的方法。原理見原理簡介（五）本文將對其進行介紹。

1 標定步驟

簡單來說，Opencv中基於二維標定平面的標定方法主要步驟有：

1 讀取相關設置信息，包括采用的pattern 信息（類型，尺寸），輸入標定數據的信息（圖像列表文件，視頻采樣方法），輸出文件設置等，這些信息可以存為XML或YAML文件的形式或者在代碼里直接顯示設置。這里給出Opencv中提供的configuration file：

<?xml version="1.0"?>
<opencv_storage>
<Settings>
<!-- Number of inner corners per a item row and column. (square, circle) -->
<BoardSize_Width>9</BoardSize_Width>
<BoardSize_Height>6</BoardSize_Height>
<!-- The size of a square in some user defined metric system (pixel, millimeter) -->
<Square_Size>50</Square_Size>
<!-- The type of input used for camera calibration. One of: CHESSBOARD CIRCLES_GRID ASYMMETRIC_CIRCLES_GRID -->
<Calibrate_Pattern>"CHESSBOARD"</Calibrate_Pattern>
<!-- The input to use for calibration. To use an input camera -> give the ID of the camera, like "1" To use an input video -> give the path of the input video, like "/tmp/x.avi" To use an image list -> give the path to the XML or YAML file containing the list of the images, like "/tmp/circles_list.xml" -->
<Input>"images/CameraCalibraation/VID5/VID5.xml"</Input>
<!-- If true (non-zero) we flip the input images around the horizontal axis. -->
<Input_FlipAroundHorizontalAxis>0</Input_FlipAroundHorizontalAxis>
<!-- Time delay between frames in case of camera. -->
<Input_Delay>100</Input_Delay>
<!-- How many frames to use, for calibration. -->
<Calibrate_NrOfFrameToUse>25</Calibrate_NrOfFrameToUse>
<!-- Consider only fy as a free parameter, the ratio fx/fy stays the same as in the input cameraMatrix. Use or not setting. 0 - False Non-Zero - True -->
<Calibrate_FixAspectRatio>1</Calibrate_FixAspectRatio>
<!-- If true (non-zero) tangential distortion coefficients are set to zeros and stay zero. -->
<Calibrate_AssumeZeroTangentialDistortion>1</Calibrate_AssumeZeroTangentialDistortion>
<!-- If true (non-zero) the principal point is not changed during the global optimization. -->
<Calibrate_FixPrincipalPointAtTheCenter>1</Calibrate_FixPrincipalPointAtTheCenter>
<!-- The name of the output log file. -->
<Write_outputFileName>"out_camera_data.xml"</Write_outputFileName>
<!-- If true (non-zero) we write to the output file the feature points. -->
<Write_DetectedFeaturePoints>1</Write_DetectedFeaturePoints>
<!-- If true (non-zero) we write to the output file the extrinsic camera parameters. -->
<Write_extrinsicParameters>1</Write_extrinsicParameters>
<!-- If true (non-zero) we show after calibration the undistorted images. -->
<Show_UndistortedImage>1</Show_UndistortedImage>
</Settings>
</opencv_storage>

其中，圖像文件列表images/CameraCalibraation/VID5/VID5.xmlOpencv中采用列舉法：

<?xml version="1.0"?>
<opencv_storage>
<images>
images/CameraCalibraation/VID5/xx1.jpg 
images/CameraCalibraation/VID5/xx2.jpg 
images/CameraCalibraation/VID5/xx3.jpg 
images/CameraCalibraation/VID5/xx4.jpg 
images/CameraCalibraation/VID5/xx5.jpg 
images/CameraCalibraation/VID5/xx6.jpg 
images/CameraCalibraation/VID5/xx7.jpg 
images/CameraCalibraation/VID5/xx8.jpg
</images>
</opencv_storage>

文件中參數的含義比較清晰明了，此處就不累述。

2 依次從圖像中檢測pattern信息，如果檢測成功，角點信息將會存儲記錄，用於標定解算。

cv::Mat viewGray;
if ( view.channels() == 3 )
    cv::cvtColor( view, viewGray, CV_BGR2GRAY );
else
    view.copyTo( viewGray );

std::vector<cv::Point2f> imagePoints;   
bool success = cv::findChessboardCorners( viewGray , boardSize, imagePoints);

3 優化角點檢測精度，將上述檢測成功的角點，通過精確角點定位方法，提高精度，下圖為Opencv提供的檢測結果。

cv::cornerSubPix( viewGray, 
              imagePoints, 
              cv::Size(11,11),
              cv::TermCriteria( CV_TERMCRIT_EPS+CV_TERMCRIT_ITER, 30, 0.1 ));

Opencv

4 標定解算，每幅圖像都進行上述的角點檢測后，一般給像點對應的物方角點虛擬坐標的方式賦予對應的坐標，即可進行相機標定解算，包括相機內參，相機畸變系數，以及相機在虛擬坐標所在坐標系中相對於每幅圖像的相對位置姿態（旋轉向量和平移向量）。

double reprojectionError= cv::calibrateCamera(
    objectPoints,   // calibration pattern points in the calibration pattern coordinate space
    imagePoints,    // projections of calibration pattern points
    imageSize,      // Size of the image used only to initialize the intrinsic camera matrix
    cameraMatrix,   // camera matrix A
    distCoeffs,     // distortion coefficients (k1,k2,p1,p2[,k3[,k4,k5,k6]])
    rvecs,          // rotation vectors
    tvecs,          // translation vectors
    flag,           // different calibration model
    criteria);      // Termination criteria for iterative optimization algorithm

5 標定精度評估，為了評價標定后的結果，可以按照標定得到的相機成像模型，由像點反算出物方空間坐標，進而得到一系列點雲，通過對比解算點雲與虛擬點雲之間的差異性，就可以知道獲得模型的好壞（嚴格來講，如果誤差較小，兩者基本應該是一致的）。
6 圖像畸變校正，在opencv示例中，作為標定的最后一個步驟，但是個人認為，這個應該可以作為一個相機標定后的副產品，對於處理的圖像產品精度要求較高時，可以先進行畸變校正，再投入生產。下圖為Opencv提供的畸變校正結果。

Opencv2

2 代碼及結果

下面是個人的代碼程序，有些部分並沒完全按照Opencv的做法：

/* Calibrate camera with chess board pattern. - Editor: Menghan Xia, Yahui Liu. - Data: 2015-07-28 - Email: yahui.cvrs@gmail.com - Address: Computer Vision and Remote Sensing(CVRS) Lab, Wuhan University. **/

#include<iostream>
#include <vector>
#include <string>

#include "cv.h"
#include "highgui.h"

#include "toolFunction.h"
#define DEBUG_OUTPUT_INFO

using namespace std;
using namespace cv;

void main()
{   
    char* folderPath = "E:/Images/New";           // image folder
    std::vector<std::string> graphPaths;
    std::vector<std::string> graphSuccess;

    CalibrationAssist calAssist;

    graphPaths = calAssist.get_filelist(folderPath); // collect image list

#ifdef DEBUG_OUTPUT_INFO
    std::cout << "loaded " << graphPaths.size() << " images"<< std::endl;
#endif

    if ( !graphPaths.empty() )
    {
#ifdef DEBUG_OUTPUT_INFO
        std::cout << "Start corner detection ..." << std::endl;
#endif

        cv::Mat curGraph;  // current image
        cv::Mat gray;      // gray image of current image

        int imageCount = graphPaths.size();
        int imageCountSuccess = 0;
        cv::Size image_size; 
        cv::Size boardSize  = cv::Size(19, 19);     // chess board pattern size
        cv::Size squareSize = cv::Size(15, 15);     // grid physical size, as a scale factor

        std::vector<cv::Point2f> corners;                  // one image corner list
        std::vector<std::vector<cv::Point2f> > seqCorners; // n images corner list

        if ( graphPaths.size() < 3 )
        {
#ifdef DEBUG_OUTPUT_INFO
            std::cout << "Calibrate failed, with less than three images!" << std::endl;
#endif
            return ;
        }

        for ( int i=0; i<graphPaths.size(); i++ )
        {   
            string graphpath = folderPath;
            graphpath += "/" + graphPaths[i];
            curGraph = cv::imread(graphpath);

            if ( curGraph.channels() == 3 )
                cv::cvtColor( curGraph, gray, CV_BGR2GRAY );
            else
                curGraph.copyTo( gray );

            // for every image, empty the corner list
            std::vector<cv::Point2f>().swap( corners );  

            // corners detection
            bool success = cv::findChessboardCorners( curGraph, boardSize, corners ); 

            if ( success ) // succeed
            {
#ifdef DEBUG_OUTPUT_INFO
                std::cout << i << " " << graphPaths[i] << " succeed"<< std::endl;
#endif
                int row = curGraph.rows;
                int col = curGraph.cols;

                graphSuccess.push_back( graphpath );
                imageCountSuccess ++;

                image_size = cv::Size( col, row );

                // find sub-pixels
                cv::cornerSubPix( 
                    gray, 
                    corners, 
                    cv::Size( 11, 11 ), 
                    cv::Size( -1, -1 ),
                    cv::TermCriteria( CV_TERMCRIT_EPS + CV_TERMCRIT_ITER, 30, 0.1 ) );
                seqCorners.push_back( corners );

#if 1
                // draw corners and show them in current image
                cv::Mat imageDrawCorners;
                if ( curGraph.channels() == 3 )
                    curGraph.copyTo( imageDrawCorners );
                else
                    cv::cvtColor( curGraph, imageDrawCorners, CV_GRAY2RGB );

                for ( int j = 0; j < corners.size(); j ++)
                {
                    cv::Point2f dotPoint = corners[j];
                    cv::circle( imageDrawCorners, dotPoint, 3.0, cv::Scalar( 0, 255, 0 ), -1 );
                    cv::Point2f pt_m = dotPoint + cv::Point2f(4,4);
                    char text[100];
                    sprintf( text, "%d", j+1 );  // corner indexes which start from 1
                    cv::putText( imageDrawCorners, text, pt_m, 1, 0.5, cv::Scalar( 255, 0, 255 ) );
                }

                std::string pathTemp;
                pathTemp = folderPath;
                pathTemp += "/#" + graphPaths[i];

                // save image drawn with corners and labeled with indexes
                cv::imwrite( pathTemp, imageDrawCorners ); 
#endif
            }
#ifdef DEBUG_OUTPUT_INFO
            else // failed
            {
                std::cout << graphPaths[i] << " corner detect failed!" << std::endl;
            }
#endif

        }
#ifdef DEBUG_OUTPUT_INFO
        std::cout << "Corner detect done!" << std::endl 
            << imageCountSuccess << " succeed! " << std::endl;
#endif


        if ( imageCountSuccess < 3 )
        {
#ifdef DEBUG_OUTPUT_INFO
            std::cout << "Calibrated success " << imageCountSuccess 
                << " images, less than 3 images." << std::endl;
#endif
            return ;
        }
        else
        {
#ifdef DEBUG_OUTPUT_INFO
            std::cout << "Start calibration ..." << std::endl;
#endif
            cv::Point3f point3D;
            std::vector<cv::Point3f> objectPoints;
            std::vector<double> distCoeffs;
            std::vector<double> rotation;
            std::vector<double> translation;

            std::vector<std::vector<cv::Point3f>> seqObjectPoints;
            std::vector<std::vector<double>> seqRotation;
            std::vector<std::vector<double>> seqTranslation;
            cv::Mat_<double> cameraMatrix;

            // calibration pattern points in the calibration pattern coordinate space
            for ( int t=0; t<imageCountSuccess; t++ )
            {
                objectPoints.clear();
                for ( int i=0; i<boardSize.height; i++ )
                {
                    for ( int j=0; j<boardSize.width; j++ )
                    {
                        point3D.x = i * squareSize.width;
                        point3D.y = j * squareSize.height;
                        point3D.z = 0;
                        objectPoints.push_back(point3D);
                    }
                }
                seqObjectPoints.push_back(objectPoints);
            }

            double reprojectionError = calibrateCamera(
                seqObjectPoints, 
                seqCorners, 
                image_size, 
                cameraMatrix, 
                distCoeffs, 
                seqRotation, 
                seqTranslation,
                CV_CALIB_FIX_ASPECT_RATIO|CV_CALIB_FIX_PRINCIPAL_POINT );

#ifdef DEBUG_OUTPUT_INFO
            std::cout << "Calibration done!" << std::endl;
#endif
            // calculate the calibration pattern points with the camera model
            std::vector<cv::Mat_<double>> projectMats;

            for ( int i=0; i<imageCountSuccess; i++ )
            {
                cv::Mat_<double> R, T;
                // translate rotation vector to rotation matrix via Rodrigues transformation
                cv::Rodrigues( seqRotation[i], R ); 
                T = cv::Mat( cv::Matx31d( 
                    seqTranslation[i][0], 
                    seqTranslation[i][1],
                    seqTranslation[i][2]) );

                cv::Mat_<double> P = cameraMatrix * cv::Mat( cv::Matx34d( 
                    R(0,0), R(0,1), R(0,2), T(0),  
                    R(1,0), R(1,1), R(1,2), T(1),  
                    R(2,0), R(2,1), R(2,2), T(2) ) ); 

                projectMats.push_back(P);
            }

            std::vector<cv::Point2d> PointSet;
            int pointNum = boardSize.width*boardSize.height;
            std::vector<cv::Point3d> objectClouds;
            for ( int i=0; i<pointNum; i++ )
            {
                PointSet.clear();
                for ( int j=0; j<imageCountSuccess; j++ )
                {
                    cv::Point2d tempPoint = seqCorners[j][i];
                    PointSet.push_back(tempPoint);
                }
                // calculate calibration pattern points
                cv::Point3d objectPoint = calAssist.triangulate(projectMats,PointSet);
                objectClouds.push_back(objectPoint);
            }
            std::string pathTemp_point;
            pathTemp_point = folderPath;
            pathTemp_point += "/point.txt";
            calAssist.save3dPoint(pathTemp_point,objectClouds);

            std::string pathTemp_calib;
            pathTemp_calib = folderPath;
            pathTemp_calib += "/calibration.txt";

            FILE* fp = fopen( pathTemp_calib.c_str(), "w" );
            fprintf( fp, "The average of re-projection error : %lf\n", reprojectionError );
            for ( int i=0; i<imageCountSuccess; i++ )
            {
                std::vector<cv::Point2f> errorList;
                cv::projectPoints( 
                    seqObjectPoints[i], 
                    seqRotation[i], 
                    seqTranslation[i], 
                    cameraMatrix, 
                    distCoeffs, 
                    errorList );

                corners.clear();
                corners = seqCorners[i];

                double meanError(0.0);
                for ( int j=0; j<corners.size(); j++ )
                {   
                    meanError += std::sqrt((errorList[j].x - corners[j].x)*(errorList[j].x - corners[j].x) + 
                        (errorList[j].y - corners[j].y)*(errorList[j].y - corners[j].y));
                }
                rotation.clear();
                translation.clear();

                rotation = seqRotation[i];
                translation = seqTranslation[i];
                fprintf( fp, "Re-projection of image %d：%lf\n", i+1, meanError/corners.size() );
                fprintf( fp, "Rotation vector :\n" );
                fprintf( fp, "%lf %lf %lf\n", rotation[0], rotation[1], rotation[2] );
                fprintf( fp, "Translation vector :\n" );
                fprintf( fp, "%lf %lf %lf\n\n", translation[0], translation[1], translation[2] );
            }
            fprintf( fp, "Camera internal matrix :\n" );
            fprintf( fp, "%lf %lf %lf\n%lf %lf %lf\n%lf %lf %lf\n", 
                cameraMatrix(0,0), cameraMatrix(0,1), cameraMatrix(0,2),
                cameraMatrix(1,0), cameraMatrix(1,1), cameraMatrix(1,2),
                cameraMatrix(2,0), cameraMatrix(2,1), cameraMatrix(2,2));
            fprintf( fp,"Distortion coefficient :\n" );
            for ( int k=0; k<distCoeffs.size(); k++)
                fprintf( fp, "%lf ", distCoeffs[k] );
#ifdef DEBUG_OUTPUT_INFO
            std::cout << "Results are saved!" << std::endl;
#endif 
        }
    }
}

// toolFunction.h
#ifndef TOOL_FUNCTION_H
#pragma once
#define TOOL_FUNCTION_H

#include<iostream>
#include <Windows.h>
#include <math.h>
#include <fstream>
#include <vector>
#include <string>

#include "cv.h"
#include "highgui.h"

using namespace cv;
using namespace std;

class CalibrationAssist
{
public:
    CalibrationAssist() {}
    ~CalibrationAssist() {}

public:
    std::vector<std::string>get_filelist( std::string foldname );

    cv::Point3d triangulate( std::vector<cv::Mat_<double>> &ProjectMats, 
        std::vector<cv::Point2d> &imagePoints );

    void save3dPoint( std::string path_, std::vector<cv::Point3d> &Point3dLists );
};
#endif // TOOL_FUNCTION_H

// toolFunction.cpp
#include "toolFunction.h"

std::vector<std::string> CalibrationAssist::get_filelist( std::string foldname )
{
    foldname += "/*.*";
    const char * mystr=foldname.c_str();
    std::vector<std::string> flist;
    std::string lineStr;
    std::vector<std::string> extendName;
    extendName.push_back("jpg");
    extendName.push_back("JPG");
    extendName.push_back("bmp");
    extendName.push_back("png");
    extendName.push_back("gif");

    HANDLE file;
    WIN32_FIND_DATA fileData;
    char line[1024];
    wchar_t fn[1000];
    mbstowcs( fn, mystr, 999 );
    file = FindFirstFile( fn, &fileData );
    FindNextFile( file, &fileData );
    while(FindNextFile( file, &fileData ))
    {
        wcstombs( line, (const wchar_t*)fileData.cFileName, 259);
        lineStr = line;
        // remove the files which are not images
        for (int i = 0; i < 4; i ++)
        {
            if (lineStr.find(extendName[i]) < 999)
            {
                flist.push_back(lineStr);
                break;
            }
        }   
    }
    return flist;
}


cv::Point3d CalibrationAssist::triangulate(
    std::vector<cv::Mat_<double>> &ProjectMats, 
    std::vector<cv::Point2d> &imagePoints)
{
    int i,j;
    std::vector<cv::Point2d> pointSet;
    int frameSum = ProjectMats.size();
    cv::Mat A(2*frameSum,3,CV_32FC1);
    cv::Mat B(2*frameSum,1,CV_32FC1);
    cv::Point2d u,u1;
    cv::Mat_<double> P;
    cv::Mat_<double> rowA1,rowA2,rowB1,rowB2;
    int k = 0;
    for ( i = 0; i < frameSum; i++ )     //get the coefficient matrix A and B
    {
        u = imagePoints[i];
        P = ProjectMats[i];
        cv::Mat( cv::Matx13d( 
            u.x*P(2,0)-P(0,0),
            u.x*P(2,1)-P(0,1),
            u.x*P(2,2)-P(0,2) ) ).copyTo( A.row(k) );

        cv::Mat( cv::Matx13d( 
            u.y*P(2,0)-P(1,0),
            u.y*P(2,1)-P(1,1),
            u.y*P(2,2)-P(1,2) ) ).copyTo( A.row(k+1) );

        cv::Mat rowB1( 1, 1, CV_32FC1, cv::Scalar( -(u.x*P(2,3)-P(0,3)) ) );
        cv::Mat rowB2( 1, 1, CV_32FC1, cv::Scalar(-(u.y*P(2,3)-P(1,3)) ) );
        rowB1.copyTo( B.row(k) );
        rowB2.copyTo( B.row(k+1) );
        k += 2;
    }
    cv::Mat X;  
    cv::solve( A, B, X, DECOMP_SVD );  
    return Point3d(X); 
}

void CalibrationAssist::save3dPoint( std::string path_, std::vector<cv::Point3d> &Point3dLists)
{
    const char * path = path_.c_str();
    FILE* fp = fopen( path, "w" );
    for ( int i = 0; i < Point3dLists.size(); i ++)
    {
        // fprintf(fp,"%d ",i);
        fprintf( fp, "%lf %lf %lf\n", 
            Point3dLists[i].x, Point3dLists[i].y, Point3dLists[i].z);
    }
    fclose(fp);
#if 1
    std::cout << "clouds of points are saved!" << std::endl;
#endif
}

使用數據為 9 張 1200×800 的圖像：

Opencv4

程序運行結果：

1 運行控制台輸出結果
2 角點檢測圖

Opencv5

3 反投影點雲（CloudCompare顯示）

Opencv6

對於上述結果的生成文件，此處用了C語言寫成txt的方式，讀者完全可以考慮使用XML或YAML格式文件保存，至於畸變糾正的問題，也很簡單，直接利用標定得到的相機內參和畸變系數，查詢remap函數的使用方法即可。此外，處理較大圖像時，Opencv提供的方法速度可能會較慢，遇到這種情況，可以考慮把圖像縮小或重寫角點檢測算法。

免責聲明！

本站轉載的文章為個人學習借鑒使用，本站對版權不負任何法律責任。如果侵犯了您的隱私權益，請聯系本站郵箱yoyou2525@163.com刪除。

猜您在找 OpenCV 相機參數標定（Camera Calibration）相機標定(Camera calibration) 相機標定（Camera calibration）原理、步驟攝像機標定(Camera calibration)筆記 OpenCV 相機標定 opencv相機標定(Python) Opencv——相機標定 OpenCV相機標定 MATLAB二維相機標定的解決方案 calibration Halcon使用的3D相機模型-camera_calibration