相機的校正感悟

 在你看到這篇博文的時候，希望你能先看下我的上一篇博文，對相關的知識點有所了解后再看這篇博文

  下面的幾個函數就是實現相機校正的關鍵

• addChessboardPoints() 函數 - 用於讀入一系列的棋盤圖像並檢測角點；
• calibrate() 函數 - 用於進行相機校正，得到相機的參數矩陣和畸變系數；
• remap() 函數 - 用於根據相機校正結果修復圖像的畸變；//主要靠這個函數實現校正initUndistortRectifyMap
• addPoints() 函數 - addChessboardPoints() 在檢測完角點后會調用這個函數。也可自己手動調用這個函數添加已知的角點位置和對應的空間坐標點。

main.c 
1 #include <QCoreApplication>
#include <iostream>
#include <iomanip>
#include <vector>
#include <opencv2/core/core.hpp>
#include <opencv2/imgproc/imgproc.hpp>
#include <opencv2/highgui/highgui.hpp>
#include <opencv2/features2d/features2d.hpp>

#include "CameraCalibrator.h"

int main()
{

    cv::namedWindow("Image");
    cv::Mat image;
    std::vector<std::string> filelist;

    // generate list of chessboard image filename
    for (int i=1; i<=20; i++) {

        std::stringstream str;
        str << "/home/bi/pic/chessboards/chessboard" << std::setw(2) << std::setfill('0') << i << ".jpg";
        std::cout << str.str() << std::endl;

        filelist.push_back(str.str());
        image= cv::imread(str.str(),0);//直接變為灰度圖
        //cvtColor(image,image,CV_BGR2GRAY);
        cv::imshow("Image",image);

         cv::waitKey(1000);
    }

    // Create calibrator object
    CameraCalibrator cameraCalibrator;
    // add the corners from the chessboard
    cv::Size boardSize(6,4);
    cameraCalibrator.addChessboardPoints(
        filelist,    // filenames of chessboard image
        boardSize);    // size of chessboard
        // calibrate the camera
    //    cameraCalibrator.setCalibrationFlag(true,true);
    cameraCalibrator.calibrate(image.size());

    // Image Undistortion
    image = cv::imread(filelist[6]);
    cv::Mat uImage= cameraCalibrator.remap(image);

    // display camera matrix
    cv::Mat cameraMatrix= cameraCalibrator.getCameraMatrix();
    std::cout << " Camera intrinsic: " << cameraMatrix.rows << "x" << cameraMatrix.cols << std::endl;
    std::cout << cameraMatrix.at<double>(0,0) << " " << cameraMatrix.at<double>(0,1) << " " << cameraMatrix.at<double>(0,2) << std::endl;
    std::cout << cameraMatrix.at<double>(1,0) << " " << cameraMatrix.at<double>(1,1) << " " << cameraMatrix.at<double>(1,2) << std::endl;
    std::cout << cameraMatrix.at<double>(2,0) << " " << cameraMatrix.at<double>(2,1) << " " << cameraMatrix.at<double>(2,2) << std::endl;

    imshow("Original Image", image);
    imshow("Undistorted Image", uImage);

    cv::waitKey();
    return 0;
}

CameraCalibrator.h 

 

• addChessboardPoints() 函數 - 用於讀入一系列的棋盤圖像並檢測角點；
• calibrate() 函數 - 用於進行相機校正，得到相機的參數矩陣和畸變系數；
• remap() 函數 - 用於根據相機校正結果修復圖像的畸變；//主要靠這個函數實現校正initUndistortRectifyMap
• addPoints() 函數 - addChessboardPoints() 在檢測完角點后會調用這個函數。也可自己手動調用這個函數添加已知的角點位置和對應的空間坐標點。

 

#ifndef CAMERACALIBRATOR_H
#define CAMERACALIBRATOR_H

#include <vector>
#include <iostream>

#include <opencv2/core/core.hpp>
#include "opencv2/imgproc/imgproc.hpp"
#include "opencv2/calib3d/calib3d.hpp"
#include <opencv2/highgui/highgui.hpp>

class CameraCalibrator {

    // input points
    std::vector<std::vector<cv::Point3f>> objectPoints;
    std::vector<std::vector<cv::Point2f>> imagePoints;
    // output Matrices
    cv::Mat cameraMatrix;
    cv::Mat distCoeffs;
    // flag to specify how calibration is done
    int flag;
    // used in image undistortion 
    cv::Mat map1,map2; 
    bool mustInitUndistort;

  public:
    CameraCalibrator() : flag(0), mustInitUndistort(true) {};

    // Open the chessboard images and extract corner points
    int addChessboardPoints(const std::vector<std::string>& filelist, cv::Size & boardSize);
    // Add scene points and corresponding image points
    void addPoints(const std::vector<cv::Point2f>& imageCorners, const std::vector<cv::Point3f>& objectCorners);
    // Calibrate the camera
    double calibrate(cv::Size &imageSize);
    // Set the calibration flag
    void setCalibrationFlag(bool radial8CoeffEnabled=false, bool tangentialParamEnabled=false);
    // Remove distortion in an image (after calibration)
    cv::Mat CameraCalibrator::remap(const cv::Mat &image);

    // Getters
    cv::Mat getCameraMatrix() { return cameraMatrix; }
    cv::Mat getDistCoeffs()   { return distCoeffs; }
};

#endif // CAMERACALIBRATOR_H

 CameraCalibrator.cpp

#include "CameraCalibrator.h"

// Open chessboard images and extract corner points
int CameraCalibrator::addChessboardPoints(
         const std::vector<std::string>& filelist, 
         cv::Size & boardSize) {

    // the points on the chessboard
    std::vector<cv::Point2f> imageCorners;
    std::vector<cv::Point3f> objectCorners;

    // 3D Scene Points:
    // Initialize the chessboard corners 
    // in the chessboard reference frame
    // The corners are at 3D location (X,Y,Z)= (i,j,0)
    for (int i=0; i<boardSize.height; i++) {
        for (int j=0; j<boardSize.width; j++) {

            objectCorners.push_back(cv::Point3f(i, j, 0.0f));
        }
    }

    // 2D Image points:
    cv::Mat image; // to contain chessboard image
    int successes = 0;
    // for all viewpoints
    for (int i=0; i<filelist.size(); i++) {

        // Open the image
        image = cv::imread(filelist[i],0);

        // Get the chessboard corners
        bool found = cv::findChessboardCorners(
                        image, boardSize, imageCorners);

        // Get subpixel accuracy on the corners
        cv::cornerSubPix(image, imageCorners, 
                  cv::Size(5,5), 
                  cv::Size(-1,-1), 
            cv::TermCriteria(cv::TermCriteria::MAX_ITER +
                          cv::TermCriteria::EPS, 
             30,        // max number of iterations 
             0.1));     // min accuracy

          // If we have a good board, add it to our data
          if (imageCorners.size() == boardSize.area()) {

            // Add image and scene points from one view
            addPoints(imageCorners, objectCorners);
            successes++;
          }

        //Draw the corners
        cv::drawChessboardCorners(image, boardSize, imageCorners, found);
        cv::imshow("Corners on Chessboard", image);
        cv::waitKey(100);
    }

    return successes;
}

// Add scene points and corresponding image points
void CameraCalibrator::addPoints(const std::vector<cv::Point2f>& imageCorners, const std::vector<cv::Point3f>& objectCorners) {

    // 2D image points from one view
    imagePoints.push_back(imageCorners);          
    // corresponding 3D scene points
    objectPoints.push_back(objectCorners);
}

// Calibrate the camera
// returns the re-projection error
double CameraCalibrator::calibrate(cv::Size &imageSize)
{
    // undistorter must be reinitialized
    mustInitUndistort= true;

    //Output rotations and translations
    std::vector<cv::Mat> rvecs, tvecs;

    // start calibration
    return 
     calibrateCamera(objectPoints, // the 3D points
                    imagePoints,  // the image points
                    imageSize,    // image size
                    cameraMatrix, // output camera matrix
                    distCoeffs,   // output distortion matrix
                    rvecs, tvecs, // Rs, Ts 
                    flag);        // set options
//                    ,CV_CALIB_USE_INTRINSIC_GUESS);

}

// remove distortion in an image (after calibration)
cv::Mat CameraCalibrator::remap(const cv::Mat &image) {

    cv::Mat undistorted;

    if (mustInitUndistort) { // called once per calibration
    
        cv::initUndistortRectifyMap(
            cameraMatrix,  // computed camera matrix
            distCoeffs,    // computed distortion matrix
            cv::Mat(),     // optional rectification (none) 
            cv::Mat(),     // camera matrix to generate undistorted
            cv::Size(640,480),
//            image.size(),  // size of undistorted
            CV_32FC1,      // type of output map
            map1, map2);   // the x and y mapping functions

        mustInitUndistort= false;
    }

    // Apply mapping functions
    cv::remap(image, undistorted, map1, map2, 
        cv::INTER_LINEAR); // interpolation type

    return undistorted;
}


// Set the calibration options
// 8radialCoeffEnabled should be true if 8 radial coefficients are required (5 is default)
// tangentialParamEnabled should be true if tangeantial distortion is present
void CameraCalibrator::setCalibrationFlag(bool radial8CoeffEnabled, bool tangentialParamEnabled) {

    // Set the flag used in cv::calibrateCamera()
    flag = 0;
    if (!tangentialParamEnabled) flag += CV_CALIB_ZERO_TANGENT_DIST;
    if (radial8CoeffEnabled) flag += CV_CALIB_RATIONAL_MODEL;
}

 

 

findChessboardCorners

Finds the positions of internal corners of the chessboard.
C++: bool findChessboardCorners(InputArray image, Size patternSize, OutputArray corners, int
flags=CV_CALIB_CB_ADAPTIVE_THRESH+CV_CALIB_CB_NORMALIZE_IMAGE
)//如果找到了棋盤內部的角點，會返回true這個時候正好對應繪制角點的patternWasFound

Parameters
image – Source chessboard view. It must be an 8-bit grayscale or color image.
patternSize – Number of inner corners per a chessboard row and column ( patternSize
= cvSize(points_per_row,points_per_colum) = cvSize(columns,rows) ).
corners – Output array of detected corners.
flags – Various operation flags that can be zero or a combination of the following values:
– CV_CALIB_CB_ADAPTIVE_THRESH Use adaptive thresholding to convert the image
to black and white, rather than a fixed threshold level (computed from the average
image brightness).
– CV_CALIB_CB_NORMALIZE_IMAGE Normalize the image gamma with
equalizeHist() before applying fixed or adaptive thresholding.
– CV_CALIB_CB_FILTER_QUADS Use additional criteria (like contour area, perimeter,
square-like shape) to filter out false quads extracted at the contour retrieval stage.
– CALIB_CB_FAST_CHECK Run a fast check on the image that looks for chessboard
corners, and shortcut the call if none is found. This can drastically speed up the call in
the degenerate condition when no chessboard is observed.

drawChessboardCorners
Renders the detected chessboard corners.
C++: void drawChessboardCorners(InputOutputArray image, Size patternSize, InputArray corners, bool
patternWasFound)
Python: cv2.drawChessboardCorners(image, patternSize, corners, patternWasFound)! None
C: void cvDrawChessboardCorners(CvArr* image, CvSize patternSize, CvPoint2D32f* corners, int
count, int patternWasFound)
Python: cv.DrawChessboardCorners(image, patternSize, corners, patternWasFound)! None
Parameters
image – Destination image. It must be an 8-bit color image.
patternSize – Number of inner corners per a chessboard row and column (patternSize =
cv::Size(points_per_row,points_per_column)).//注意這里表示的是棋盤的大小，別弄錯了
corners – Array of detected corners, the output of findChessboardCorners.
patternWasFound – Parameter indicating whether the complete board was found or not.
The return value of findChessboardCorners() should be passed here.
The function draws individual chessboard corners detected either as red circles if the board was not found, or as
colored corners connected with lines if the board was found.

initUndistortRectifyMap
Computes the undistortion and rectification transformation map.
C++: void initUndistortRectifyMap(InputArray cameraMatrix, InputArray distCoeffs, InputArray R,
InputArray newCameraMatrix, Size size, int m1type, OutputArray
map1, OutputArray map2)

Parameters 

cameraMatrix – Input camera matrix A =

cameraMatrix //不用解釋了吧 相機的內參矩陣

distCoeffs畸變矩陣 – Input vector of distortion coefficients (k1; k2; p1; p2[; k3[; k4; k5; k6]]) of 4,

5, or 8 elements. If the vector is NULL/empty, the zero distortion coefficients are assumed.

R代表的是旋轉矩陣

R – Optional rectification transformation in the object space (3x3 matrix). R1 or R2 , computed
by stereoRectify() can be passed here. If the matrix is empty, the identity transformation
is assumed. In cvInitUndistortMap R assumed to be an identity matrix.

newCameraMatrix – New camera matrix A0 =

size – Undistorted image size.
m1type – Type of the first output map that can be CV_32FC1 or CV_16SC2 . See
convertMaps() for details.
map1 – The first output map.//即是 u

map2 – The second output map.//即是 v