圖像的視差匹配（Stereo Matching）

這里要求用我們自己計算得到的視差圖和給的視差圖作比較來比較我們得到的視差圖的好壞程度，我視差圖返回的值是計算得到的視差乘以3之后的圖，所以在計算時我不是兩個值相差大於1，而是大於3。由於兩個圖像都乘3了。所以要大於3。我傳入的參數是兩個圖像的矩陣。由於我是寫了一個腳本咯跑全部測例的。在腳本里邊已經把圖像讀出來了

<span style="font-size:18px;">function [percentnumberbadpixels] = PercentBadPixels(mydisparitymap,groundtruthdisparitymap)
mydisparitymap = double(mydisparitymap);
groundtruthdisparitymap = double(groundtruthdisparitymap);
[Rmydisparitymap, Cmydisparitymap] = size(mydisparitymap);
numbadpixels = 0;
for i = 1:Rmydisparitymap
    for j = 1:Cmydisparitymap
        if (abs(mydisparitymap(i,j) - groundtruthdisparitymap(i,j)) > 3)
              numbadpixels = numbadpixels+1;
        end
    end
end
percentnumbadpixels = numbadpixels/(Rmydisparitymap*Cmydisparitymap*1.0);
percentnumbadpixels = floor(percentnumbadpixels*100);
str = strcat(num2str(percentnumbadpixels),'%');
percentnumberbadpixels = str;
end
</span>

SSD:

  左視差圖

  右視差圖

   

實現細節：

   輸入的圖像是24位的圖像，可是這些計算都是基於強度（intensities）來計算的，在matlab中把彩色圖轉為灰度圖的函數rgb2gray實質上就是轉為其強度灰度圖像。

原理：

如圖：

對於同一圖像。左眼看到的圖像相對於右眼看到的圖像的位置是不一樣的，如上圖，對於同樣位置在右眼看到的圖像的位置在左眼看到的圖像的位置的前邊，所以左視差圖和右視差圖的搜索方法不同，對於左視差圖：

 因為其位置比右眼看到的位置要遠，則對於在右眼圖像同樣的物理位置應該往該位置的前邊搜索才可能找到最相近的部分，即要減去d。

同理。對於右視差圖則剛好相反。要加d。

實現細節：

1）首先把圖像通過intensities轉為灰度圖像：

2）設定搜索深度d的值和patch的值。然后對圖像進行補邊界，使得能夠從原圖像的第一個像素開始進行匹配，我用的補邊界的方法和曾經做濾波時候原理一樣。我都是補周圍的值

3）對左圖或右圖中的每個像素（i。j）作為patch的中心在另外的左圖或右圖上來尋找與之最相近的patch部分的搜索深度d，詳細為：

 對於模板圖像中每個不移動的patch。把該patch里邊的每個像素和另外一張目標圖相應得patch（加上搜索深度d的）上的像素點做差的平方和然后再累加起來，每個d相應一個patch，每個patch相應一個值（sum），然后把求出在搜索深度d的范圍內最小的sum，從而得到該最小的sum的搜索深度d的值，當遇到邊界時用推斷來處理，詳細實現例如以下： 

假設是求右視差圖的話，上邊原理以及介紹了，要加上d來進行搜索

NCC

左視差圖：

右視差圖:

實現細節：

圖像轉為灰度圖及邊界處理和SSD同樣，公式實現：

對左圖或右圖中的每個像素（i，j）作為patch的中心，在另外的左圖或右圖上來尋找與之最相近的patch部分的搜索深度d。

 1）對於模板圖像中每個不移動的patch，把該patch里邊的每個像素和另外一張目標圖相應的patch（加上搜索深度d的）上的相應像素點分別相乘相加，把模板圖中的patch和目標圖的patch中的像素值分別平方相加，然后再兩個值相乘開方。

2) 每個d相應一個patch。每個patch相應一個值（ncctemp），然后把求出在搜索深度d的范圍內最大的ncctemp，從而得到該最大的ncctemp的搜索深度d的值，當遇到邊界時用推斷來處理

NCC與SSD的差別：

NCC抗光照性強，SSD抗光照性弱

如：Add a small constant amount of intensity (e.g. 10) to all right eye images, and re-run the above two methods. Analyze how the intensity change affects the results (i.e. the quality) of the two methods. Explain in which ways that NCC is a better matching cost than SSD.

這里我在view5加了常數10

NCC結果例如以下：Aloe disp1壞點率為24%

View5強度沒有加10是壞點率為：24%

SSD 結果例如以下：Aloe disp1壞點率為38%

View5強度沒有加10是壞點率為：26%

分析：

由上邊公式可SSD是依據兩幅圖上patch里兩個像素點差值的平方求和來找到最小的值。依據這個最小值我們覺得距離為d的patch是匹配的部分，可是在拍照片是。會受到一些因素的影響，比方光照強度等，這樣會照成左眼和右眼看到的圖像的強度不一樣，假如右眼看到的圖像強度添加10，這樣對於patch里邊像素值相減得到的結果就會有非常大的變化，從而造成求得的最小值的d也會不一樣，因此的圖像和右眼圖像沒有強度變化的結果有非常大的不同。

而對於NCC來說，分子上是兩個patch里邊的像素分別相乘相加，而分母是每一個patch里邊的像素值平方然后求和在相乘開方，即使右眼的亮度加了。可是分子和分母的值都會同一時候增大，並且增大的值差點兒相同同樣。增大的值相除接近為1，所以得到的結果和右眼強度沒有加10的結果基本上一樣，沒有多大的變化。

所以假設左眼圖和右眼圖受光照強度干擾比較大的情況下，NCC比SSD好。

nccdispl.m

function [output_img] = nccdispl(leftimg,rightimg)
[nrleft,ncleft,nleft] = size(leftimg);
[nrright,ncright,nright] =size(rightimg);
disparitymap = zeros(nrright,ncright);
if (nleft > 2)
   leftimg = rgb2gray(leftimg);
end
if (nright > 2)
    rightimg = rgb2gray(rightimg);
    rightimg = rightimg+10;
end
d = 70;
winsize = 11;
disparitymap = zeros(nrright,ncright);
leftimg = double(leftimg);
rightimg = double(rightimg);
leftimg = supplyborder(leftimg,winsize);
rightimg = supplyborder(rightimg,winsize);
win = (winsize-1)/2;
for i = 1+win:nrright+win
    for j = 1+win:ncright+win
        min = -9999999;
        for k = 0:d
            numerator = 0.0;
            powerrightwin = 0.0;
            powerleftwin = 0.0;
            for a=-win:win
                for b = -win:win
                    if j+b-k > 0
                        numerator = numerator+(leftimg(i+a,j+b)*rightimg(i+a,j+b-k));
                        powerleftwin = powerleftwin+(leftimg(i+a,j+b)*leftimg(i+a,j+b));
                        powerrightwin = powerrightwin+(rightimg(i+a,j+b-k)*rightimg(i+a,j+b-k));
                    end
                end
            end
            ncctemp = numerator/(sqrt(powerrightwin*powerleftwin));
            if (min < ncctemp && j+b-k > 0)
                min = ncctemp;
                recordk = k;
            end
        end
        disparitymap(i-win,j-win) = recordk;
    end
end
disparitymap = uint8(3*disparitymap);
output_img = disparitymap;
% imshow(disparitymap);
% imwrite(disparitymap,strcat('C:\Users\samsung-\Desktop\output\nccdisp1.png'));
end

</pre><pre name="code" class="plain"><pre name="code" class="plain" style="font-size:18px;">ssddispl.m

<pre name="code" class="plain">function [output_img] = ssddispl(leftimg,rightimg)
[nrleft,ncleft,nleft] = size(leftimg);
[nrright,ncright,nright] =size(rightimg);
if (nleft > 2)
   leftimg = rgb2gray(leftimg);
end

if (nright > 2)
    rightimg = rgb2gray(rightimg);
    rightimg = rightimg+10;
end
disparitymap = zeros(nrleft,ncleft);
d = 70;
winsize = 11;
leftimg = double(leftimg);
rightimg = double(rightimg);
leftimg = supplyborder(leftimg,winsize);
rightimg = supplyborder(rightimg,winsize);
win = (winsize-1)/2;
for i = 1+win:nrright+win
    for j = 1+win:ncright+win
        max = 99999999;
        for k = 0:d
            sum = 0.0;
            for a=-win:win
                for b = -win:win
                    if j+b-k > 0
                         temp = leftimg(i+a,j+b)-rightimg(i+a,j+b-k);
                         temp = temp*temp;
                         sum = sum+temp;
                    end                   
                end
            end
            if (max > sum && j+b-k > 0)
                max = sum;
                mind = k;
            end
        end
        disparitymap(i-win,j-win) = mind;
    end
end
disparitymap = uint8(3*disparitymap);
output_img = disparitymap;
% imshow(disparitymap);
% imwrite(disparitymap,strcat('C:\Users\samsung-\Desktop\output\ssddisp1.png'));
end

<pre name="code" class="plain">nccdispr.m

function [output_img] = nccdispr(leftimg,rightimg)
[nrleft,ncleft,nleft] = size(leftimg);
[nrright,ncright,nright] =size(rightimg);
if (nleft > 2)
   leftimg = rgb2gray(leftimg);
end
if (nright > 2)
    rightimg = rgb2gray(rightimg);
    rightimg = rightimg+10;
end
d = 70;
winsize = 11;
disparitymap = zeros(nrleft,ncleft);
leftimg = double(leftimg);
rightimg = double(rightimg);
leftimg = supplyborder(leftimg,winsize);
rightimg = supplyborder(rightimg,winsize);
win = (winsize-1)/2;
for i = 1+win:nrleft+win
    for j = 1+win:ncleft+win
        min = -999999;
        for k = 0:d
             numerator = 0.0;
             powerrightwin = 0.0;
             powerleftwin = 0.0;
            for a = -win:win
                for b = -win:win
                    if j+b+k <= ncleft+2*win
                        numerator = numerator+(rightimg(i+a,j+b)*leftimg(i+a,j+b+k));
                        powerrightwin = powerrightwin+(rightimg(i+a,j+b)*rightimg(i+a,j+b));
                        powerleftwin = powerleftwin+(leftimg(i+a,j+b+k)*leftimg(i+a,j+b+k));
                    end
                     
                end
            end
            ncctemp = numerator/(sqrt(powerrightwin*powerleftwin));
            if (min < ncctemp && j+b+k <= ncleft+2*win)
                min = ncctemp;
                recordk = k;
            end
        end
        disparitymap(i-win,j-win) = recordk;
    end
end
disparitymap = uint8(3*disparitymap);
output_img = disparitymap;
% imshow(disparitymap);
% imwrite(disparitymap,strcat('C:\Users\samsung-\Desktop\output\nccdisp5.png'));
end

</pre><pre name="code" class="plain"><pre name="code" class="plain" style="font-size:18px;">ssddispr.m

<pre name="code" class="plain">function [output_img] = ssddispr(leftimg,rightimg)
[nrleft,ncleft,nleft] = size(leftimg);
[nrright,ncright,nright] =size(rightimg);
if (nleft > 2)
   leftimg = rgb2gray(leftimg);
end
if (nright > 2)
    rightimg = rgb2gray(rightimg);
    rightimg = rightimg+10;
end
disparitymap = zeros(nrleft,ncleft);
d = 70;
winsize = 11;
leftimg = double(leftimg);
rightimg = double(rightimg);
leftimg = supplyborder(leftimg,winsize);
rightimg = supplyborder(rightimg,winsize);
win = (winsize-1)/2;
for i = 1+win:nrleft+win
    for j = 1+win:ncleft+win
        max = 999999999;
        for k = 0:d
            sum = 0.0;
            for a=-win:win
                for b = -win:win
                    if j+b+k <= ncleft+2*win
                         temp = rightimg(i+a,j+b)-leftimg(i+a,j+b+k);
                         temp = temp*temp;
                         sum = sum+temp;
                    end
                end
            end
            if (max > sum && j+b+k <= ncleft+win)
                max = sum;
                mind = k;
            end
        end
        disparitymap(i-win,j-win) = mind;
    end
end
disparitymap = uint8(3*disparitymap);
output_img = disparitymap;
% imshow(disparitymap);
% imwrite(disparitymap,strcat('C:\Users\samsung-\Desktop\output\ssddisp5.png'));
end