ORB 特征提取算法（實踐篇一）

Oriented FAST and Rotated BRIEF (ORB)

目標檢測是計算機視覺中最具挑戰性的問題之一。目標檢測即識別圖像中特定的對象，並能夠確定這些對象在圖像中的位置。例如，如果我們在下面的圖像中檢測汽車，我們不僅要檢測出圖像中有多少輛車，而且還要檢測出這些車在圖像中的位置。

為了進行這種基於對象的圖像分析，我們將使用ORB。ORB是一種非常快速的算法，可以從檢測到的關鍵點創建特征向量。ORB 算法具有一些很好的特性，比如旋轉不變性、光照不變性、噪音不變性等。

在本篇文章中，我們將驗證 ORB 的這些不變特性，使用 ORB 檢測人臉圖像中的關鍵點並進行匹配實驗。

加載訓練圖像

在 ORB 中，將要被尋找的目標圖像被稱為訓練圖像（training image）。此次使用的訓練圖像是一位女士的人臉圖像。

定位關鍵點

我們將使用 OpenCV 的 ORB 實現定位關鍵點並創建相應的 ORB 描述符。ORB 算法的參數是使用orb_create（）函數設置的。orb_create（）函數的參數及其默認值如下：

cv2.ORB_create(nfeatures = 500,
               scaleFactor = 1.2,
		       nlevels = 8,
		       edgeThreshold = 31,
		       firstLevel = 0,
		       WTA_K = 2,
		       scoreType = HARRIS_SCORE,
		       patchSize = 31,
		       fastThreshold = 20)

參數解釋如下：

• nfeatures - int
  Determines the maximum number of features (keypoints) to locate.

• scaleFactor - float
  Pyramid decimation ratio, must be greater than 1. ORB uses an image pyramid to find features, therefore you must provide the scale factor between each layer in the pyramid and the number of levels the pyramid has. A scaleFactor = 2 means the classical pyramid, where each next level has 4x less pixels than the previous. A big scale factor will diminish the number of features found.

• nlevels - int
  The number of pyramid levels. The smallest level will have a linear size equal to input_image_linear_size/pow(scaleFactor, nlevels).

• edgeThreshold - - int
  The size of the border where features are not detected. Since the keypoints have a specific pixel size, the edges of images must be excluded from the search. The size of the edgeThreshold should be equal to or greater than the patchSize parameter.

• firstLevel - int
  This parameter allows you to determine which level should be treated as the first level in the pyramid. It should be 0 in the current implementation. Usually, the pyramid level with a scale of unity is considered the first level.

• WTA_K - int
  The number of random pixels used to produce each element of the oriented BRIEF descriptor. The possible values are 2, 3, and 4, with 2 being the default value. For example, a value of 3 means three random pixels are chosen at a time to compare their brightness. The index of the brightest pixel is returned. Since there are 3 pixels, the returned index will be either 0, 1, or 2.

• scoreType - int
  This parameter can be set to either HARRIS_SCORE or FAST_SCORE. The default HARRIS_SCORE means that the Harris corner algorithm is used to rank features. The score is used to only retain the best features. The FAST_SCORE produces slightly less stable keypoints, but it is a little faster to compute.

• patchSize - int
  Size of the patch used by the oriented BRIEF descriptor. Of course, on smaller pyramid layers the perceived image area covered by a feature will be larger.

可見， cv2. ORB_create()函數支持的參數很多。前兩個參數（nfeatures和 scaleFactor）可能是最常用的參數。其他參數一般保持默認值既能獲得比較良好的結果。

在下面的代碼中，將使用 ORB_create()函數，並將要檢測的最大關鍵點數量nfeatures設置為 200，將縮放比率scaleFactor設置為 2.1。然后使用.detectandcompute（image）方法來定位給定訓練圖片 training_gray中的關鍵點並計算它們對應的 ORB 描述符。最后使用 cv2.drawKeypoints()函數來可視化 ORB 算法找到的關鍵點。

# Import copy to make copies of the training image
import copy

# Set the default figure size
plt.rcParams['figure.figsize'] = [14.0, 7.0]

# Set the parameters of the ORB algorithm by specifying the maximum number of keypoints to locate and
# the pyramid decimation ratio
orb = cv2.ORB_create(200, 2.0)

# Find the keypoints in the gray scale training image and compute their ORB descriptor.
# The None parameter is needed to indicate that we are not using a mask.
keypoints, descriptor = orb.detectAndCompute(training_gray, None)

# Create copies of the training image to draw our keypoints on
keyp_without_size = copy.copy(training_image)
keyp_with_size = copy.copy(training_image)

# Draw the keypoints without size or orientation on one copy of the training image 
cv2.drawKeypoints(training_image, keypoints, keyp_without_size, color = (0, 255, 0))

# Draw the keypoints with size and orientation on the other copy of the training image
cv2.drawKeypoints(training_image, keypoints, keyp_with_size, flags = cv2.DRAW_MATCHES_FLAGS_DRAW_RICH_KEYPOINTS)

# Display the image with the keypoints without size or orientation
plt.subplot(121)
plt.title('Keypoints Without Size or Orientation')
plt.imshow(keyp_without_size)

# Display the image with the keypoints with size and orientation
plt.subplot(122)
plt.title('Keypoints With Size and Orientation')
plt.imshow(keyp_with_size)
plt.show()

# Print the number of keypoints detected
print("\nNumber of keypoints Detected: ", len(keypoints))

正如我們在右圖中看到的，每個關鍵點都有一個中心、一個大小和一個角度。中心決定圖像中每個關鍵點的位置；每個關鍵點的大小由 BRIEF 用於創建其特征向量的 patch 大小決定；角度告訴我們由 rBRIEF 決定的關鍵點的方向。
一旦找到訓練圖像的關鍵點並計算出相應的 ORB 描述符，就可以對查詢圖像執行相同的操作。為了更清楚地了解 ORB 算法的特性，在下一節中，訓練圖像和查詢圖像將使用相同的圖片。

特征匹配

在下面的代碼中，我們將使用 OpenCV 的 BFMatcher 類比較訓練和查詢圖像中的關鍵點。“Brute-Force”匹配器的參數是使用 cv2.BFMatcher()函數設置的。cv2.BFMatcher()函數的參數及其默認值如下：

cv2.BFMatcher(normType = cv2.NORM_L2,
		 	  crossCheck = false)

Parameters:

• normType
  Specifies the metric used to determine the quality of the match. By default, normType = cv2.NORM_L2, which measures the distance between two descriptors. However, for binary descriptors like the ones created by ORB, the Hamming metric is more suitable. The Hamming metric determines the distance by counting the number of dissimilar bits between the binary descriptors. When the ORB descriptor is created using WTA_K = 2, two random pixels are chosen and compared in brightness. The index of the brightest pixel is returned as either 0 or 1. Such output only occupies 1 bit, and therefore the cv2.NORM_HAMMING metric should be used. If, on the other hand, the ORB descriptor is created using WTA_K = 3, three random pixels are chosen and compared in brightness. The index of the brightest pixel is returned as either 0, 1, or 2. Such output will occupy 2 bits, and therefore a special variant of the Hamming distance, known as the cv2.NORM_HAMMING2 (the 2 stands for 2 bits), should be used instead. Then, for any metric chosen, when comparing the keypoints in the training and query images, the pair with the smaller metric (distance between them) is considered the best match.

• crossCheck - bool
  A Boolean variable and can be set to either True or False. Cross-checking is very useful for eliminating false matches. Cross-checking works by performing the matching procedure two times. The first time the keypoints in the training image are compared to those in the query image; the second time, however, the keypoints in the query image are compared to those in the training image (i.e. the comparison is done backwards). When cross-checking is enabled a match is considered valid only if keypoint A in the training image is the best match of keypoint B in the query image and vice-versa (that is, if keypoint B in the query image is the best match of point A in the training image).

一旦設置了BFMatcher 的參數，就可以使用 .match(descriptors_train, descriptors_query)方法通過 ORB 描述符查找訓練圖像和查詢圖像之間的匹配關鍵點。最后使用 cv2.drawMatches ()函數來可視化 Brute-Force 匹配器找到的匹配關鍵點。此函數水平堆疊訓練和查詢圖像，並從訓練圖像中的關鍵點繪制線，以與查詢圖像中的最佳匹配關鍵點相對應。為了更清楚地了解 ORB 算法的特性，在下面的示例中，訓練圖像和查詢圖像使用內容相同的圖片。

import cv2
import matplotlib.pyplot as plt

# Set the default figure size
plt.rcParams['figure.figsize'] = [14.0, 7.0]

# Load the training image
image1 = cv2.imread('./images/face.jpeg')

# Load the query image
image2 = cv2.imread('./images/face.jpeg')

# Convert the training image to RGB
training_image = cv2.cvtColor(image1, cv2.COLOR_BGR2RGB)

# Convert the query image to RGB
query_image = cv2.cvtColor(image2, cv2.COLOR_BGR2RGB)

# Display the training and query images
plt.subplot(121)
plt.title('Training Image')
plt.imshow(training_image)
plt.subplot(122)
plt.title('Query Image')
plt.imshow(query_image)
plt.show()

# Convert the training image to gray scale
training_gray = cv2.cvtColor(training_image, cv2.COLOR_BGR2GRAY)

# Convert the query image to gray scale
query_gray = cv2.cvtColor(query_image, cv2.COLOR_BGR2GRAY)

# Set the parameters of the ORB algorithm by specifying the maximum number of keypoints to locate and
# the pyramid decimation ratio
orb = cv2.ORB_create(1000, 2.0)

# Find the keypoints in the gray scale training and query images and compute their ORB descriptor.
# The None parameter is needed to indicate that we are not using a mask in either case.
keypoints_train, descriptors_train = orb.detectAndCompute(training_gray, None)
keypoints_query, descriptors_query = orb.detectAndCompute(query_gray, None)

# Create a Brute Force Matcher object. Set crossCheck to True so that the BFMatcher will only return consistent
# pairs. Such technique usually produces best results with minimal number of outliers when there are enough matches.
bf = cv2.BFMatcher(cv2.NORM_HAMMING, crossCheck = True)

# Perform the matching between the ORB descriptors of the training image and the query image
matches = bf.match(descriptors_train, descriptors_query)

# The matches with shorter distance are the ones we want. So, we sort the matches according to distance
matches = sorted(matches, key = lambda x : x.distance)

# Connect the keypoints in the training image with their best matching keypoints in the query image.
# The best matches correspond to the first elements in the sorted matches list, since they are the ones
# with the shorter distance. We draw the first 300 mathces and use flags = 2 to plot the matching keypoints
# without size or orientation.
result = cv2.drawMatches(training_gray, keypoints_train, query_gray, keypoints_query, matches[:300], query_gray, flags = 2)

# Display the best matching points
plt.title('Best Matching Points')
plt.imshow(result)
plt.show()

# Print the number of keypoints detected in the training image
print("Number of Keypoints Detected In The Training Image: ", len(keypoints_train))

# Print the number of keypoints detected in the query image
print("Number of Keypoints Detected In The Query Image: ", len(keypoints_query))

# Print total number of matching points between the training and query images
print("\nNumber of Matching Keypoints Between The Training and Query Images: ", len(matches))

在上面的示例中，由於訓練圖像和查詢圖像完全相同，我們希望在兩個圖像中找到相同數量的關鍵點，並且所有關鍵點都匹配。可以清楚地看到，事實確實如此，ORB在兩個圖像中都找到了相同數量的關鍵點，而且 Brute-Force 匹配器能夠正確匹配訓練和查詢圖像中的所有關鍵點。

后記

在本部分使用 ORB 算法進行了最基本的特征匹配，下一步將驗證 ORB 算法的尺度不變性、旋轉不變性、光照不變性、噪聲不變性。

本文翻譯整理自 Udacity 計算機視覺納米學位練習，官方源碼連接：

https://github.com/udacity/CVND_Exercises/blob/master/1_4_Feature_Vectors/2. ORB.ipynb