ORB 主要特性實驗
我們現在將探討 ORB 算法的幾個主要屬性：
- 尺度不變性
- 旋轉不變性
- 光照不變性
- 噪聲不變性
同樣，為了更清楚地了解 ORB 算法的特性，在下面的示例中訓練圖像和查詢圖像將使用相同內容的圖片。
1. 尺度不變性
ORB 算法具有尺度不變性。這意味着它能夠檢測圖像中的對象，而不管其大小。為了驗證這一點，我們現在將使用 Brute-Force 匹配器來匹配訓練圖像和查詢圖像之間的關鍵點。注意這里的查詢圖像是原始訓練圖像大小的 1/4。
import cv2
import matplotlib.pyplot as plt
# Set the default figure size
plt.rcParams['figure.figsize'] = [14.0, 7.0]
# Load the training image
image1 = cv2.imread('./images/face.jpeg')
# Load the query image
image2 = cv2.imread('./images/faceQS.png')
# Convert the training image to RGB
training_image = cv2.cvtColor(image1, cv2.COLOR_BGR2RGB)
# Convert the query image to RGB
query_image = cv2.cvtColor(image2, cv2.COLOR_BGR2RGB)
# Display the images
plt.subplot(121)
plt.title('Training Image')
plt.xlim([training_image.shape[1], 0])
plt.ylim([training_image.shape[0], 0])
plt.imshow(training_image)
plt.subplot(122)
plt.xlim([training_image.shape[1], 0])
plt.ylim([training_image.shape[0],0])
plt.title('Query Image')
plt.imshow(query_image)
plt.show()
# Convert the training image to gray scale
training_gray = cv2.cvtColor(training_image, cv2.COLOR_BGR2GRAY)
# Convert the query image to gray scale
query_gray = cv2.cvtColor(query_image, cv2.COLOR_BGR2GRAY)
# Set the parameters of the ORB algorithm by specifying the maximum number of keypoints to locate and
# the pyramid decimation ratio
orb = cv2.ORB_create(1000, 2.0)
# Find the keypoints in the gray scale training and query images and compute their ORB descriptor.
# The None parameter is needed to indicate that we are not using a mask in either case.
keypoints_train, descriptors_train = orb.detectAndCompute(training_gray, None)
keypoints_query, descriptors_query = orb.detectAndCompute(query_gray, None)
# Create a Brute Force Matcher object. Set crossCheck to True so that the BFMatcher will only return consistent
# pairs. Such technique usually produces best results with minimal number of outliers when there are enough matches.
bf = cv2.BFMatcher(cv2.NORM_HAMMING, crossCheck = True)
# Perform the matching between the ORB descriptors of the training image and the query image
matches = bf.match(descriptors_train, descriptors_query)
# The matches with shorter distance are the ones we want. So, we sort the matches according to distance
matches = sorted(matches, key = lambda x : x.distance)
# Connect the keypoints in the training image with their best matching keypoints in the query image.
# The best matches correspond to the first elements in the sorted matches list, since they are the ones
# with the shorter distance. We draw the first 30 mathces and use flags = 2 to plot the matching keypoints
# without size or orientation.
result = cv2.drawMatches(training_gray, keypoints_train, query_gray, keypoints_query, matches[:30], query_gray, flags = 2)
# Display the best matching points
plt.title('Best Matching Points')
plt.imshow(result)
plt.show()
# Print the shape of the training image
print('\nThe Training Image has shape:', training_gray.shape)
#Print the shape of the query image
print('The Query Image has shape:', query_gray.shape)
# Print the number of keypoints detected in the training image
print("\nNumber of Keypoints Detected In The Training Image: ", len(keypoints_train))
# Print the number of keypoints detected in the query image
print("Number of Keypoints Detected In The Query Image: ", len(keypoints_query))
# Print total number of matching points between the training and query images
print("\nNumber of Matching Keypoints Between The Training and Query Images: ", len(matches))
在上面的示例中，訓練圖像為 553 x 471 像素，而查詢圖像為 138 x 117 像素，為原始訓練圖像的 1/4 大小。另外請注意，在查詢圖像中檢測到的關鍵點數量只有 65 個，遠小於在訓練圖像中發現的 831 個關鍵點。然而，還是可以看到 Brute-Force 匹配器可以將查詢圖像中的大多數關鍵點與其訓練圖像中的相應關鍵點匹配。
2.旋轉不變性
ORB 算法具有旋轉不變性。這意味着它能夠檢測圖像中的對象而不管它們的方向如何。為了驗證這一點，現在將使用 Brute-Force 匹配器來匹配訓練圖像和旋轉 90 度的查詢圖像之間的關鍵點。
# Set the parameters of the ORB algorithm by specifying the maximum number of keypoints to locate and
# the pyramid decimation ratio
orb = cv2.ORB_create(1000, 2.0)
# Find the keypoints in the gray scale training and query images and compute their ORB descriptor.
# The None parameter is needed to indicate that we are not using a mask in either case.
keypoints_train, descriptors_train = orb.detectAndCompute(training_gray, None)
keypoints_query, descriptors_query = orb.detectAndCompute(query_gray, None)
# Create a Brute Force Matcher object. Set crossCheck to True so that the BFMatcher will only return consistent
# pairs. Such technique usually produces best results with minimal number of outliers when there are enough matches.
bf = cv2.BFMatcher(cv2.NORM_HAMMING, crossCheck = True)
# Perform the matching between the ORB descriptors of the training image and the query image
matches = bf.match(descriptors_train, descriptors_query)
# The matches with shorter distance are the ones we want. So, we sort the matches according to distance
matches = sorted(matches, key = lambda x : x.distance)
# Connect the keypoints in the training image with their best matching keypoints in the query image.
# The best matches correspond to the first elements in the sorted matches list, since they are the ones
# with the shorter distance. We draw the first 100 mathces and use flags = 2 to plot the matching keypoints
# without size or orientation.
result = cv2.drawMatches(training_gray, keypoints_train, query_gray, keypoints_query, matches[:100], query_gray, flags = 2)
# Display the best matching points
plt.title('Best Matching Points')
plt.imshow(result)
plt.show()
在上面的示例中，我們看到在兩個圖像中檢測到的關鍵點數量非常相似，即使查詢圖像被旋轉，Brute-Force 匹配器仍然可以匹配大約 78% 的關鍵點。另外值得一提的是，大多數匹配的關鍵點都接近於特定的面部特征，如眼睛、鼻子和嘴巴。
3.光照不變性
ORB 算法具有光照不變性。這意味着它能夠檢測圖像中的對象，而不管其亮度如何。為了驗證這一點，使用 Brute-Force 匹配器來匹配訓練圖像和更亮的查詢圖像之間的關鍵點。
# Set the parameters of the ORB algorithm by specifying the maximum number of keypoints to locate and
# the pyramid decimation ratio
orb = cv2.ORB_create(1000, 2.0)
# Find the keypoints in the gray scale training and query images and compute their ORB descriptor.
# The None parameter is needed to indicate that we are not using a mask in either case.
keypoints_train, descriptors_train = orb.detectAndCompute(training_gray, None)
keypoints_query, descriptors_query = orb.detectAndCompute(query_gray, None)
# Create a Brute Force Matcher object. Set crossCheck to True so that the BFMatcher will only return consistent
# pairs. Such technique usually produces best results with minimal number of outliers when there are enough matches.
bf = cv2.BFMatcher(cv2.NORM_HAMMING, crossCheck = True)
# Perform the matching between the ORB descriptors of the training image and the query image
matches = bf.match(descriptors_train, descriptors_query)
# The matches with shorter distance are the ones we want. So, we sort the matches according to distance
matches = sorted(matches, key = lambda x : x.distance)
# Connect the keypoints in the training image with their best matching keypoints in the query image.
# The best matches correspond to the first elements in the sorted matches list, since they are the ones
# with the shorter distance. We draw the first 100 mathces and use flags = 2 to plot the matching keypoints
# without size or orientation.
result = cv2.drawMatches(training_gray, keypoints_train, query_gray, keypoints_query, matches[:100], query_gray, flags = 2)
# Display the best matching points
plt.title('Best Matching Points')
plt.imshow(result)
plt.show()
在上面的示例中，可以看到在兩個圖像中檢測到的關鍵點數量非常相似，即使查詢圖像更亮，Brute-Force 匹配器仍然可以匹配找到的大約 63% 的關鍵點。
4.噪聲不變性
ORB 算法具備噪聲不變性。這意味着它能夠檢測圖像中的對象，即使圖像有一定程度的噪聲。為了驗證這一點，現在使用Brute-Force 匹配器來匹配訓練圖像和具有大量噪聲的查詢圖像之間的點。
# Set the parameters of the ORB algorithm by specifying the maximum number of keypoints to locate and
# the pyramid decimation ratio
orb = cv2.ORB_create(1000, 1.3)
# Find the keypoints in the gray scale training and query images and compute their ORB descriptor.
# The None parameter is needed to indicate that we are not using a mask in either case.
keypoints_train, descriptors_train = orb.detectAndCompute(training_gray, None)
keypoints_query, descriptors_query = orb.detectAndCompute(query_gray, None)
# Create a Brute Force Matcher object. We set crossCheck to True so that the BFMatcher will only return consistent
# pairs. Such technique usually produces best results with minimal number of outliers when there are enough matches.
bf = cv2.BFMatcher(cv2.NORM_HAMMING, crossCheck = True)
# Perform the matching between the ORB descriptors of the training image and the query image
matches = bf.match(descriptors_train, descriptors_query)
# The matches with shorter distance are the ones we want. So, we sort the matches according to distance
matches = sorted(matches, key = lambda x : x.distance)
# Connect the keypoints in the training image with their best matching keypoints in the query image.
# The best matches correspond to the first elements in the sorted matches list, since they are the ones
# with the shorter distance. We draw the first 100 mathces and use flags = 2 to plot the matching keypoints
# without size or orientation.
result = cv2.drawMatches(training_gray, keypoints_train, query_gray, keypoints_query, matches[:100], query_gray, flags = 2)
# we display the image
plt.title('Best Matching Points')
plt.imshow(result)
plt.show()
在上面的示例中，可以再次看到在兩個圖像中檢測到的關鍵點數量非常相似，即使查詢圖像有很多噪聲，Brute-Force 匹配器仍然可以匹配找到的大約 63% 的關鍵點。另外，注意大多數匹配的關鍵點都接近於特定的面部特征，如眼睛、鼻子和嘴巴。此外，可以看到一些沒有不相同的特征點也被匹配了，這可能是由具有相同源的噪聲導致的。需要注意的是，在本例中使用的是 1.3 的金字塔縮放比率，而不是前面示例中使用的 2.0。
簡單目標識別
下面將實現使用 ORB 算法來基於訓練圖像檢測另一個圖像中的人臉。
import cv2
import matplotlib.pyplot as plt
# Set the default figure size
plt.rcParams['figure.figsize'] = [14.0, 7.0]
# Load the training image
image1 = cv2.imread('./images/face.jpeg')
# Load the query image
image2 = cv2.imread('./images/Team.jpeg')
# Convert the training image to RGB
training_image = cv2.cvtColor(image1, cv2.COLOR_BGR2RGB)
# Convert the query image to RGB
query_image = cv2.cvtColor(image2, cv2.COLOR_BGR2RGB)
# Display the images
plt.subplot(121)
plt.imshow(training_image)
plt.title('Training Image')
plt.subplot(122)
plt.imshow(query_image)
plt.title('Query Image')
plt.show()
# Set the default figure size
plt.rcParams['figure.figsize'] = [34.0, 34.0]
# Convert the training image to gray scale
training_gray = cv2.cvtColor(training_image, cv2.COLOR_BGR2GRAY)
# Convert the query image to gray scale
query_gray = cv2.cvtColor(query_image, cv2.COLOR_BGR2GRAY)
# Set the parameters of the ORB algorithm by specifying the maximum number of keypoints to locate and
# the pyramid decimation ratio
orb = cv2.ORB_create(5000, 2.0)
# Find the keypoints in the gray scale training and query images and compute their ORB descriptor.
# The None parameter is needed to indicate that we are not using a mask in either case.
keypoints_train, descriptors_train = orb.detectAndCompute(training_gray, None)
keypoints_query, descriptors_query = orb.detectAndCompute(query_gray, None)
# Create copies of the query images to draw our keypoints on
query_img_keyp = copy.copy(query_image)
# Draw the keypoints with size and orientation on the copy of the query image
cv2.drawKeypoints(query_image, keypoints_query, query_img_keyp, flags = cv2.DRAW_MATCHES_FLAGS_DRAW_RICH_KEYPOINTS)
# Display the query image with the keypoints with size and orientation
plt.title('Keypoints With Size and Orientation', fontsize = 30)
plt.imshow(query_img_keyp)
plt.show()
# Print the number of keypoints detected
print("\nNumber of keypoints Detected: ", len(keypoints_query))
我們可以看到查詢圖像在圖像的許多部分都有關鍵點。既然我們已經掌握了訓練和查詢圖像的關鍵點和 ORB 描述符，那么就可以使用 Burte-Force 匹配器來嘗試在查詢圖像中定位女人的臉。
# Set the default figure size
plt.rcParams['figure.figsize'] = [34.0, 34.0]
# Create a Brute Force Matcher object. We set crossCheck to True so that the BFMatcher will only return consistent
# pairs. Such technique usually produces best results with minimal number of outliers when there are enough matches.
bf = cv2.BFMatcher(cv2.NORM_HAMMING, crossCheck = True)
# Perform the matching between the ORB descriptors of the training image and the query image
matches = bf.match(descriptors_train, descriptors_query)
# The matches with shorter distance are the ones we want. So, we sort the matches according to distance
matches = sorted(matches, key = lambda x : x.distance)
# Connect the keypoints in the training image with their best matching keypoints in the query image.
# The best matches correspond to the first elements in the sorted matches list, since they are the ones
# with the shorter distance. We draw the first 85 mathces and use flags = 2 to plot the matching keypoints
# without size or orientation.
result = cv2.drawMatches(training_gray, keypoints_train, query_gray, keypoints_query, matches[:85], query_gray, flags = 2)
# we display the image
plt.title('Best Matching Points', fontsize = 30)
plt.imshow(result)
plt.show()
# Print the number of keypoints detected in the training image
print("Number of Keypoints Detected In The Training Image: ", len(keypoints_train))
# Print the number of keypoints detected in the query image
print("Number of Keypoints Detected In The Query Image: ", len(keypoints_query))
# Print total number of matching Keypoints between the training and query images
print("\nNumber of Matching Keypoints Between The Training and Query Images: ", len(matches))
可以清楚地看到，即使在查詢圖像中有許多人臉，Burte-Force 匹配器也能夠在查詢圖像中正確地定位女人的臉。
后記
關於 ORB 的討論就到這里就告一段落了，下一篇將討論 HOG 算法。
本文翻譯整理自 Udacity 計算機視覺納米學位練習，官方源碼連接：
https://github.com/udacity/CVND_Exercises/blob/master/1_4_Feature_Vectors/2. ORB.ipynb