Template Matching in Python

In this article, we will be tackling an enjoyable image processing technique called template matching. In particular, template matching can be very useful when used for object detection and recognition. This technique has the same intuition as the ground-truth white balancing technique. It uses the source image to get a reference image; this reference image will be used as a comparison point for other parts of the source image. It would be best to explain this process with an example image of an aircraft.

from skimage.io import imread, imshow
carrier = imread('aircraft_carrier.jpg')
imshow(carrier);

Aircraft Carrier

For simplicity purposes, we will convert the image to grayscale.

from skimage.color import rgb2gray
carrier_gray = rgb2gray(carrier)
imshow(carrier_gray);

Grayscaled Aircraft Carrier

Now that we have our grayscale image, we will use it as the source image and pick one of the planes as a reference image.

template = carrier_gray[648:744,775:838]
imshow(template);

One of the airplanes

Now that we have defined our reference image, we can apply template matching by importing the match_template function.

from skimage.feature import match_template
result = match_template(carrier_gray, template)
imshow(result, cmap='viridis');

The template matching reveals that several areas are brightly colored. We can assume that template can be found once in the image; we can try to find it by looking for the pixel with the highest value. The highest value would mean the matching is closest to the reference image.

import numpy as np
x, y = np.unravel_index(np.argmax(result), result.shape)
import matplotlib.pyplot as plt
imshow(carrier_gray)
template_width, template_height = template.shape
rect = plt.Rectangle((y, x), template_height, template_width, color='y', 
                     fc='none')
plt.gca().add_patch(rect);

Grayscaled Aircraft Carrier with the Referenced Plane

The yellow box indicates where the reference image is located in the source image. Furthermore, we can finally locate multiple matches of the reference airplane by looking for peaks with a certain value for correlation.

from skimage.feature import peak_local_max
imshow(carrier_gray)
template_width, template_height = template.shape
for x, y in peak_local_max(result, threshold_abs=0.8):
    rect = plt.Rectangle((y, x), template_height, template_width, color='y', 
                         fc='none')
    plt.gca().add_patch(rect);

With a threshold of 0.8, the template matching was able to detect another airplane in the aircraft. We can decrease the threshold to 0.4 so that more airplanes can be detected.

With a threshold of 0.4, more airplanes were detected. However, it is still not enough to detect all the airplanes. We can decrease the threshold to 0.3 so that we can detect almost all the airplanes.

With a threshold of 0.3, the matching detected almost all the airplanes but also detected areas that did not have an airplane.

Overall, template matching is a pretty good and fun image processing technique, but we just have to find the right threshold to get the right match. We can see that its application can be very advantageous with image detection and recognition models. Happy matching everyone!