Image alignment and registration with OpenCV

Image alignment and registration is the process of:

$1.$ Taking two images of the same object from slightly different angles.

$2.$ Automatically finding the transformation needed to align the images, using methods like feature

matching, similarity measures.

$3.$ Applying this transformation to warp one image so it matches the other.

How can OpenCV help with image alignment and registration?

There are several methods for aligning and registering images:

Image from: Benefits of dynamic susceptibility$-$weighted contrast$-$enhanced perfusion MRI for glioma

diagnosis and therapy

$1.$ The most common methods are feature$-$based. These include keypoint detectors $($like DoG,

Harris$),$ local invariant descriptors $($like SIFT, SURF, and ORB$),$ and keypoint matching

techniques $($like RANSAC$).$

$2.$ In medical imaging, similarity measures such as cross$-$correlation, sum of squared intensity

differences, and mutual information are often used.

We'll be using feature$-$based methods for our image alignment and registration, starting with detecting

keypoints in the two images.

Keypoints are meant to identify salient regions of an input image. For each keypoint, we extract local

invariant descriptors, which quantify the region surrounding each keypoint in the input image. SIFT

features, for example, are $128-$d$,$ so if we detected $528$ keypoints in a given input image, then we$$ll have a

Input images:

Use the given image as a first image. For the second one create a distorted version of the first

image by resizing and rotating the image. Note that imrotate rotates images in a

counterclockwise direction when you specify a positive angle of rotation.

$$ scaleFactor $=0\mathrm{.}7$;

$$ distorted $=$ imresize$($original$,$scaleFactor$)$;

$$ theta $=30$;

$$ distorted $=$ imrotate$($distorted$,$theta$)$;

$$ imshow$($distorted$)$

Here are the steps that you need to follow:

$1.$ Load image, convert to double and to grayscale.

$2.$ Create second image

$3.$ Detect feature points in both images. Use this Harris.

$4.$ Extract local neighborhoods around every keypoint in both images, and form descriptors using

SIFT. You can use the OpenCV library to extract keypoints and compute descriptors through the

function cv$2.$SIFT$\_$create$().$detect And Compute. This tutorial provides details about using SIFT

in OpenCV.

$5.$ Compute distances between every descriptor in one image and every descriptor in the other

image. In Python, you can use scipy.spatial.distance.cdist$($X$,$Y$,$'sqeuclidean'$)$ for fast computation

of Euclidean distance.

$6.$ Select putative matches based on the matrix of pairwise descriptor distances obtained above. You

can select all pairs whose descriptor distances are below a specified threshold, or select the top

few hundred descriptor pairs with the smallest pairwise distances.

$7.$ Implement RANSAC to estimate a homography mapping one image onto the other. Report the

number of inliers and the average residual for the inliers $($squared distance between the point

coordinates in one image and the transformed coordinates of the matching point in the other

image$).$

A very simple RANSAC implementation is sufficient. Use four matches to initialize the

homography in each iteration. You should output a single transformation that gets the most inliers

during all the iterations. For the various RANSAC parameters $($number of iterations, inlier

threshold$),$ play around with a few "reasonable" values and pick the ones that work best.

Homography fitting, calls for homogeneous least squares. The solution to the homogeneous least

squares system AX$=0$ is obtained from the SVD of A by the singular vector corresponding to the

smallest singular value. In Python, U$,$ s$,$ V $=$ numpy.linalg.svd$($A$)$ performs the singular value

decomposition and V$[$len$($V$)-1]$ gives the smallest singular value.

$8.$ Warp one image onto the other using the estimated transformation. In Python,

useskimage.transform.ProjectiveTransform and skimage.transform.warp.

$9.$ Create a new image contain two images.