Accord.Net makes it easy to add Image Stitching/Panoramas to your application
http://channel9.msdn.com/coding4fun/blog/AccordNet-makes-it-easy-to-add-Image-StitchingPanoramas-to-your-application
“I know I’m on a little bit of an “image” kick, but when I saw today’s project, though originally published a year ago, I liked how it shows off something that you might think is hard (it is) and something you might have thought you could never include in your application. Yet this project, which shows off a cool framework, makes it look easy. (And I promise, no more “image” stuff for a bit now. I’ve got it out of my system… well unless I come across something really cool… lol)
Automatic Image Stitching with Accord.NET
Introduction
The Accord.NET Framework is a relatively new extension framework for AForge.NET, a popular framework for computer vision and artificial intelligence. Accord.NET implements tools and features that are still not available in AForge.NET, such as Kernel Support Vector Machines, Discriminative and Projective Analysis, Hidden Markov Models, new Neural networks learning algorithms, new imaging filters, and other useful methods and tools. Accord.NET allows, among other things, to develop programs for automatic image stitching and possibly for automatic panorama creation.
A panorama is a picture that is made by combining a series of photos into one large picture. By combining a series of photos, it is possible to provide a complete view of an area or location that cannot fit in a single shot.
In the example below, we will demonstrate how to use features already available in Accord.NET to stitch two images together and create a simple and small panorama. In the last section, we will discuss ways to generalize the method for multiple pairs of images, creating full blown image mosaics.
For this demonstration, we will be using these two pictures taken from the UFSCar lake. The pictures have been taken a few degrees apart from each other. We can see that the gymnasium appears only in the second picture, whereas the leftmost portion of the lake appears only in the first picture.
What I liked was the depth that this article went into. It doesn’t simply cover the sample, but the science and math behind the magic that the sample makes look easy.Feature Extraction
In order to create our panorama, the general idea will consist in identifying common points between the two images and then projecting one of the images on top of the other in an effort to match those points. In order to identify those points, which from now on we will be calling interest points, we will be using a simple interest point detector known as the Harris Corners Detector.
While the full description of the Harris detector is beyond the scope of this article (but is available here), I thought it was interesting to include a little history about corner detectors and the Harris operator.
…Feature Matching
After our interest points have been detected, we need to correlate them somehow. For this, we will be using a maximum correlation rule to determine matches between our two images. The cross-correlation works by analyzing a window of pixels around every point in the first image and correlating them with a window of pixels around every other point in the second image. Points which have maximum bidirectional correlation will be taken as corresponding pairs.
…Homography Estimation
Now that we have two sets of correlated points, all we have to do is define a model which can translate points from one set to the other. What we are looking for is some kind of image transformation which can be used to project one of the two images on top of the other while matching most of the correlated feature points – we need a homography matrix matching the two images.
A homography is a projective transformation, a kind of transformation used in projective geometry. It describes what happens to the perceived positions of observed objects when the point of view of the observer changes. In more formal terms, a homography is an invertible transformation from the real projective plane to the projective plane that maps straight lines to straight lines.
By using homogeneous coordinates, one can represent an homography matrix as a 3×3 matrix with 8 degrees of freedom.
…
Once you have a firm foundation, the article does cover the code;Source Code
Using Accord.NET, the code for image stitching becomes extremely simple. Remember, we will need to perform four steps: Interest point detection, Correlation matching, Robust homography estimation, and Gradient blending. So before going into those steps, let’s define a few class variables (this code was taken from the sample application code):
The source code accompanying this article contains the sample application plus only a small subset of the Accord.NET Framework to avoid cluttering namespaces. If you wish to use Accord.NET, please be sure to download the latest version from the project page at Origo. By the way, the sample application assumes you will just push buttons in the right order. If you call a step before the previous steps have been computed, you will most probably get an exception. This is just to reflect the code that is needed for the stitching. It is also exactly the same code shown in the next sections.
…
Here’s a snap of the Solution (which compiled and ran the first time for me, which given the number of projects I look at, I really appreciate );
Here’s the app running;
Remember I said the blend/merge/stitching looked “easy”? Here’s pretty much all the code behind…
private void btnHarris_Click(object sender, EventArgs e)
{
// Step 1: Detect feature points using Harris Corners Detector
HarrisCornersDetector harris = new HarrisCornersDetector(0.04f, 1000f);
harrisPoints1 = harris.ProcessImage(img1).ToArray();
harrisPoints2 = harris.ProcessImage(img2).ToArray();
// Show the marked points in the original images
Bitmap img1mark = new PointsMarker(harrisPoints1).Apply(img1);
Bitmap img2mark = new PointsMarker(harrisPoints2).Apply(img2);
// Concatenate the two images together in a single image (just to show on screen)
Concatenate concatenate = new Concatenate(img1mark);
pictureBox.Image = concatenate.Apply(img2mark);
}
private void btnCorrelation_Click(object sender, EventArgs e)
{
// Step 2: Match feature points using a correlation measure
CorrelationMatching matcher = new CorrelationMatching(9);
IntPoint[][] matches = matcher.Match(img1, img2, harrisPoints1, harrisPoints2);
// Get the two sets of points
correlationPoints1 = matches[0];
correlationPoints2 = matches[1];
// Concatenate the two images in a single image (just to show on screen)
Concatenate concat = new Concatenate(img1);
Bitmap img3 = concat.Apply(img2);
// Show the marked correlations in the concatenated image
PairsMarker pairs = new PairsMarker(
correlationPoints1, // Add image1’s width to the X points to show the markings correctly
correlationPoints2.Apply(p => new IntPoint(p.X + img1.Width, p.Y)));
pictureBox.Image = pairs.Apply(img3);
}
private void btnRansac_Click(object sender, EventArgs e)
{
// Step 3: Create the homography matrix using a robust estimator
RansacHomographyEstimator ransac = new RansacHomographyEstimator(0.001, 0.99);
homography = ransac.Estimate(correlationPoints1, correlationPoints2);
// Plot RANSAC results against correlation results
IntPoint[] inliers1 = correlationPoints1.Submatrix(ransac.Inliers);
IntPoint[] inliers2 = correlationPoints2.Submatrix(ransac.Inliers);
// Concatenate the two images in a single image (just to show on screen)
Concatenate concat = new Concatenate(img1);
Bitmap img3 = concat.Apply(img2);
// Show the marked correlations in the concatenated image
PairsMarker pairs = new PairsMarker(
inliers1, // Add image1’s width to the X points to show the markings correctly
inliers2.Apply(p => new IntPoint(p.X + img1.Width, p.Y)));
pictureBox.Image = pairs.Apply(img3);
}
private void btnBlend_Click(object sender, EventArgs e)
{
// Step 4: Project and blend the second image using the homography
Blend blend = new Blend(homography, img1);
pictureBox.Image = blend.Apply(img2);
}
private void btnDoItAll_Click(object sender, EventArgs e)
{
// Do it all
btnHarris_Click(sender, e);
btnCorrelation_Click(sender, e);
btnRansac_Click(sender, e);
btnBlend_Click(sender, e);
}
If you’re looking to add image processing to your application, then this sample may be that first step down the Accord.Net/AForge.Net road you need…
Here’s a few more links you might find interesting:”Contour Analysis for Image Recognition in C#” Forging Player Card Detection and Recognition program with AForge.Net”
-Sent from Weave for Windows Phone 7
Ra Puke Moana 