How to reduce image descriptor dimension to speed up image retrival [closed]
This is a image retrival example, get image sift descritprs which is n rows 128 cols, then call flann function to match, I want to speed it up, so i use PCA to reduce vector dimension such as 64 cols, but after PCA it can't find any match, Is then PCA code is wrong or can't call flann function to match after PCA processing
code:
#include <stdio.h>
#include <unistd.h>
#include <iostream>
#include <opencv2 opencv.hpp="">
#include <opencv2 xfeatures2d.hpp="">
using namespace cv;
using namespace cv::xfeatures2d;
using namespace std;
Mat PcaDecColumn(const Mat &src, int dstColNum)
{
Mat beforePCA = src.clone();
PCA pca(beforePCA, Mat(), CV_PCA_DATA_AS_ROW, dstColNum);
Mat afterProject = pca.project(beforePCA);
//Mat dst = pca.backProject(afterProject);
Mat dst = afterProject;
return dst;
}
int
main( int argc, char **argv )
{
cv::Mat queryImage = cv::imread(argv[1]);
cv::Mat frameImage = cv::imread(argv[2]);
if (frameImage.empty() || queryImage.empty())
{
printf( "%s %d err\n", __func__, __LINE__ );
return -1;
}
vector<keypoint> queryKeypoints;
vector<keypoint> frameKeypoints;
Mat queryDescriptors;
Mat frameDescriptors;
//Ptr<orb> orb = ORB::create(1000);
Ptr<feature2d> orb = xfeatures2d::SIFT::create();
orb->detectAndCompute(queryImage, noArray(), queryKeypoints, queryDescriptors);
orb->detectAndCompute(frameImage, noArray(), frameKeypoints, frameDescriptors);
queryDescriptors = PcaDecColumn(queryDescriptors, 64);
frameDescriptors = PcaDecColumn(frameDescriptors, 64);
if (frameDescriptors.empty() || queryDescriptors.empty())
{
printf( "%s %d err\n", __func__, __LINE__ );
return -1;
}
vector<dmatch> matches;
vector<vector<dmatch> > totalMatches;
FlannBasedMatcher matcher;
matcher.knnMatch(frameDescriptors, queryDescriptors, totalMatches, 2);
const float minRatio = 1.f / 1.5f;
for (size_t i = 0; i < totalMatches.size(); i++) {
const DMatch& bestMatch = totalMatches[i][0];
const DMatch& betterMatch = totalMatches[i][1];
float distanceRatio = bestMatch.distance / betterMatch.distance;
if (distanceRatio < minRatio)
matches.push_back(bestMatch);
}
printf( "matches size:%d\n", matches.size() );
Mat img_match;
drawMatches(frameImage, frameKeypoints, queryImage, queryKeypoints, matches, img_match);
imshow("match", img_match);
waitKey(0);
return 0;
}
if at all, you should make a single PCA offline (maybe even from independant data) and use that to project both the train & the query data(again, using the SAME projection matrix), else you're comparing apples to pears in the end.
but maybe you could just use the 64bit SURF (with UPRIGHT flag) ? it's also almost as fast as ORB !
since you cant use the above feature matching in a useful way for CBIR like tasks (it would need a "global" image descriptor, and clustering/classification based on that),
please also lookup "BagOfWords" and "vlad descriptors"
tks break gravatar image break, this is new code:
// modify here PCA pca(queryDescriptors, Mat(), CV_PCA_DATA_AS_ROW, col); queryDescriptors = pca.project(queryDescriptors); frameDescriptors = pca.project(frameDescriptors); //queryDescriptors = PcaDecColumn(queryDescriptors, 64); //frameDescriptors = PcaDecColumn(frameDescriptors, 64);
It work, but match size reduce from 130 to 20, PCA can speed up match but also reduce match size, how to improve it.
imho, you're on the wrong bus here, with both PCA and feature matching.
i think, you need something like this instead, with an SVM(if you want to do classification) or a flann::Index (if the goal is clustering / nearest neighbour retrieval)
My goal is match one page from a picture book, i have try use BOW without PCA, BOW is fast but accuracy not good enough, i only need one target, BOW may find several similar.