Ask Your Question
1

Testing tensorflow model with opencv dnn

asked 2017-11-03 04:18:09 -0600

xxxcucus gravatar image

updated 2017-11-10 07:12:54 -0600

I have asked the following question on StackOverflow : https://stackoverflow.com/questions/4....

I am adding here more details and asking opencv specific question.

In the code given in StackOverflow the dimension of inputData is 64 X 32 X 32 X 3 and this format is required by the tf.nn.conv2d operator in tensorflow.

I am reading the network with opencv dnn: m_Net = cv::dnn::readNetFromTensorflow(m_TrainedModelPath);

To run the network I initialize with

 cv::Mat netInput = cv::dnn::blobFromImages(imagesBatch);
 m_Net.setInput(netInput, "input/Identity");

where images batch is as required by the opencv api a vector of cv::Mat. The cv::Mat were read with cv::imread and the image conversion BGR to RGB was performed.

But then the input of the network is 64 X 3 x 32 X 32. The network does not work from OpenCV. My question is does opencv internally swaps the data or am I required to do so ?

As required I add the code here:

for (auto testItem : m_TestData) {        
    if (batchCounter > 0 && batchCounter % m_BatchSize == 0) {
        //recognize and update counters
        cv::Mat netInput = cv::dnn::blobFromImages(imagesBatch);                  
        m_Net.setInput(netInput, "input/Identity");        //set the network input
        cv::Mat result = m_Net.forward("output/Mul");     //compute output

        ///compute maximum in the fully connected layer output 
        for (int i = 0; i < m_BatchSize; ++i) {
            int maxIdx = 0;
            double maxVal = result.at<float>(i, 0);
            for (int j = 1; j < result.size[1]; ++j) {
                double val = result.at<float>(i, j);
                if (val > maxVal) {
                    maxVal = val;
                    maxIdx = j;
                }
            }
            printf("Groundtruth: %d, Recognized %d\n", labelsBatch[i], maxIdx);
            if (labelsBatch[i] == maxIdx)
               countCorrRecog++; 
        }

        batchCounter = 0;
        imagesBatch.clear();
        labelsBatch.clear();
        countTestedImages += m_BatchSize;
    }

    cv::Mat img = cv::imread(testItem.first.toUtf8().constData());
    if (img.empty()) {
        std::cerr << "Can't read image from the file: " << testItem.first.toUtf8().constData() << std::endl;
        exit(-1);
    }

    cv::cvtColor(img, img, cv::COLOR_BGR2RGB);
    if (m_InputImageSize != img.size())
        cv::resize(img, img, m_InputImageSize); //Resize image to input size       
    imagesBatch.push_back(img);
    labelsBatch.push_back(testItem.second);
    batchCounter++;

}

I do not receive errors but countCorrRecog stays very small when compared to countTestedImages.

edit retag flag offensive close merge delete

Comments

sidenote: you probably should not convert to rgb manually, since this is already done from here (swapRB=true)

berak gravatar imageberak ( 2017-11-03 05:13:10 -0600 )edit

thanks - I have not noticed the default value.

xxxcucus gravatar imagexxxcucus ( 2017-11-03 05:32:33 -0600 )edit

1 answer

Sort by ยป oldest newest most voted
1

answered 2017-11-05 09:12:46 -0600

xxxcucus gravatar image

Looking in the opencv source code we find in /modules/dnn/src/tensorflow/tf_importer.cpp in function parseTensor the following code:

  if (dims == 4)
   {
       int num = shape[0], channels = shape[1], height = shape[2], width = shape[3];
       int total = num*channels*height*width;
       for(int i_n = 0; i_n < shape[0]; i_n++) {
            for(int i_c = 0; i_c < shape[1]; i_c++) {
                for(int i_h = 0; i_h < shape[2]; i_h++) {
                    for(int i_w = 0; i_w < shape[3]; i_w++) {
                       int dst_i = channels*height*width*i_n + height*width*i_c + width*i_h + i_w;
                       int src_i = channels*height*width*i_n + i_c + channels*width*i_h + channels*i_w;

                       CV_Assert(dst_i < total);
                       CV_Assert(src_i < total);

                      dstData[dst_i] = data[src_i];
                   }
               }
           }
       }
 }

That is 4 tensors are transposed to account for the opencv format N X C X H X W. So the answer to the question is - the change in the data format is taken into account.

edit flag offensive delete link more

Comments

1
xxxcucus gravatar imagexxxcucus ( 2017-11-10 04:38:26 -0600 )edit

Question Tools

1 follower

Stats

Asked: 2017-11-03 04:18:09 -0600

Seen: 2,219 times

Last updated: Nov 10 '17