Tensorflow/Keras model inference using OpenCV difficulties

edit

Hi all:

I have made a neural network classification model using Keras (Tensorflow) backend.

The architecture is as below:

from keras.applications.mobilenet_v2 import MobileNetV2 
from keras.layers import MaxPooling2D, Dropout, Dense, Reshape, Permute
from keras.models import Sequential

base_model = MobileNetV2(include_top=False, weights='imagenet', input_shape = (224, 224, 3))  

model = Sequential () 
model.add (base_model)
model.add (MaxPooling2D())  
n, h, w, c = model.output_shape
n = c*h*w
model.add (Permute([3, 1, 2]))  # Indicate NHWC data layout
model.add (Reshape((n,)))
model.add (Dropout(0.92))   
model.add (Dense(2, activation='sigmoid')) 

model.summary ()

Note: since I read OpenCV can't handle the "flatten" layer (and experimented), I replaced it with "permute" and "reshape" layer following instructions here. This works fine, I got 95%+ val accuracy.

I then converted the HDF5 format of the Keras model to .pb graph using this script that I found, this seems to work:

from tensorflow.python.tools import freeze_graph
from tensorflow.python.tools import optimize_for_inference_lib

MODEL_PATH = 'keras_test'
MODEL_NAME = 'test'
input_node_name = 'mobilenetv2_1.00_224_input'
output_node_name = 'dense_18/Sigmoid'

tf.train.write_graph(sess.graph_def, MODEL_PATH, f'{MODEL_NAME}_graph.pb', as_text=False)
tf.train.write_graph(sess.graph_def, MODEL_PATH, f'{MODEL_NAME}_graph.pbtxt')
tf.train.Saver().save(sess, f'{MODEL_PATH}/{MODEL_NAME}.chkp')

freeze_graph.freeze_graph(f'{MODEL_PATH}/{MODEL_NAME}_graph.pbtxt',
                          None, False,
                          f'{MODEL_PATH}/{MODEL_NAME}.chkp',
                          output_node_name,
                          "save/restore_all",
                          "save/Const:0",
                          f'{MODEL_PATH}/frozen_{MODEL_NAME}.pb', 
                          True, "")

The inference code is as below:

import cv2 as cv
cvNet = cv.dnn.readNetFromTensorflow('test_graph.pb')
img = cv.imread('some_img.jpg')
cvNet.setInput(cv.dnn.blobFromImage(img, size=(300, 300), swapRB=True, crop=False))
dnn_out = cvNet.forward()

But I keep getting this message when doing inference:

error: OpenCV(4.1.2) /io/opencv/modules/dnn/src/layers/fully_connected_layer.cpp:154: error: (-215:Assertion failed) srcMat.dims == 2 && srcMat.cols == weights.cols && dstMat.rows == srcMat.rows && dstMat.cols == weights.rows && srcMat.type() == weights.type() && weights.type() == dstMat.type() && srcMat.type() == CV_32F && (biasMat.empty() || (biasMat.type() == srcMat.type() && biasMat.isContinuous() && (int)biasMat.total() == dstMat.cols)) in function 'run'

Any advice would be appreciated, thanks in advance.