Ask Your Question
0

Importing Bidirectional LSTM model via onnx shows an error

asked 2020-08-07 04:09:04 -0600

cherie gravatar image

updated 2020-08-11 03:14:02 -0600

Hi,

I'm trying to export model from easyOCR to ONNX and import it through openCV. I successfully exported it to ONNX, checked that it runs well, but failed to import the model through openCV using readNetFromONNX. Below is my model export code and the error:

batch_size=1
x = torch.rand(batch_size,1,64,256).float().cpu()
torch.onnx.export(model, (x,''), "ocr0807_0.onnx")
net = cv2.dnn.readNetFromONNX('ocr0807_0.onnx') <- where error occurs

 error: (-215:Assertion failed) (int)_numAxes == inputs[0].size() in function 'getMemoryShapes'

The error occurs at Bidirectional LSTM layer of the model even though I'm using openCV 4.4.0 and python 3.7.

Below is the part of model code from easyOCR, which include Bidirectional LSTM:

""" Sequence modeling"""
        self.SequenceModeling = nn.Sequential(
            BidirectionalLSTM(self.FeatureExtraction_output, hidden_size, hidden_size),
            BidirectionalLSTM(hidden_size, hidden_size, hidden_size))
        self.SequenceModeling_output = hidden_size

Would you please help me with this problem? Thank you.

(NEWLY ADDED) The full error message is:

error msg is cv2.error: OpenCV(4.4.0) ../modules/dnn/src/layers/permute_layer.cpp:134: error: (-215:Assertion failed) (int)_numAxes == inputs[0].size() in function 'getMemoryShapes'

Also, here is the error reproducing minimal test case code:

import torch.nn as nn
import torch

class BidirectionalLSTM(nn.Module):
   def __init__(self, input_size, hidden_size, output_size):
       super(BidirectionalLSTM, self).__init__()
       self.rnn = nn.LSTM(input_size, hidden_size, bidirectional=True, batch_first=True)
       self.linear = nn.Linear(hidden_size * 2, output_size)

   def forward(self, input):
       self.rnn.flatten_parameters()
       recurrent, _ = self.rnn(input)
       output = self.linear(recurrent)
       return output


class Model(nn.Module):
   def __init__(self):
       super(Model, self).__init__()
       self.SequenceModeling = nn.Sequential(
           BidirectionalLSTM(512, 512, 512),
           BidirectionalLSTM(512, 512, 512))

   def forward(self, input, text):
       contextual_feature = self.SequenceModeling(input)
       return contextual_feature


x = torch.rand(1, 65, 512).float().cpu()
model = Model()
model.eval()
torch.onnx.export(model, x, "ocr0811_1.onnx")

import cv2
net = cv2.dnn.readNetFromONNX('ocr0811_1.onnx')

Thank you.

edit retag flag offensive close merge delete

Comments

please show, how you construct the model, also please give us the exact error msg (you truncated it, so now we can't see where it came from)

also: (x,'') looks weird. you have 2 inputs (one empty?) ?

berak gravatar imageberak ( 2020-08-10 07:51:03 -0600 )edit
1

The model construction code is same as the one in the link(https://github.com/JaidedAI/Easy.... The exact error msg is cv2.error: OpenCV(4.4.0) ../modules/dnn/src/layers/permute_layer.cpp:134: error: (-215:Assertion failed) (int)_numAxes == inputs[0].size() in function 'getMemoryShapes'

Also, the model takes 2 inputs but the second one is actually does nothing. It even works without that input. That's why I made input as (x, ' '), the second one left empty

cherie gravatar imagecherie ( 2020-08-10 22:29:46 -0600 )edit

thanks for the nice reproducer code (really useful !) ;)

It even works without that input.

not here. TypeError: forward() missing 1 required positional argument: 'text'

while exporting it. but if i keep it like (x,''), at least i can reproduce the cv2 problem

yea, text is never used, how silly ...

berak gravatar imageberak ( 2020-08-11 03:07:27 -0600 )edit

indeed, the 4th permutation layer has _numAxes=3, and inputs[0].size() = 2

hmm, error goes away, if i remove one of the BidirectionalLSTM layers from the Model

.... but it is back, once i do net.setInput(x.numpy()); net.forward();

berak gravatar imageberak ( 2020-08-11 05:55:40 -0600 )edit

1 answer

Sort by ยป oldest newest most voted
2

answered 2020-08-12 20:49:27 -0600

cherie gravatar image

updated 2020-08-12 21:06:04 -0600

I uploaded this problem to issue and the bug under pull request now. https://github.com/opencv/opencv/issu...

edit flag offensive delete link more

Question Tools

1 follower

Stats

Asked: 2020-08-07 04:09:04 -0600

Seen: 986 times

Last updated: Aug 12 '20