Ask Your Question
7

Best way to store a Mat object in Android

asked 2013-03-10 15:24:28 -0600

user1446598 gravatar image

What is the best way to store a Mat object in Android? (there is a similar, unanswered, question here)

In image processing apps; it requires quite a lot of processing time to perform necessary pipelines which could be redundant. Thus, Mats would be preferably stored to prevent re-computation every time. I considered using SQLlite databases, but those seem to add unnecessary complexity in reading and writing Mat objects, where the method I understand would be writing each individual pixel in a Mat object in a separate row.

I came across solutions (regarding saving custom classes, not OpenCV in specific) that suggested saving the file externally to the SD card. That seems quite plausible, but the problem is that class Mat must implement Serializable class (according to many online references, as here), which is not the case. I don't wish to alter OpenCV's library albeit being open source, because I don't have much experience around the library.

Are there any time, and memory, efficient ways to preserving Mat objects for later launches of the app?

Thank you for your time.

edit retag flag offensive close merge delete

Comments

1

I'm not sure about android, but OpenCV provides a FileStorage class that allows you to write matrices to text files as XML or YAML. That is usually sufficient. If your Mat can reasonably be represented as an image, you can just save them to pngs as well

awknaust gravatar imageawknaust ( 2013-03-10 16:09:53 -0600 )edit
1

Last time I checked FileStorage is not ported for android - like it says in the stackoverflow linked in the question.

Rui Marques gravatar imageRui Marques ( 2013-03-10 16:55:39 -0600 )edit

6 answers

Sort by » oldest newest most voted
5

answered 2013-03-11 06:38:24 -0600

Andrey Pavlenko gravatar image

You need to use FileStorage (http://docs.opencv.org/modules/core/doc/xml_yaml_persistence.html#xml-yaml-persistence) but in C++ on native level since it isn't yet wrapped to Java.

edit flag offensive delete link more
5

answered 2014-02-18 18:09:45 -0600

Rui Marques gravatar image

updated 2014-02-20 18:26:42 -0600

In my opinion the most universal way to store a Mat is to first convert it to a data-interchange format like JSON.

After you are able to do that conversion you have a lot of flexibility to store it. JSON is easily converted to a String and/or sent through a network connection.

With OpenCV C++ you are able to store data as YAML, but that is not available for Android yet like it was pointed by Andrey Pavlenko. JSON here has the same purpose as YAML.

To parse JSON in Java you can use this easy to use library Google GSON.

And here is my first attempt to do exactly that (I did a simple test and it worked, let me know if there are problems):

public static String matToJson(Mat mat){        
    JsonObject obj = new JsonObject();

    if(mat.isContinuous()){
        int cols = mat.cols();
        int rows = mat.rows();
        int elemSize = (int) mat.elemSize();    

        byte[] data = new byte[cols * rows * elemSize];

        mat.get(0, 0, data);

        obj.addProperty("rows", mat.rows()); 
        obj.addProperty("cols", mat.cols()); 
        obj.addProperty("type", mat.type());

        // We cannot set binary data to a json object, so:
        // Encoding data byte array to Base64.
        String dataString = new String(Base64.encode(data, Base64.DEFAULT));

        obj.addProperty("data", dataString);            

        Gson gson = new Gson();
        String json = gson.toJson(obj);

        return json;
    } else {
        Log.e(TAG, "Mat not continuous.");
    }
    return "{}";
}

public static Mat matFromJson(String json){
    JsonParser parser = new JsonParser();
    JsonObject JsonObject = parser.parse(json).getAsJsonObject();

    int rows = JsonObject.get("rows").getAsInt();
    int cols = JsonObject.get("cols").getAsInt();
    int type = JsonObject.get("type").getAsInt();

    String dataString = JsonObject.get("data").getAsString();       
    byte[] data = Base64.decode(dataString.getBytes(), Base64.DEFAULT); 

    Mat mat = new Mat(rows, cols, type);
    mat.put(0, 0, data);

    return mat;
}
edit flag offensive delete link more

Comments

@Rui Marques

i want to xml convert json but my xml or yaml file too big row = 65536 , cols = 5

so how to convert it?

황인학 gravatar image황인학 ( 2014-05-22 03:19:42 -0600 )edit

I do not know if I understood your question correctly, you want to convert XML to JSON? Why would you do that?

Rui Marques gravatar imageRui Marques ( 2014-05-23 05:40:41 -0600 )edit

@Rui Marques

saved xml file in pc and load in android

황인학 gravatar image황인학 ( 2014-05-25 21:54:22 -0600 )edit

Do you have any reason against using JSON in the PC instead of XML?

Rui Marques gravatar imageRui Marques ( 2014-06-02 08:13:18 -0600 )edit

This code works only for CV_8U a CV_8S matrices.

jnovacho gravatar imagejnovacho ( 2014-06-28 11:09:09 -0600 )edit
1

Any idea how to get this running for CV_32f matrices? Would be highly appreciated.

jhoffjann gravatar imagejhoffjann ( 2014-11-14 05:04:06 -0600 )edit

it's late, but maybe changing byte[] data into float[] data for CV_32f matrices can do it. And other type for other matrices. Based from here

gi gravatar imagegi ( 2016-08-18 08:52:57 -0600 )edit
1

answered 2014-03-26 21:14:55 -0600

firstwave gravatar image

I use my own Mat <-> xml read/write.

/*
 * Only implement XML <-> opencv Mat read and write.
 */

public class TaFileStorage {
    // static
    public static final int READ = 0;
    public static final int WRITE = 1;

    // varaible
    private File file;
    private boolean isWrite;
    private Document doc;
    private Element rootElement;

    public TaFileStorage() {
        file = null;
        isWrite = false;
        doc = null;
        rootElement = null;
    }


    // read or write
    public void open(String filePath, int flags ) {
        try {
            if( flags == READ ) {
                open(filePath);
            }
            else {
                create(filePath);
            }
        } catch(Exception e) {
            e.printStackTrace();
        }

    }


    // read only
    public void open(String filePath) {
        try {
            file = new File(filePath);
            if( file == null || file.isFile() == false ) {
                System.err.println("Can not open file: " + filePath );
            }
            else {
                isWrite = false;
                doc = DocumentBuilderFactory.newInstance().newDocumentBuilder().parse(file);
                doc.getDocumentElement().normalize();
            }
        } catch(Exception e) {
            e.printStackTrace();
        }

    }

    // write only
    public void create(String filePath) {
        try {
            file = new File(filePath);
            if( file == null ) {
                System.err.println("Can not wrtie file: " + filePath );
            }
            else {
                isWrite = true;
                doc = DocumentBuilderFactory.newInstance().newDocumentBuilder().newDocument();

                rootElement = doc.createElement("opencv_storage");
                doc.appendChild(rootElement);
            }
        } catch(Exception e) {
            e.printStackTrace();
        }
    }

    public Mat readMat(String tag) {
        if( isWrite ) {
            System.err.println("Try read from file with write flags");
            return null;
        }

        NodeList nodelist = doc.getElementsByTagName(tag);
        Mat readMat = null;

        for( int i = 0 ; i<nodelist.getLength() ; i++ ) {
            Node node = nodelist.item(i);

            if( node.getNodeType() == Node.ELEMENT_NODE ) {
                Element element = (Element)node;

                String type_id = element.getAttribute("type_id");
                if( "opencv-matrix".equals(type_id) == false) {
                    System.out.println("Fault type_id ");
                }

                String rowsStr = element.getElementsByTagName("rows").item(0).getTextContent();
                String colsStr = element.getElementsByTagName("cols").item(0).getTextContent();
                String dtStr = element.getElementsByTagName("dt").item(0).getTextContent();
                String dataStr = element.getElementsByTagName("data").item(0).getTextContent();

                int rows = Integer.parseInt(rowsStr);
                int cols = Integer.parseInt(colsStr);
                int type = CvType.CV_8U;

                Scanner s = new Scanner(dataStr);

                if( "f".equals(dtStr) ) {
                    type = CvType.CV_32F;
                    readMat = new Mat( rows, cols, type );
                    float fs[] = new float[1];
                    for( int r=0 ; r<rows ; r++ ) {
                        for( int c=0 ; c<cols ; c++ ) {
                            if( s.hasNextFloat() ) {
                                fs[0] = s.nextFloat();
                            }
                            else {
                                fs[0] = 0;
                                System.err.println("Unmatched number of float value at rows="+r + " cols="+c);
                            }
                            readMat.put(r, c, fs);
                        }
                    }
                }
                else if( "i".equals(dtStr) ) {
                    type = CvType.CV_32S;
                    readMat = new Mat( rows, cols, type );
                    int is[] = new int[1];
                    for( int r=0 ; r<rows ; r++ ) {
                        for( int c=0 ; c<cols ; c++ ) {
                            if( s.hasNextInt() ) {
                                is[0] = s.nextInt();
                            }
                            else {
                                is[0] = 0;
                                System.err.println("Unmatched number of int value at rows="+r + " cols="+c);
                            }
                            readMat.put(r, c, is);
                        }
                    }
                }
                else if( "s".equals(dtStr) ) {
                    type = CvType.CV_16S;
                    readMat = new Mat( rows, cols, type );
                    short ss[] = new short[1];
                    for( int r=0 ; r<rows ; r++ ) {
                        for( int c=0 ; c<cols ; c++ ) {
                            if( s.hasNextShort() ) {
                                ss[0] = s.nextShort();
                            }
                            else {
                                ss[0] = 0;
                                System.err.println("Unmatched number of int value at rows="+r + " cols="+c);
                            }
                            readMat.put(r, c, ss);
                        }
                    }
                }
                else if( "b".equals(dtStr) ) {
                    readMat = new Mat( rows, cols, type );
                    byte bs[] = new byte[1];
                    for( int r=0 ; r<rows ; r++ ) {
                        for( int c=0 ; c<cols ; c++ ) {
                            if( s.hasNextByte() ) {
                                bs[0 ...
(more)
edit flag offensive delete link more

Comments

1

@firstwave

this code is too slow

황인학 gravatar image황인학 ( 2014-05-25 23:19:29 -0600 )edit
0

answered 2017-05-23 10:19:10 -0600

sengsational gravatar image

Although I have not run timings, I suspect that going through formats such as XML or JSON would require more space and computing resources than using primitives.

Below is some code that can be used to save and retrieve MatOfKeyPoint to SQLite database in an Android environment. This is using OpenCV 2.4.11 so SIFT is available.

What you see when you run this application is your test image (which you need to supply and put in the drawable folder) with added keypoints.

The siftTest() method starts by computing keyPoints which is MatOfKeyPoint type. The code saves the underlying data of that object in the database, then reads that data out and creates a new object keyPointsFromDb, the contents of which are applied to the original image and the result is displayed.

public class MainActivity extends AppCompatActivity {
    static {
        System.loadLibrary("native-lib");
        System.loadLibrary("opencv_java");
        System.loadLibrary("nonfree");

    }
    private ImageView imageView;
    private Bitmap inputImage; // make bitmap from image resource
    private FeatureDetector detector = FeatureDetector.create(FeatureDetector.SIFT);

    @Override
    protected void onCreate(Bundle savedInstanceState) {
        super.onCreate(savedInstanceState);
        setContentView(R.layout.activity_main);
        inputImage = BitmapFactory.decodeResource(getResources(), R.drawable.test);
        imageView = (ImageView) this.findViewById(R.id.imageView);
        siftTest();
    }

    public void siftTest() {
        Mat rgba = new Mat();
        Utils.bitmapToMat(inputImage, rgba);
        MatOfKeyPoint keyPoints = new MatOfKeyPoint();
        Imgproc.cvtColor(rgba, rgba, Imgproc.COLOR_RGBA2GRAY);
        detector.detect(rgba, keyPoints);

        // Save to database
        MatchingDatabaseAdapter.addKeypoints(keyPoints, getApplicationContext());

        // Opens database cursor
        MatchingDatabaseAdapter.getAllRecordsCursor(getApplicationContext()); 

        // Gets the first item in the database (as an example... you could loop through many/all)
        MatOfKeyPoint keyPointsFromDb = MatchingDatabaseAdapter.getKeypointsFromNextCursorPosition();

        // Closes database
        MatchingDatabaseAdapter.closeDb(); 

        Features2d.drawKeypoints(rgba, keyPointsFromDb, rgba);
        Utils.matToBitmap(rgba, inputImage);
        imageView.setImageBitmap(inputImage);

    }
}

Here is the database code which contains some details associated with converting to the byte array required for the database. I didn't include everything associated with using a database, since that's really a different topic.

public class MatchingDatabaseAdapter {
    ...
    ...
    ...

    public static void addKeypoints(MatOfKeyPoint keyPoints, Context context) {
        float[] data = new float[(int)keyPoints.total() * keyPoints.channels()]; // make a spot to save the data
        keyPoints.get(0,0,data); // load the data;
        ByteBuffer buffer = ByteBuffer.allocate(data.length * 4);
        for (int i = 0; i < data.length; i++){
            buffer.putFloat(data[i]);
        }
        byte[] byteArray = buffer.array();
        addBlob(byteArray, keyPoints.rows(), keyPoints.cols(), keyPoints.type(), context);
    }

    public static void addBlob(byte[] blob, int rows, int columns, int mattype, Context context) throws SQLException {
        if (mDb == null) mDb = openDb(context);
        ContentValues contentValues = new ContentValues();
        contentValues.put(DatabaseHelper.BLOB_FIELD_NAME, blob);
        contentValues.put(DatabaseHelper.ROWS_FIELD_NAME, rows);
        contentValues.put(DatabaseHelper.COLUMNS_FIELD_NAME, columns);
        contentValues.put(DatabaseHelper.MATTYPE_FIELD_NAME, mattype);
        long x = mDb.insert(DatabaseHelper.TABLE_NAME, null, contentValues);
        Log.v(TAG, "insert said " + x + " and blob was " + blob.length + " long.");
        closeDb();
    }

    public static Cursor getAllRecordsCursor(Context context) {
        if (mDb == null || !mDb.isOpen()) mDb = openDb(context);
        mCursor = mDb.query(DatabaseHelper.TABLE_NAME, null, null, null, null,null, null);
        boolean hasRecords = mCursor.moveToFirst();
        Log.v(TAG, "MatchingDatabaseAdapter.getAllRecordsCursor() cursor created. " + mCursor + " and " + (hasRecords?"has records":"has NO RECORDS"));
        return mCursor;
    }

     public static MatOfKeyPoint getKeypointsFromNextCursorPosition() {
        MatOfKeyPoint keyPoints = null;
        if (mCursor != null) {
            Log.v(TAG, "mCursor has ...
(more)
edit flag offensive delete link more
0

answered 2013-03-13 17:04:21 -0600

user1446598 gravatar image

Thank you for the answers and comments. I prefer not to use Android NDK at the moment, and thus have tried saving Mat objects as Bitmap images. A boolean variable would be set (and amended) via Android's SharedPreferences class to signify whether the data needs to be read from the phone's storage as .Bmp (already saved), or need to be processed from scratch (which should, ideally, be only once).

  • To convert from Bmp to Mat, check this) OpenCV method, bitmapToMat.
  • To convert from Mat to Bmp, check this) OpenCV method, matToBitmap.

I still need to try this method in more depth, but I think it should offer a reasonable alternative.

edit flag offensive delete link more
0

answered 2013-03-20 06:50:03 -0600

greven gravatar image

Like others pointed FileStorage is not available on Android, so in order to avoid using the Android NDK I just save it to a Bitmap.

Here's the code I use:

    /**
 * Saves a Mat to the SD card application folder as a jpg.  
 * 
 * @param source The image to save.
 * @param filename The name of the file to be saved.
 * @param directoryName The directory where the 
 * @param ctx The activity context.
 * @param colorConversion The openCV color conversion to apply to the image. -1 will use no color conversion.
 */
public void saveImageToDisk(Mat source, String filename, String directoryName, Context ctx, int colorConversion){

    Mat mat = source.clone();
    if(colorConversion != -1)
        Imgproc.cvtColor(mat, mat, colorConversion, 4);

    Bitmap bmpOut = Bitmap.createBitmap(mat.cols(), mat.rows(), Bitmap.Config.ARGB_8888);
    Utils.matToBitmap(mat, bmpOut);
    if (bmpOut != null){

        mat.release();
        OutputStream fout = null;
        String root = Environment.getExternalStorageDirectory().getAbsolutePath();
        String dir = root + "/" + ctx.getResources().getString(R.string.app_name) + "/" + directoryName;
        String fileName = filename + ".jpg";
        File file = new File(dir);
        file.mkdirs();
        file = new File(dir, fileName);

        try {
            fout = new FileOutputStream(file);
            BufferedOutputStream bos = new BufferedOutputStream(fout);
            bmpOut.compress(Bitmap.CompressFormat.JPEG, 100, bos);
            bos.flush();
            bos.close();
            bmpOut.recycle();
        } catch (FileNotFoundException e) {
            e.printStackTrace();
        }

        catch (IOException e) {
            e.printStackTrace();
        }
    }
    bmpOut.recycle();
}

Hope it helps someone.

edit flag offensive delete link more

Comments

Thanks man

Dab gravatar imageDab ( 2015-11-24 10:47:02 -0600 )edit

Question Tools

1 follower

Stats

Asked: 2013-03-10 15:24:28 -0600

Seen: 24,285 times

Last updated: May 23 '17