OpenCV for Xamarin.iOS

Updated: Last 3 years I’ve worked on a Xamarin cross-platforms applications via iOS, Android, Windows Phone 8 which required using OpenCV for face detector and recognition. Even OpenCV has official frameworks for both iOS (ObjC) but it is pretty hard to make fully binding on Xamarin.iOS because most parts of framework are written on C/C++.

Below tutorial is a workaround solution, but effectively, for getting native OpenCV for Xamarin.iOS.

Overview

The idea is creating iOS Static library (CocoaTouch library) as a wrapper for specific functions (by requirements).  This wrapper is written on Objective-C and implemented by communicating with OpenCV C++ classes.

Let’s go through a sample spec: detect faces from an image

iOS Static Library: Simple library with one function: Detect faces on an image with OpenCV, export .a file for using on Xamarin

Xamarin.iOS Binding project: Create a C# wrapper for static library, iOS Application: demo how binding and function of static library, detect faces on an image and draw them with red rectangle on image

Step by step: OpenCV on Xamarin.iOS

First, create iOS Static Library

Should have an Objective-C library for wrapping all features that will work with OpenCV:

  • Download opencv2.framework from OpenCV site (http://opencv.org/downloads.html)
  • XCode: Create iOS Static Library project with a Objective-C class that contains required functions (can be multiple classes for management), named FaceDetector
#import <Foundation/Foundation.h>

@interface OpenCVUtils : NSObject
+ (cv::Mat)cvMatFromUIImage:(UIImage *)image;
+ (UIImage *)UIImageFromCVMat:(cv::Mat)cvMat;
+ (CGRect) cgRectFromCVRect:(cv::Rect) cvRect;
@end
@interface FaceDetector : NSObject
- (id) initWithFaceCascade:(NSString*) filePath;
- (NSArray*) detectFaces:(UIImage*) image;
@end
  • Build iOS Static Library and get libFaceDetector.a file

Second, Create Xamarin.iOS Binding

  • Xamarin Studio: New Solution -> iOS -> Unified API -> iOS Binding Project
  • Drag libFaceDetector.a to Binding project
  • Back to opencv2.framework file, open it, go to opencv2.framework -> Versions -> A -> opencv2. Rename this file to opencv2.a. Drag this to Binding project. (This binary file is required as reference from libFaceDetector.a)
  • Open opencv2.linkwith.cs, add LinkerFlags and Frameworks
[assembly: LinkWith ("opencv2.a", LinkTarget.ArmV7 | LinkTarget.ArmV7s | LinkTarget.Simulator, ForceLoad = true, LinkerFlags="-lc++", Frameworks="Accelerate AssetsLibrary AVFoundation CoreGraphics CoreImage CoreMedia CoreVideo QuartzCore UIKit Foundation")]
  • ApiDefinition.cs : Define interfaces for working with libFaceDetector.a . Can create manually if experienced with Xamarin.iOS Binding Metadata or using Objective Sharpie (command line tool)
[BaseType (typeof (NSObject))]
interface FaceDetector {
 //- (id) initWithFaceCascade:(NSString*) filePath;
 [Export ("initWithFaceCascade:")]
 IntPtr Constructor (string filePath);

 //- (NSArray*) detectFaces:(UIImage*) image;
 [Export ("detectFaces:")]
 NSArray DetectFaces(UIImage image);
}
  • Create Xamarin.iOS project, add reference to above Xamarin.iOS Binding project, try use C# wrapper classes

UIImage srcImage = UIImage.FromBundle("lena1");
NSArray arrFaces = _faceDetector.DetectFaces(srcImage);

Source code

Get the full source code: https://github.com/trinnguyen/xamarin.ios-opencv