Tri Nguyen's Blog
Mobile app development, Kotlin, Swift, Xamarin, KMM


OpenCV for Xamarin.iOS

Posted on

Updated: 3 years ago I've worked on a Xamarin cross-platforms applications via iOS, Android, Windows Phone 8 which required using OpenCV for face detector and recognition. Even OpenCV has official frameworks for both iOS (ObjC) but it is pretty hard to make fully binding on Xamarin.iOS because most parts of framework are written on C/C++.

Below tutorial is a workaround solution, but effectively, for getting native OpenCV for Xamarin.iOS.

Overview

The idea is creating iOS Static library (CocoaTouch library) as a wrapper for specific functions (by requirements). This wrapper is written on Objective-C and implemented by communicating with OpenCV C++ classes.

Let's go through a sample spec: detect faces from any image

Step by step: OpenCV on Xamarin.iOS

First, create iOS Static Library

Should have an Objective-C library for wrapping all features that will work with OpenCV:

@interface OpenCVUtils : NSObject
+ (cv::Mat)cvMatFromUIImage:(UIImage *)image;
+ (UIImage *)UIImageFromCVMat:(cv::Mat)cvMat;
+ (CGRect) cgRectFromCVRect:(cv::Rect) cvRect;
@end
- (id) initWithFaceCascade:(NSString*) filePath;
- (NSArray*) detectFaces:(UIImage*) image;
@end

Second, Create Xamarin.iOS Binding

[assembly: LinkWith ("opencv2.a", LinkTarget.ArmV7 | LinkTarget.ArmV7s | LinkTarget.Simulator, ForceLoad = true, LinkerFlags="-lc++", Frameworks="Accelerate AssetsLibrary AVFoundation CoreGraphics CoreImage CoreMedia CoreVideo QuartzCore UIKit Foundation")]
interface FaceDetector {
 //- (id) initWithFaceCascade:(NSString*) filePath;
 [Export ("initWithFaceCascade:")]
 IntPtr Constructor (string filePath);

 //- (NSArray*) detectFaces:(UIImage*) image;
 [Export ("detectFaces:")]
 NSArray DetectFaces(UIImage image);
}

UIImage srcImage = UIImage.FromBundle("lena1");
NSArray arrFaces = _faceDetector.DetectFaces(srcImage);

Source code

Get the full source code: https://github.com/trinnguyen/xamarin.ios-opencv