How can i use the Object tracking API of vision framework on ios11?

Question

// init bounding
CGRect rect = CGRectMake(0, 0, 0.3, 0.3);
VNSequenceRequestHandler* reqImages = [[VNSequenceRequestHandler alloc] init];
VNRectangleObservation* ObserveRect = [VNRectangleObservation observationWithBoundingBox:rect];
VNTrackRectangleRequest* reqRect = [[VNTrackRectangleRequest alloc] initWithRectangleObservation:ObserveRect];
NSArray<VNRequest *>* requests = [NSArray arrayWithObjects:reqRect, nil];
BOOL bsucc = [reqImages performRequests:requests onCGImage:img.CGImage error:&error];

// get tracking bounding
VNDetectRectanglesRequest* reqRectTrack = [VNDetectRectanglesRequest new];
NSArray<VNRequest *>* requestsTrack = [NSArray arrayWithObjects:reqRectTrack, nil];
[reqImages performRequests:requestsTrack onCGImage:img.CGImage error:&error];

VNRectangleObservation* Observe = [reqRectTrack.results firstObject];
CGRect boundingBox = Observe.boundingBox;

Why the boundingBox value is incorrect?

How can i find the demo of vision.framework of ios11 ?

I run into the same problem as you, I have found the example used in Vision Keynote, they do resizing on `boundingBox` values. But not working on my side. Here is the sample : https://developer.apple.com/sample-code/wwdc/2017/ImageClassificationwithVisionandCoreML.zip Let me know if you find the solution — Akhu, Jun 09 '17 at 20:03
I find the demo in the Keynote too， ' // Create request handler let requestHandler = VNSequenceRequestHandler() // Start the tracking with an observation let observations = detectionRequest.results as! [VNDetectedObjectObservation] let objectsToTrack = observations.map { VNTrackObjectRequest(detectedObjectObservation: $0) } // Run the requests requestHandler.perform(objectsToTrack, on: pixelBuffer) // Lets look at the results for request in objectsToTrack for observation in request.results as! [VNDetectedObjectObservation] ' but it don't working. — Alberl, Jun 12 '17 at 04:36
@Alberl Do you got any solution for detecting object from static image , I am facing problem for converting the points from one coordinate system to another. — The iCoder, Sep 20 '17 at 06:50

score 4 · Answer 1 · answered Jun 19 '17 at 11:32

4

Vision Framework tracking an object, a demo for this can be found at this link:

https://github.com/jeffreybergier/Blog-Getting-Started-with-Vision

The Blogger goes into great details here of getting the demo working and has a gif showing a working build.

Hope this is what you are after.

answered Jun 19 '17 at 11:32

Luke James Stephens

382
4
14

Thanks Luke, you are great ! :) – kxko Jul 02 '17 at 14:33

score 4 · Accepted Answer · answered Jun 20 '17 at 16:43

4

Here is my simple example of using Vision framework: https://github.com/artemnovichkov/iOS-11-by-Examples. I guess you have a problem with different coordinate systems. Pay attention to rect converting:

cameraLayer.metadataOutputRectConverted(fromLayerRect: originalRect)

and

cameraLayer.layerRectConverted(fromMetadataOutputRect: transformedRect)

answered Jun 20 '17 at 16:43

Artem Novichkov

2,356
2
24
34

Thanks Artem, you are great ! :) – kxko Jul 02 '17 at 14:33
Is it possible to detect object in an image and track it into another set of images? – The iCoder Sep 20 '17 at 06:51
1

Unfortunately, no. – Artem Novichkov Sep 20 '17 at 07:04
1

@ArtemNovichkov Thanks for reply. In other way can we implement to local video as I am not able to find method for converting rectangle of interest to specific coordinate space. – The iCoder Sep 21 '17 at 06:18
@TheiCoder do you find any examples of object tracking on local videos ? – srjohnhuang Jul 18 '18 at 09:37
@srjohnhuang No, But I used dlib library for object detection from image. You can try with dlib for video. – The iCoder Jul 18 '18 at 10:12

How can i use the Object tracking API of vision framework on ios11?

2 Answers2