I am using Vision and a CoreML Model whose role is prediction rather than classification or image-to-image processing. I input the pixelBuffer and get back a [VNCoreMLFeatureValueObservation]1.
func createCoreMLRequest() -> VNCoreMLRequest? {
guard let model = try? VNCoreMLModel(for: yolo.model.model) else { return nil }
let request = VNCoreMLRequest(model: model) { (finReq, err) in
if let results = finReq.results as? [VNCoreMLFeatureValueObservation] {
let features = results.compactMap({ (obs) -> MLMultiArray? in
obs.featureValue.multiArrayValue
})
let boundingBoxes = self.yolo.computeBoundingBoxes(features: features)
guard let prediction = boundingBoxes.first else { return }
self.observation = VNDetectedObjectObservation(boundingBox: prediction.rect)
DispatchQueue.main.async {
self.highlightView.frame = prediction.rect
}
}
}
return request
}
I then fetch the [MLMultiArray]2 from the Observation and feed it to YOLO.swift to compute the bounding boxes. The bounding boxes are an array of predictions. The predictions is a struct containing a rect(boundingBox), className and confidence.
Using the predictions I want to track the objects. So I use Vision and and create a [VNDetectedObjectObservation]3 with the prediction's boundingBox. Then I create a [VNTrackObjectRequest]4 and pass in the detected observation but the resulting observation always returns the same boundingBox as the initial prediction.
func createTrackRequest() -> VNTrackObjectRequest {
let trackRequest = VNTrackObjectRequest(detectedObjectObservation: self.observation) { (finReq, err) in
if let results = finReq.results as? [VNDetectedObjectObservation] {
if let observation = results.first {
self.observation = observation
DispatchQueue.main.async {
self.highlightView.frame = observation.boundingBox
}
}
}
}
return trackRequest
}
I am not sure why this is happening. Any suggestions?
