Why my Object Detection app is not returning anything?

Question

I have a simple app that contains a button, UIImageView, and a label. Once you click on the button, you will be able to take a photo using the camera. Then the model has to predict what is the object in the photo, and finally the label has to display the output (the predicted object).

Everything is working fine and there are no errors, but after I take a picture, the label is not being changed, the model is not returning anything, why is that ?

NOTE: The model is working fine and it has been tested using another app, but I think I am missing something in this code.

Here is my code:

import UIKit
import CoreML

class secondViewController: UIViewController, UINavigationControllerDelegate {

    @IBOutlet weak var imageView: UIImageView!
    @IBOutlet weak var classifier: UILabel!
    var model: VGG16!
    let cameraPicker = UIImagePickerController()

    override func viewDidLoad() {
        super.viewDidLoad()
        // Do any additional setup after loading the view, typically from a nib.
    }

    override func viewWillAppear(_ animated: Bool) {
        model = VGG16()
    }

    override func didReceiveMemoryWarning() {
        super.didReceiveMemoryWarning()
        // Dispose of any resources that can be recreated.
    }


    @IBAction func camera(_ sender: Any) {
        if !UIImagePickerController.isSourceTypeAvailable(.camera) {
            return
        }

        cameraPicker.delegate = self
        cameraPicker.sourceType = .camera
        cameraPicker.allowsEditing = false
        present(cameraPicker, animated: true)
    }


}



extension secondViewController: UIImagePickerControllerDelegate {
    func imagePickerControllerDidCancel(_ picker: UIImagePickerController) {
        dismiss(animated: true, completion: nil)
    }

//    private func imagePickerController(_ picker: UIImagePickerController, didFinishPickingMediaWithInfo info: [String : Any]) {


    private func imagePickerController(_ picker: UIImagePickerController, didFinishPickingMediaWithInfo info: [UIImagePickerController.InfoKey: Any]) {
        picker.dismiss(animated: true, completion: nil)
        let image = info[UIImagePickerController.InfoKey.originalImage]! as! UIImage

        picker.dismiss(animated: true)
        classifier.text = "Analyzing Image..."

        UIGraphicsBeginImageContextWithOptions(CGSize(width: 299, height: 299), true, 2.0)
        image.draw(in: CGRect(x: 0, y: 0, width: 299, height: 299))
        let newImage = UIGraphicsGetImageFromCurrentImageContext()!
        UIGraphicsEndImageContext()

        let attrs = [kCVPixelBufferCGImageCompatibilityKey: kCFBooleanTrue, kCVPixelBufferCGBitmapContextCompatibilityKey: kCFBooleanTrue] as CFDictionary
        var pixelBuffer : CVPixelBuffer?
        let status = CVPixelBufferCreate(kCFAllocatorDefault, Int(newImage.size.width), Int(newImage.size.height), kCVPixelFormatType_32ARGB, attrs, &pixelBuffer)
        guard (status == kCVReturnSuccess) else {
            return
        }

        CVPixelBufferLockBaseAddress(pixelBuffer!, CVPixelBufferLockFlags(rawValue: 0))
        let pixelData = CVPixelBufferGetBaseAddress(pixelBuffer!)

        let rgbColorSpace = CGColorSpaceCreateDeviceRGB()
        let context = CGContext(data: pixelData, width: Int(newImage.size.width), height: Int(newImage.size.height), bitsPerComponent: 8, bytesPerRow: CVPixelBufferGetBytesPerRow(pixelBuffer!), space: rgbColorSpace, bitmapInfo: CGImageAlphaInfo.noneSkipFirst.rawValue) //3

        context?.translateBy(x: 0, y: newImage.size.height)
        context?.scaleBy(x: 1.0, y: -1.0)

        UIGraphicsPushContext(context!)
        newImage.draw(in: CGRect(x: 0, y: 0, width: newImage.size.width, height: newImage.size.height))
        UIGraphicsPopContext()
        CVPixelBufferUnlockBaseAddress(pixelBuffer!, CVPixelBufferLockFlags(rawValue: 0))
        imageView.image = newImage

        // Core ML
        guard let prediction = try? model.prediction(image: pixelBuffer!) else {
            return
        }

        classifier.text = "I think this is a \(prediction.classLabel)."
    }
}

Use the debugger. Set breakpoints and don't ignore errors with `try?`, `catch` and handle them — vadian, Mar 01 '19 at 18:25
It looks like your problem is that classifier.text is not being set on the main thread. You could use GCD to run that line on the main thread, or, alternatively, you could add classifier.setNeedsDisplay(). — Tim Kokesh, Mar 01 '19 at 19:21

Shehata Gamal · Answer 1 · 2019-03-01T18:32:19.080

1

You need to hold a strong reference to

let cameraPicker = UIImagePickerController()
override func viewDidLoad() {

by making it an instance variable for the delegate methods to be called

edited Mar 01 '19 at 18:32

answered Mar 01 '19 at 18:27

Shehata Gamal

98,760
8
65
87

Thank you, but I already have this in my code. It is in `@IBAction func camera`. So I did not really understand if I should change it – ananya Mar 01 '19 at 18:30
change it's place as above – Shehata Gamal Mar 01 '19 at 18:32
Thank you so much for your help. But it did not work. The label did not change – ananya Mar 01 '19 at 18:36
which swift version is this ? – Shehata Gamal Mar 01 '19 at 18:38
This is Swift 4.2 – ananya Mar 01 '19 at 18:40
make sure you use the latest delegate method https://stackoverflow.com/a/52721886/5820010 – Shehata Gamal Mar 01 '19 at 18:41
@ananya You seem to have an IBAction, but no strong reference (such as IBOutlet) for the image picker controller. – Chris Mar 01 '19 at 19:02
Thank you @Chris, but I am not really sure what does that mean and how to fix it! – ananya Mar 01 '19 at 19:07
@Sh_Khan Thanks. I appreciate your help, but even after I changed my delegate method, the label still not changing :( – ananya Mar 01 '19 at 19:09
@Sh_Khan Yes sir I did, and I also changed my delegate function to the one you sent me – ananya Mar 01 '19 at 19:12
inside every guard return add a print statement to verify when control ends to know why the code doesn't reach the target line – Shehata Gamal Mar 01 '19 at 19:15
@Sh_Khan Did not reach to any. None of the statements were printed – ananya Mar 01 '19 at 19:19
write after `let image = info[UIImagePickerController.InfoKey.originalImage]! as! UIImage` a print statement / breakpoint – Shehata Gamal Mar 01 '19 at 19:21
Nothing printed. Did not reach there. Tested the function. Loading the model working, the button was clicked, then nothing – ananya Mar 01 '19 at 19:24

Why my Object Detection app is not returning anything?

1 Answers1