QR Reader with VNDetectBarcodeRequest

Question

I've setup an AVCaptureSession with a video data output and am attempting to use iOS 11's Vision framework to read QR codes. The camera is setup like basically any AVCaptureSession is. I will abbreviate and just show setting up the output.

let output = AVCaptureVideoDataOutput()
output.setSampleBufferDelegate(self, queue: captureQueue)
captureSession.addOutput(output)

// I did this to get the CVPixelBuffer to be oriented in portrait.
// I don't know if it's needed and I'm not sure it matters anyway.
output.connection(with: .video)!.videoOrientation = .portrait

So the camera is up and running as always. Here is the code I am using to perform a VNImageRequestHandler for QR codes.

func captureOutput(_ output: AVCaptureOutput, didOutput sampleBuffer: CMSampleBuffer, from connection: AVCaptureConnection) {
    guard let pixelBuffer = CMSampleBufferGetImageBuffer(sampleBuffer) else { return }
    let imageRequestHandler = VNImageRequestHandler(cvPixelBuffer: pixelBuffer, orientation: .up, options: [:])

    let qrRequest = VNDetectBarcodesRequest { request, error in
        let barcodeObservations = request.results as? [VNBarcodeObservation]
        guard let qrCode = barcodeObservations?.flatMap({ $0.barcodeDescriptor as? CIQRCodeDescriptor }).first else { return }
        if let code = String(data: qrCode.errorCorrectedPayload, encoding: .isoLatin1) {
            debugPrint(code)
        }
    }

    qrRequest.symbologies = [.QR]
    try! imageRequestHandler.perform([qrRequest])
}

I am using a QR code that encodes http://www.google.com as a test. The debugPrint line prints out:

AVGG\u{03}¢ò÷wwrævöövÆRæ6öÐì\u{11}ì

I have tested this same QR code with the AVCaptureMetadataOutput that has been around for a while and that method decodes the QR code correctly. So my question is, what have I missed to get the output that I am getting?

(Obviously I could just use the AVCaptureMetadataOutput as a solution, because I can see that it works. But that doesn't help me learn how to use the Vision framework.)

score 0 · Answer 1 · answered Jul 28 '17 at 18:49

0

Most likely the problem is here:

if let code = String(data: qrCode.errorCorrectedPayload, encoding: .isoLatin1)

Try to use .utf8.

Also i would suggest to look at the raw output of the 'errorCorrectedPayload' without encoding. Maybe it already has correct encoding.

answered Jul 28 '17 at 18:49

Nikita Gaidukov

771
4
14

I went with `.isoLatin1` because that is the standard for QR codes ([reference](https://www.ibm.com/support/knowledgecenter/en/SSBJG3_2.5.0/com.ibm.gen_studug.doc/c_grd_barcodes_qr_code.htm)). I will look into other encodings to see if helps. I'll also see if I can make any sense of the raw bytes. Thanks for the suggestion. – keithbhunter Jul 28 '17 at 18:58
None of the other encodings I tried were any better. Some returned `nil` and some returned the same scrambled text. – keithbhunter Jul 28 '17 at 19:32
After some testing, i think errorCorrectedPayload is not the QR data itself. In the source file of the CIQRCodeDescriptor it says that errorCorrectedPayload is - "The error-corrected codewords that comprise the QR code symbol." Which is not very clear. – Nikita Gaidukov Jul 30 '17 at 19:02

score 0 · Answer 2 · answered Jul 30 '17 at 17:33

The definition of errorCorrectedPayload says: -- QR Codes are formally specified in ISO/IEC 18004:2006(E). Section 6.4.10 "Bitstream to codeword conversion" specifies the set of 8-bit codewords in the symbol immediately prior to splitting the message into blocks and applying error correction. --

score 0 · Answer 3 · answered Sep 06 '19 at 18:01

0

This seems to work fine with VNBarcodeObservation.payloadStringValue instead of transforming VNBarcodeObservation.barcodeDescriptor.

answered Sep 06 '19 at 18:01

Florent Morin

842
1
10
13

QR Reader with VNDetectBarcodeRequest

3 Answers3