Action Extension, getting images in Instant Markup returns a blank screen

Question

Background I have an Action Extension which can be used with images:

class ActionViewController: UIViewController {

    @IBOutlet weak var imageView: UIImageView!

    override func viewDidLoad() {
        super.viewDidLoad()
        var imageFound = false
        for item in self.extensionContext!.inputItems as! [NSExtensionItem] {
            for provider in item.attachments! {
                if provider.hasItemConformingToTypeIdentifier(kUTTypeImage as String) {
                    weak var weakImageView = self.imageView
                    provider.loadItem(forTypeIdentifier: kUTTypeImage as String, options: nil, completionHandler: { (imageURL, error) in
                        OperationQueue.main.addOperation {
                            if let strongImageView = weakImageView {
                                if let imageURL = imageURL as? URL {
                                    strongImageView.image = UIImage(data: try! Data(contentsOf: imageURL))
                                }
                            }
                        }
                    })

                    imageFound = true
                    break
                }
            }
            if (imageFound) {
                break
            }
        }
    }

The above code is the default code when you create an Action Extension. The above code executes properly if I am in the Photos app > Select an image > Share > My Action Extension. The image is then displayed in a UIImageView in my Action Extension. However when I take a screenshot and press the share button (see below image for example), sharing the image with my Action Extension returns a blank screen with no errors in the console. This similarly happens with other apps. I noticed that it is possible to share from the Instant Markup tool, since I am able to share an image with Gmail.

The following is my Info.plist:

<key>NSExtensionActivationRule</key>
    <string>SUBQUERY (
        extensionItems,
        $extensionItem,
        SUBQUERY (
            $extensionItem.attachments,
            $attachment,
            ANY $attachment.registeredTypeIdentifiers UTI-CONFORMS-TO "public.item" ||
            ANY $attachment.registeredTypeIdentifiers UTI-CONFORMS-TO "public.content"
            ).@count == $extensionItem.attachments.@count
            ).@count > 0
    </string>

The problem: I am not entirely sure how to fetch the image from Instant Markup tool (built in iOS screenshot tool).

Answer 1

I figured it out using the following link: ios swift share-extension: what are all and the best ways to handle images?

What was the issue?

The problem was, because when you load images from say Photos the image is of type URL, while loading from Screenshot tool the image is of UIImage type. It was therefore necessary to handle the data loading explicitly for the different cases.

This is how I have implemented it:

if let content = extensionContext!.inputItems[0] as? NSExtensionItem {
    let contentType = kUTTypeImage as String
    if (content.attachments) != nil {
        for attachment in content.attachments! {
            if attachment.hasItemConformingToTypeIdentifier(contentType as String) {
                attachment.loadItem(forTypeIdentifier: contentType as String, options: nil) { data, error in
                    if error == nil {
                        var contentData: Data? = nil

                        // Handling if image data is raw data.
                        if let data = data as? Data {
                            contentData = data

                            // Handling if image data is an URL.
                        } else if let url = data as? URL {
                            contentData = try? Data(contentsOf: url)
                        }

                            // Handling if image data is an UIImage object, for example from the Screenshot tool.
                        else if let imageData = data as? UIImage {
                            contentData = imageData.pngData()
                        }

                        // Proceed here with contentData.
                        self.imageView.image = UIImage(data: contentData!)
                    }
                }
            }
        }
    }
}

DONE T̶O̶D̶O̶ C̶u̶r̶r̶e̶n̶t̶l̶y̶ ̶t̶h̶e̶r̶e̶ ̶i̶s̶ ̶a̶ ̶s̶i̶g̶n̶i̶f̶i̶c̶a̶n̶t̶ ̶d̶e̶l̶a̶y̶ ̶w̶h̶e̶n̶ ̶l̶o̶a̶d̶i̶n̶g̶ ̶a̶n̶ ̶i̶m̶a̶g̶e̶ ̶f̶r̶o̶m̶ ̶S̶c̶r̶e̶e̶n̶s̶h̶o̶t̶ ̶t̶o̶o̶l̶ ̶i̶n̶t̶o̶ ̶m̶y̶ ̶e̶x̶t̶e̶n̶s̶i̶o̶n̶.̶ ̶H̶o̶w̶e̶v̶e̶r̶,̶ ̶t̶h̶i̶s̶ ̶i̶s̶s̶u̶e̶ ̶i̶s̶ ̶o̶u̶t̶ ̶o̶f̶ ̶c̶o̶n̶t̶e̶x̶t̶ ̶o̶f̶ ̶w̶h̶a̶t̶ ̶I̶ ̶o̶r̶i̶g̶i̶n̶a̶l̶l̶y̶ ̶a̶s̶k̶e̶d̶ ̶s̶o̶ ̶I̶ ̶w̶i̶l̶l̶ ̶m̶a̶r̶k̶ ̶t̶h̶i̶s̶ ̶q̶u̶e̶s̶t̶i̶o̶n̶ ̶a̶s̶ ̶s̶o̶l̶v̶e̶d̶.̶ ̶I̶f̶ ̶I̶ ̶f̶i̶n̶d̶ ̶a̶ ̶s̶o̶l̶u̶t̶i̶o̶n̶ ̶t̶o̶ ̶t̶h̶e̶ ̶s̶l̶o̶w̶n̶e̶s̶s̶ ̶I̶ ̶c̶a̶n̶ ̶m̶a̶y̶b̶e̶ ̶a̶d̶d̶ ̶i̶t̶ ̶h̶e̶r̶e̶ ̶a̶s̶ ̶a̶ ̶c̶o̶m̶m̶e̶n̶t̶.̶

The delay can be solved by putting the image loading onto a different thread (as I understand it):

var imageFound = false
for item in self.extensionContext!.inputItems as! [NSExtensionItem] {
    for provider in item.attachments! {
        if provider.hasItemConformingToTypeIdentifier(kUTTypeData as String) {
            weak var weakImageView = self.imageView
            provider.loadItem(forTypeIdentifier: kUTTypeData as String, options: nil, completionHandler: { (data, error) in
                DispatchQueue.main.async {
                    if let strongImageView = weakImageView {
                        var contentData: Data? = nil
                        // Handling if image data is raw data.
                        if let data = data as? Data {
                            contentData = data
                            strongImageView.image = UIImage(data: contentData!)
                        // Handling if image data is an URL.
                        } else if let url = data as? URL {
                            contentData = try? Data(contentsOf: url)
                            strongImageView.image = UIImage(data: contentData!)
                        }
                        // Handling if image data is an UIImage object, for example from the Screenshot tool.
                        else if let imageData = data as? UIImage {
                            contentData = imageData.pngData()
                            strongImageView.image = UIImage(data: contentData!)
                        }
                    }
                }
            })
            imageFound = true
            break
        }
    }
    if (imageFound) {
        break
    }
}