1

I am trying to implement a feature that will use VoiceOver to read a selected piece of HTML displayed in a WKWebView.

My initial attempt flattened the html to text and then used AVSpeechSynthesizer() to read an AVSpeechUtterance instantiated with the text:

let synth = AVSpeechSynthesizer()
let utterance = AVSpeechUtterance(string: myFlattenedText)
if synth.isSpeaking {
    synth.stopSpeaking(at: .immediate)
}
synth.speak(utterance)

This works generally, but the experience that VoiceOver gives is superior. For example, my HTML has accessibility-enabled MathJax. VoiceOver does a nice job reading the contextual math equations while the strategy of flattening the html to text does not.

I experimented with creating a NSAttributedString with the html fragment as detailed here, but that doesn't do any better.

Is there a way to tap into the VoiceOver functionality, perhaps using UIAccessibility somehow, that will read the accessibility-enabled HTML in the same way that VoiceOver does?

Jon Brooks
  • 2,472
  • 24
  • 32
  • I _strongly_ suspect that you will only get the experience you want by embedding a web view. – Justin Nov 15 '16 at 16:28
  • I am using a webview. When the user selects content in the webview, I want to read the content to the user. I could use either a js or a native solution here, if it makes a difference. I couldn't get anywhere with a js solution, though. – Jon Brooks Nov 15 '16 at 16:38
  • I must have misunderstood. Why can't users navigate the web view using VoiceOver? – Justin Nov 15 '16 at 18:47
  • 1
    Using VoiceOver works well. Our app also allows users to select something and read it via a menu command. I'm not crazy about the feature, but I don't make those calls :) – Jon Brooks Nov 16 '16 at 18:36

1 Answers1

0

What you're seeing is WebKit's MathJax support paired with carefully considered UIAccessibility support. While the most recent documentation I found suggests that these properties are not accessible to web developers, as an iOS developer, you should be able to traverse the UIAccessibility element hierarchy to dump the accessibilityLabel and accessibilityValue of your MathJax content.

Edit: As @mattsven points out, the issue is that WKWebView content is rendered in a remote process and thus does not expose its accessibility elements to the containing app.

Justin
  • 20,509
  • 6
  • 47
  • 58
  • Thanks for the suggestion and the link. I thought I was on a path to a solution (making a temporary webview with the html from selection), but I can't seem to get any any accessibility properties from either this temporary webview or my original webview. `accessibilityLabel`, `accessibilityValue`, and `accessibilityElements` are all nil. – Jon Brooks Nov 17 '16 at 16:29
  • @JonBrooks Have you recursed through `accessibilityChildren`? – Justin Nov 17 '16 at 16:31
  • I only see a `accessibilityChildren` property in NSAccessibility not UIAccessibility. Is it a property on something other than the view? – Jon Brooks Nov 17 '16 at 17:05
  • @JonBrooks Whoops. I meant to suggest trying [`accessibilityElements`](https://developer.apple.com/reference/objectivec/nsobject/1615147-accessibilityelements), which it appears you did. Let me experiment, some. – Justin Nov 17 '16 at 20:13
  • 1
    Update for the curious: `WKWebView` renders via a remote process, and [does not expose](https://github.com/kif-framework/KIF/issues/460) its accessibility elements in a way that can be accessed via your code. – mattsven Feb 27 '19 at 12:37