3

I have difficulties understanding what is the right way to make a custom UIView accessible on iOS. As a case study, here is a custom UIControl subclass wrapping a UILabel and a UITextField subviews, that I want to test via a UI test.

View hierarchy:

→ UIControl
  ↪︎ UILabel
  ↪︎ UIView

Intended behavior:

Snapshot of expected behavior


1️⃣ The default behavior

By default, the behavior is not great:

  • The accessibility inspector inspects the two subviews as unrelated;
  • The accessibility inspector’s audit raises a “hit area is too small” warning since it is ignoring the whole UIControl can be tapped and will give focus to the UITextField.

But this testing code works fine:

let emailField = app.textFields["Email"]
emailField.tap()
emailField.typeText("toto@tata.com")

2️⃣ Becoming an accessibility element

If the UIControl becomes flagged as an accessibility element as following, the accessibility inspector will show only 1 pointer for the whole UIControl but the UI test will fail.

Here is the code:

isAccessibilityElement = true
accessibilityLabel = innerLabel.text
accessibilityValue = innerTextField.text
accessibilityTraits = innerTextField.accessibilityTraits

And the test failure stack, that seems to have an Element subtree limited to the UIControl (aka TextField) itself

UI Test Activity: 
Assertion Failure: SampleUITests.swift:21: Failed to synthesize event: Neither element nor any descendant has keyboard focus. Event dispatch snapshot: TextField, label: 'Email'
Element debug description:
Attributes: TextField, {{32.0, 229.0}, {350.0, 52.0}}, label: 'Email'
Element subtree:
 →TextField, 0x600000925340, {{32.0, 229.0}, {350.0, 52.0}}, label: 'Email'

3️⃣ Simplifying accessibility information

Inspired by Apple’s documentation on how to simplify your accessibility information , I used the following code:

var elements = [UIAccessibilityElement]()
let groupedElements = UIAccessibilityElement(accessibilityContainer: self)
groupedElements.accessibilityLabel = innerLabel.text
groupedElements.accessibilityTraits = innerTextField.accessibilityTraits
elements.append(groupedElements)

Which seems to do nothing: I’m back to the default behavior.

Wrapping things up

I’d like to have the accessibility structure from 2️⃣ and still be able to run UI tests in the most expressive way ie using the same code as 1️⃣, how can I do this? What did I do wrong in 3️⃣ that made it behave the same way as 1️⃣?


Edits

  1. As hinted by @Defragged, I tried to improve the UIControls compliance to UIResponder but it didn't really help:

    override var canBecomeFirstResponder: Bool {
        return innerTextField.canBecomeFirstResponder
    }
    
    override func becomeFirstResponder() -> Bool {
        return innerTextField.becomeFirstResponder()
    }
    
    override var isFirstResponder: Bool {
        return innerTextField.isFirstResponder
    }
    
  2. Scrutinizing the UI test error logs a little deeper, I read this:

    t =     9.65s         Synthesize event
    t =     9.68s             Get number of matches for: Elements containing elements matching predicate 'hasKeyboardFocus == 1'
    t =     9.72s                 Requesting snapshot of accessibility hierarchy for app with pid 24879
    t =     9.74s                 Find: Descendants matching type TextField
    t =     9.74s                 Find: Elements matching predicate '"Email" IN identifiers'
    t =     9.74s                 Find: Elements containing elements matching predicate 'hasKeyboardFocus == 1'
    

    It seems that XCUIElement instances have a undocumented hasKeyboardFocus attribute that can be inspected with a debugger but that I have no clue how to control ‍♂️.

Mick F
  • 7,312
  • 6
  • 51
  • 98
  • 1
    I think you're on the right track with "2", but without seeing your custom `UIControl` implementation, it's hard to see exactly what's going wrong. Is your custom `UIControl` doing its responder chain stuff correctly (as in, does it implement `canBecomeFirstResponder`, `isFirstResponder` etc? I imagine your implementations would just want to forward to the inner text field. I suspect your test is tapping inside of the main control, but outside of the inner field, so it doesn't get focus, so then the test's keyboard input then doesn't work. – Defragged Oct 15 '19 at 16:02
  • Is the keyboard up on screen when the test fails in 2? – Oletha Oct 16 '19 at 07:15
  • @Defragged I actually don't, I have a simple target handler for `.touchUpInside` events that basically calls `innerTextField.becomeFirstResponder()`, which is enough to put the keyboard up on screen (cc @Oletha) – Mick F Oct 16 '19 at 09:16
  • I added an edit section to the question to give updates on new attempts. No luck though. – Mick F Oct 16 '19 at 15:38
  • Thanks for the update. Given the error you've posted, which of your controls has the `accessibilityIdentifier` of `Email`? Is it the inner `UITextField`, or the outer `UIControl`? – Defragged Oct 17 '19 at 14:31
  • @Defragged it is the inner `UITextField`. I tried giving this identifier to the `UIControl`: no behavior change. – Mick F Oct 21 '19 at 08:14
  • 1
    I was thinking this morning about another approach that might work: If you make the outer component not accessible at all, but have only the inner component be accessible, when you set the label on the outer control, you can update the accessibility label on the inner field to have the new value ("email" or whatever). Also, whenever the frame of the outer control changes, update the accessibility frame of the inner field with the frame out of the outer control. Now as far as accessibility knows, there's only one control, but with the combined attributes of both that you're interested in. – Defragged Oct 22 '19 at 10:14
  • I've put up together a tiny project to show up my issues here: https://github.com/dirtyhenry/aristide. – Mick F Nov 04 '19 at 16:05
  • @Defragged not sure how easy it is to play with the accessibility frame with a auto layout lifecycle but will report results here when I do. – Mick F Nov 04 '19 at 16:06
  • It should be safe. Autolayout is essentially just computing frames from your constraints anyway, and the accessibility frameworks don't give you any other mechanism. Since you're just taking one object's `accessibilityFrame` and applying it to something else, you shouldn't need to compute anything yourself. Just make sure you're updating it at the correct times. If you are computing the frame yourself, just be aware that `accessibilityFrame` is in screen coordinates. – Defragged Nov 04 '19 at 16:21

0 Answers0