Scrub SeekBar with Accessibility Services

Question

I'm trying to scrub a seekbar of a 3rd party app with Accessibility Services. This is what I'm using to scrub.

 val arguments = Bundle()
 arguments.putFloat(AccessibilityNodeInfo.ACTION_ARGUMENT_PROGRESS_VALUE, 50.0.toFloat())
 seekBarNode?.performAction(AccessibilityNodeInfo.AccessibilityAction.ACTION_SET_PROGRESS.id, arguments)

Since I'm scrubbing a video, the SeekBar position changes but the content does not change.

Does anyone know what the issue is here? Or are there any alternatives to scrub a 3rd party SeekBar with accessibility services?

Also, I've read about GestureDescription to perform swipes. But I don't know how to use that to perform SeekBar scrub.

score 0 · Answer 1 · answered Apr 10 '20 at 10:28

Try and click the center of the seek bar with dispatchGesture:

fun AccessibilityService.tapCenterOfNode(node: AccessibilityNodeInfo, onDone: (Boolean) -> Any){
    this.dispatchPath(
        drawPath = pathOnPoint(node.centerInScreen()),
        pathDuration = 10,
        onDone = { 
            success -> Log.d("dispatch", "success? $success")
        }
    )
}

fun AccessibilityService.dispatchPath(drawPath: Path, pathDuration: Long, onDone: (Boolean) -> Any) {
    val stroke = GestureDescription.StrokeDescription(drawPath, 0, pathDuration)
    val gesture = GestureDescription.Builder().addStroke(stroke).build()

    this.dispatchGesture(gesture, object : AccessibilityService.GestureResultCallback() {
        override fun onCompleted(gestureDescription: GestureDescription) {
            super.onCompleted(gestureDescription)
            onDone(true)
        }

        override fun onCancelled(gestureDescription: GestureDescription) {
            super.onCancelled(gestureDescription)
            onDone(false)
        }
    }, null)
}

fun AccessibilityNodeInfo.centerInScreen(): Pair<Float, Float> = 
    Pair(this.getBoundsInScreen().exactCenterX(), this.getBoundsInScreen().exactCenterY())

fun AccessibilityNodeInfo.getBoundsInScreen(): Rect {
    val rect = Rect()
    getBoundsInScreen(rect)
    return rect
}

fun pathOnPoint(point: Pair<Float, Float>) = Path().apply {
    val (x, y) = point
    moveTo(x, y)
    lineTo(x, y)
}

But tapping to the center of the seekbar would again seek to that center position right? i.e suppose I've set my seek progress to 20% and tapping the seekbar at center would disrupt the actual intention. Please correct me if I'm wrong — Veeresh Charantimath, Apr 10 '20 at 10:44
I believe `performAction` won't work on the seekbar (https://stackoverflow.com/questions/22765040/performaction-action-set-selection-not-working-properly), thus, I think the general dispatchGesture should work. I suggest you first try and run this piece of code to see if it changes the seekbar as well as the **Content** (which is the question in hand, no?). To find the "new" middle of the seekbar you could either try and use node's `getRangeInfo`, or, listen to accessibility events data that tells the progress of it. — Sean, Apr 10 '20 at 11:04
Yes, the center click works. As you mention the node's getRangeInfo has information regarding the current seek position. Is it possible to use that value to somehow sync another seekbar from the same app to use that value? — Veeresh Charantimath, Apr 10 '20 at 11:53

Scrub SeekBar with Accessibility Services

1 Answers1