7

So Google has just released a new feature for Google Now called Now on Tap. It enables a user to get contextual help from different apps (The Verge video example).

  • Does it simply take a screenshot of the current app and use OCR to extract the text?
  • Does it have "special Google permission" which allows it to read the values within the views?
Gil Julio
  • 812
  • 1
  • 9
  • 18
  • 1
    Disclaimer: speaking only as myself... I don't know for certain, but Google generally avoids "special" capabilities in Android and typically makes the APIs it uses the same as those consumable by other apps. My guess is that it uses the accessibility APIs that exist in Android. With those APIs, it is possible to read the content of text views as well as the accessibility description of any image that is on the screen. – Michael Aaron Safyan May 31 '15 at 11:29

1 Answers1

2

Primarily, it should work using the assist API added to the M Developer Preview. However as "Now on Tap" is proprietary and unreleased, we can only really guess at this time.

CommonsWare
  • 986,068
  • 189
  • 2,389
  • 2,491