Custom built On Screen Keyboard in Adobe AIR for oAuth authentication

Question

I am in the process of building a touch screen photobooth where users will take pictures, add graphics, etc in Adobe AIR. At the end of the process, they will get to email themselves the picture, share the picture on Facebook, and share it on Twitter.

I am using this tutorial: http://tv.adobe.com/watch/adc-presents/oauth-in-adobe-air-applications-built-with-flash-or-flex/ to figure out how to get oAuth to work in AIR. But it seems simple. Just load the authorization request URL into AIR's equivalent of an iFrame.

The tricky part is that there will be no physical keyboard on site. For all text input, we designed an onscreen keyboard. Programming an on screen keyboard in AS3 is pretty easy. But we don't have access to the text fields within the AIR HTML/Browser wrapper so we can't manually add characters to the input strings...

Is there any workaround we can implement for this? Adding a physical keyboard is unfortunately out of the question.

Answer 1

I realize this is 5 months old, but I was working on something similar for a client this evening and ran across your question. I also wanted to omit the need for a system level onscreen keyboard, or a physical real world keyboard, just wanted to use a Flash/AIR based keyboard completely under control of the app via the attached touch screen.

I came up with a hacky way to do this all in Flash/AIR and it works for any oAuth dialog (with some caveats). I've got it working as a test for Facebook and Twitter oAuth permission pages just this evening. It requires that you use a StageWebView for the display of the oAuth dialog and some JavaScript injection via loadUrl method of StageWebView. Using the stageWebView, setting up a listener for the Complete event on the load of the oauth permission page in your StageWebView is the first step.

Next step, is once its loaded and the complete event fires, in your complete handler, you can then inject javascript directly into the page via the StageWebView reference, and then via that function set up communication to populate the fields dynamically from your AS3 on screen keyboard or other function. For example, to populate the username field in Twitters oAuth permission page, you could do this:

webView.loadURL("javascript: form = document.forms[0]; form['username_or_email'].value='your_username';");

That populates the username field in Twitters oAuth permission page.

So you could inject a function into the page, which you could then call from ActionScript to pass in key presses from your virtual onscreen keyboard, or completed strings, to populate the proper fields. That is the only tricky part, identifying the form and field names in the source of the oAuth pages ahead of time. So far Twitter and Facebook look pretty straightforward to identify, but of course could break at anytime, a little regexp could help make sure they are correct. Only problem I see here beyond that would be if the field names changes dynamically for security purposes, but from a quick glance at twitter and facebook at least they don't appear to do that. Other oAuth pages may or may not, YMMV.

There is a useful library for working with StageWebView that might be helpful in doing some of this communication - take a look here: http://code.google.com/p/stagewebviewbridge/wiki/Communication

In any case, that is the overall approach, leverage StageWebView, and then dynamically inject your own JavaScript function to handle passing in the value to the fields from your onscreen keyboard, or simply populate the fields directly with code similar to the above. Hope this helps anyone out there attempting something similar.

Wanted to add that I came up with this approach because it was/is a requirement to allow a user to do this directly on the exhibit I am working on, but my personal preference would be to simply collect a users email address, or phone number, and have the interactive send them an email or SMS message to their own device, which they could then easily share more safely and securely. Another solution would be to display a QR code, a user could snap and quickly resolve to share more securely - I prefer those approaches, but the above will do the trick for users willing to enter in their credentials in a public place. Make sure you have time outs on the session and properly logout users credentials for subsequent users, otherwise bad things are guaranteed to occur.