3

I have an idea for an app that I'd like to develop, but before I invest a lot of time learning objective C and the iOS APIs, I'd like to make sure that what I want to do is feasible.

The app I want to make is a purely auditory (sound-only) version of Google Glass. I'm visually impaired, so spending a lot of money on something visual, even though it can read content to you, would not be worth it. But if I could use an iPhone to give many of the same options as Google Glass, that would be great.

Many times, I've wanted a piece of information while walking down the street, where I couldn't easily get to my iPhone, because I have my cane in one hand, and something else in the other. In such cases, it'd be awesome if I could say a command, and get a voice response.

I'd use the microphone built into the Apple earphones for audio input, but I'm not sure if it's possible to listen for audio input while the screen is locked. I'm certain it's not possible with a non-jailbroken iPhone.

Can anyone can tell me if this is possible?

Nate
  • 31,017
  • 13
  • 83
  • 207
bigblind
  • 12,539
  • 14
  • 68
  • 123

1 Answers1

2

Yes, you can do this.

In order to keep your app running all the time, even when the iPhone is locked, you could build a Launch Daemon. A launch daemon can start when the phone does, and is not subject to the restrictions that iOS puts on sandboxed apps, installed to /var/mobile/Applications/.

You do need to have a jailbroken device to take advantage of Launch Daemons. Here is a good tutorial on building one.

Launch Daemons are also a normal part of OS X, so if you need more information, you might try consulting the OS X docs online. Most aspects of Launch Daemons work the same way on a jailbroken iPhone.

You'll also want to be able to detect certain events, to activate your app. You certainly don't want to be processing an audio stream at all times (maybe you only activate the app when you start walking with your cane). To detect events, like a home button press (or however you want to activate your code), I would take a look at RPetrich's libactivator library.

Nate
  • 31,017
  • 13
  • 83
  • 207
  • Thanks a lot. If you have any ideas for an activation method, that'd be awesome. I want to avoid having to reach the iphone, because I'd like to be able to use it hands-free. However, I don't want to have conflicts with audio apps by using the headphone buttons for that, unless I could define my own key combinations on there, like pressing both volume buttons at the same time. – bigblind May 27 '13 at 20:21
  • It sounds like, when you're walking, you want the app to be listening for input at all times. Is that right? If that's true, then you probably are going to need to be processing the audio stream for that whole time. What I was suggesting is that you don't let the app listen for audio 24/7. You'll know when you're about to stand up, grab the cane, and have your hands tied up for a few minutes. So, I was suggesting *activating* the sound listener then. In that situation, would it still be a problem to use hands (just to start the listener)? The daemon would still be running at all times. – Nate May 27 '13 at 20:24
  • So, essentially, the daemon is always on. But, only when you double-press Home (or some other action), would it start listening to the audio feed, which is what consumes lots of battery. The rest of the time, it's sitting idly, waiting to be called upon. LibActivator supports lots of different activation methods, not just home button presses. Volume/lock keys, swipes, everything. Of course, you could also start listening the old fashioned way, with a normal UI application launch icon. Not as jailbreaky and cool, though :) – Nate May 27 '13 at 20:26
  • that's awesome. I'm excited to get started. At first I thought you meant doing something in the app using the home button, but activating the listening function with the home button is a great idea. – bigblind May 27 '13 at 20:29
  • Hi Nate, I have made the experience that Launch Daemons are paused after under a minute when the device is locked (iOS 9.3.3). The daemon I made continues to run and executes properly when locked while charging. However, when unplugged, the daemon is quickly paused. How could I circumvent that? – Clawish May 08 '17 at 20:31
  • 1
    @Clawish, on battery, iOS probably is trying to conserve power by sleeping regularly. Try looking into the MacOS IOKit APIs for preventing sleep. – Nate May 08 '17 at 21:33
  • @Nate wow you just made my night! Just wrapped the main method of the daemon in what's described [here under Listing 2](https://developer.apple.com/library/content/qa/qa1340/_index.html). Seems to work (tested with 6 minutes of locked screen). Thanks so much, been searching all day – Clawish May 08 '17 at 23:43
  • 1
    @Clawish, glad to hear it worked. There's an amazing amount of stuff from MacOS that's implemented in iOS, but simply isn't **documented**. Source of lots of good features. – Nate May 08 '17 at 23:54