The GPS signals transmitted by GPS satellites are very weak if you are not located outside. Even if you are outside, weather conditions, tall buildings and other large metal structures can hide or degrade the GPS signal. In addition, the GPS receivers inside of your typical phone or tablet are very low grade receivers and antennas, they are not as capable as the more expensive dedicated GPS devices, due to compromises associated price and physical size.
You should really take a two prong approach: get an initial reading using the "coarse" geo reading, with the geo cache enabled. The coarse reading will use the wifi, network, GPS and cached readings and return it's best estimate based on the parameters you specify, usually within a very reasonable time frame.
Then, after you've established an approximate position (which can be very accurate, as @SeahawksRdaBest points out, if based on the current wifi), switch to the "fine" location setting to get an accurate position. This part of your app can run in the background so it doesn't interrupt the responsiveness of your app, and can then update the position information in your app as more accurate position data becomes available.
One thing to keep in mind, many users have the GPS turned off because it severely degrades battery life. If all your app needs is to "find a restaurant nearby" then the coarse location should be good enough, since you're probably more interested in placing the destination on the map. This will be much more battery friendly, and is one of the reasons that the geo settings on the more recent Android devices have changed to make it harder for an app to enable GPS programmatically.
I see you've added the intel-xdk tag, so you might want to checkout this little geolocation test app that I've put together at https://github.com/xmnboy/hello-cordova. It needs a little work but should help you with some experimentation between coarse and fine settings.