Recently I was asked to implement client-side failover for an app I wrote. Personally, all the failover/load-balancing schemes I can recall working with have been implemented server-side, through a middleman proxying server. The head of the server team though, wants me to do something different, which has lead me to a number of questions, and some frustration in not finding much in the way of discussion on the related techniques and frameworks. Having spent a little time thinking about the problem, I have a few concerns, and I was hoping folks here could help educate me :)...
So, a basic scheme where you might start with multiple server addresses in some form which I would rotate through tracking failures before giving up, etc. seems simple enough. He wants something a bit different. His desire is to set up our DNS servers to return multiple A Records for our various web services/servers. He then wants my client to intelligently failover between the various ip addresses when one fails. Similar in conception to how a lot of big web farms work. Again simple in conception, but in the details, I have concerns.
At this point I think, I should describe the environment I'm working in. The app in question is a mobile app running on Android/iOS. Internally, not that it's wildly important other than for leveraging frameworks, the app is written as an angular spa built on top of cordova (and thus is a hybrid mobile app). At the moment, I use ngResource for all my web api needs (which uses $http underneath).
A) So then to my first concern. To my knowledge, a DNS entry with multiple A records has no guarantee of what IP address(es) get delivered to any given client. Resolved DNS records are cached on intermediary DNS servers, usually only saving 1 record unless the DNS server is "smart". While some DNS servers might weight ip addresses by response times, tracerts, etc. most do not, even if they have more than one ip address. If I am correct, it would seem this scheme will not likely work well in practice.
B) My next conceptual hurdle is much more implementation specific. Apriori, it seems the only way I can get a full set of IP Addresses for a domain name is via a native plugin (e.g InetAddress.getAllByName in Java on Android). This would seem preferable to the second obvious answer--an external web service that would do the lookup for me. I'm not aware of any way to do this in Javascript, let alone any angular/$http/$resource module with this capability rolled into it? Any suggestions?
C) Having gotten multiple ip addresses (somehow), I can setup a promise chain with timeouts and failover. This is not wildly desireable to me, however. Currently I've built my app to have use an angular factory produced $resource objects with the appropriates paths defined (e.g. Accounts), which I use the standard dependency injection mechanism to gain access to in various controllers as needed. I'd rather not reinvent the wheel nor create a huge code footprint or maintenance issues for myself or anyone else, and thus I'd like to come up with the most "common practice" method for handling this that won't make some other angular/cordova coder lose his mind and start sacrificing swedish code fish to his cephalopodal altar erected in some perpetually dark corner of his (or her!) mind. Therein, assuming I've had to get a collection of IP Addresses for a dns name from an external-to-angular source at some point after the app has bootstrapped itself up (since these will be async calls at possibly arbitrary points in the app's lifecycle), (catches breath) how would folks recommend structuring the creation of my ngResource objects to reference them? I can imagine a couple different techniques, but I'm rather hoping this isn't as untread territory has my googling has lead me to believe, and folks will have some recommendations for me 8).
Thanks in advance!
