For posterity, and since I had the same problem, I want to document the solution I used from Graham Dumpleton's excellent comment.
As it turns out, a cookie is set during the first request that redirects subsequent requests to the same back-end. To disable this behavior on a per-route basis:
oc annotate routes myroute haproxy.router.openshift.io/disable_cookies='true'
This prevents the cookie from being set and allows the balance algorithm to select the appropriate back-end for subsequent requests from the same client. To change the balance algorithm:
oc annotate routes myroute haproxy.router.openshift.io/balance='roundrobin'
With these two annotations set, requests from the same client IP address are sent to each back-end in turn, instead of the same back-end over and over.