Yes. I have a set of external VIPs "paired" with a set of internal VIPs that have this same requirement. I also have failover working in both directions.
Here's a look at my config and some explanation. I've redacted the public IPs.
server1 10.0.9.1/24/eth0/10.0.9.255
server2 10.0.9.4/24/eth0/10.0.9.255
server1 x.y.z.2/27/eth0/x.y.z.31
server2 x.y.z.25/27/eth0/x.y.z.31
So, there are two internal floating VIPs, 10.0.9.1/24 and 10.0.9.4/24. 10.0.9.1/24 is primary on server1 and 10.0.9.4/24 is primary on server2.
For external VIPs, x.y.z.2/27 is primary on server1 and x.y.z.25/27 is primary on server2.
I just realized this but I am using the eth0 interface for all these IPs without issue. In reality, the x.y.z.0/27 network is physically connecting to eth1. Give it a shot.
EDIT:
See also keepalived/vrrpd and vrrp_sync_groups:
http://www.keepalived.org/LVS-NAT-Keepalived-HOWTO.html
http://www.keepalived.org/pdf/UserGuide.pdf