Netplan routes do not seem to wrok with scaleway ip failover

Question

I have a scaleway dedicated server (dedibox) with 1.1.1.1 as my primary IP.

i have linked a "failover ip" 2.2.2.2 to this server

i can ssh into the server with the two ip, but i can't get the outgoing traffic to go through 2.2.2.2

if i completely remove the primary IP 1.1.1.1 from the netplan config file the server becomes unreachable ( can't even ssh into it with 2.2.2.2 )

here is my netplan configuration:

network:
  renderer: networkd
  ethernets:
    enp0s20f0:
      critical: true
      dhcp4: false
      addresses: [1.1.1.1/24, 2.2.2.2/32]
      gateway4: x.x.x.x
      nameservers:
        addresses:
        - 62.210.16.6
        - 62.210.16.7
        search:
        - online.net
      routes:
        - to: 0.0.0.0/0
          via: 2.2.2.2
          metric: 100

Ultimately i want the server to act as if the ip 2.2.2.2 was the primary ip and i pretty much doesn't care about the actual primary ip 1.1.1.1

Also when i do netplan apply (even with no modification of the config) the server goes offline and i must restart to get it back online