establishing IKE_SA failed, peer not responding - Strongswan with Centos 7 [Possibly duplicate]

Question

I've been on this VPN tunnel for over a week now and keep getting Peer not responding when I bring up the tunnel. I have Strongswan installed and I have also created 1 tunnel which is working fine and connection established, then I added a second one, but whenever I bring up the second tunnel, after 5 attempts, I get Peer not responding. The remote server cannot see any connection from me. Here's my ipsec.conf file:

config setup
    charondebug="all"

conn %default
        ikelifetime=24h
        keylife=20m
        rekeymargin=3m
        keyingtries=1
        authby=secret
        mobike=no

conn Foo-to-Bar
        keyexchange=ikev1
        left=196.xxx.xxx.xx #PUBLIC IP of my server
        leftsubnet=196.xxx.xx.xx/32 #PUBLIC IP of my server
        leftid=196.xxx.xx.xx #PUBLIC IP of my server
        leftfirewall=yes
        right=41.xxx.xx.xx #Remote Peer IP
        rightsubnet=41.xxx.xx.xx/32 #Remote Host IP
        rightid=41.xxx.xx.xx
        auto=route
        esp=3des-sha1
        ike=3des-sha1-modp1024
        type=tunnel
        lifetime=24h
        dpdaction=clear
        ike_dhgroup=group2

conn Foo-to-Bar2
        also=Foo-to-Bar
        rightsubnet=xxx.xxx.xx.xx/32 #Another remote host
        esp=3des-sha1

When I do strongswan up Foo-to-Bar here's what I get:

strongswan up Foo-to-Bar
initiating Main Mode IKE_SA Foo-to-Bar[2] to xxx.xxx.xx.xx
generating ID_PROT request 0 [ SA V V V V V ]
sending packet: from 196.xxx.xxx.xx[500] to 41.xxx.xx.xx[500] (248 bytes)

sending retransmit 1 of request message ID 0, seq 1
sending packet: from 196.xxx.xxx.xx[500] to 41.xxx.xx.xx[500] (248 bytes)

sending retransmit 2 of request message ID 0, seq 1
sending packet: from 196.xxx.xxx.xx[500] to 41.xxx.xx.xx[500] (248 bytes)
sending retransmit 3 of request message ID 0, seq 1
sending packet: from 196.xxx.xxx.xx[500] to 41.xxx.xx.xx[500] (248 bytes)
sending retransmit 4 of request message ID 0, seq 1
sending packet: from 196.xxx.xxx.xx[500] to 41.xxx.xx.xx[500] (248 bytes)
sending retransmit 5 of request message ID 0, seq 1
sending packet: from 196.xxx.xxx.xx[500] to 41.xxx.xx.xx[500] (248 bytes)
giving up after 5 retransmits
establishing IKE_SA failed, peer not responding
establishing connection 'Foo-to-Bar' failed

Also when I check my /var/log/messages I get

# localhost charon: 04[NET] sending packet: from 196.xxx.xxx.xx[500] to 41.xxx.xx.xx[500] (248 bytes)
# localhost charon: 03[NET] error writing to socket: Network is unreachable

What could be the cause? I am a bit confuse that I have one config just above this on the same file which is able to establish the connection, while this one does not and the remote host cannot see my connection attempts on their log.

I'd appreciate any help.