Answering to UDP datagram: response doesn't reach sender

Question

I'm struggling with implementing UDP communication. I'm trying to create two instances of UdpClient, which exchange some information. I'm currently at a stage, where client1 (on port 13006) sends bytes to client2 (port 13007), client2 receives them correctly and send information back, but client1 doesn't receive that response. Both clients work on the same machine (I'm testing as integration test, using XUnit).

Both parties begins to listen the following way:

public override void SetupListener(int port)
{
    udpClient = new UdpClient(port);

    udpClient.BeginReceive(new AsyncCallback(HandleNewConnection), null);
}

Then client1 sends message to client2:

ConnectAsync(IPAddress.Loopback, client2.ListenerPort);

[...]

public override async Task ConnectAsync(IPAddress address, int port)
{
    UdpClientSender = new UdpClient();
    UdpClientSender.Connect(address, port);
    _logger.LogInformation($"({CurrentConnectionId}) Sending ConnectionRequest to {address}:{port}");

    var message = ProtocolHandler.InjectFrame((int)MessageType.ConnectionRequest, null);
    UdpClientSender.Send(bytes, bytes.Length);

    BeginReading();
}

And immadiately starts to listen:

public override void BeginReading()
{
    Thread _thread = new Thread(() =>
    {
        UdpClientSender.BeginReceive(new AsyncCallback(HandleRead), null);
    });
    _thread.Start();
}

client2 receives properly a message and calls this function:

protected override Task WriteToNetworkStream(byte[] bytes)
{
    _logger.LogDebug($"({CurrentConnectionId}) Sending. LocalEndpoint: {UdpClientSender.Client.LocalEndPoint}");
    _logger.LogDebug($"({CurrentConnectionId}) Sending. RemoteEndpoint: {UdpClientSender.Client.RemoteEndPoint}");
    UdpClientSender.Send(bytes, bytes.Length);

    return Task.CompletedTask;
}

But response message doesn't reach client1.

I guess the problem lies in port translation. client1 sends message to port 13007 (which is proper port of client2 and that's why it is sent properly), but client2 receiving a message sees that it came from port 59464 and sends response there. Unfortunately I'm not really sure how to fix that. Any suggestions?