How to transform 3d world matrix to fly an aircraft?

Question

I am programming a simple flight game in directX. I've modeled the aircraft and successfully imported it in the game. I made an input controller successfully. I made the turning left, right, up and down successfully.

I planned to have 3 vectors that keeps the roll, pitch and yaw axis of aircraft while he is flying.

DirectX::XMFLOAT3 flightDirection, flightRight, flightUp;

Initial states of vectors are:

flightDirection(0.0f, 0.0f, 1.0f);
flightRight(1.0f, 0.0f, 0.0f);
flightUp(0.0f, 1.0f, 0.0f);

if (newPointerPosition.x != pointerPosition.x || newPointerPosition.y != pointerPosition.y)
{
    xPointer = (newPointerPosition.x / resources->GetLogicalSize().Width) - 0.5f;
    yPointer = (newPointerPosition.y / resources->GetLogicalSize().Height) - 0.5f;
    rollAngle = xPointer;
    yawAngle = xPointer;
    pitchAngle = yPointer;

    XMStoreFloat4x4(&roll,XMMatrixRotationAxis(XMLoadFloat3(&flightDirection), rollAngle));
    XMStoreFloat4x4(&pitch, XMMatrixRotationAxis(XMLoadFloat3(&flightRight), pitchAngle));
    XMStoreFloat4x4(&yaw, XMMatrixRotationAxis(XMLoadFloat3(&flightUp), yawAngle));

    rotate = DirectX::XMMatrixMultiply(XMLoadFloat4x4(&pitch),  DirectX::XMMatrixMultiply(XMLoadFloat4x4(&yaw), XMLoadFloat4x4(&roll)));

    auto d = XMVector3Transform(XMLoadFloat3(&flightDirection), rotate);
    position.x = XMVectorGetX(d);
    position.y = XMVectorGetY(d);
    position.z = XMVectorGetZ(d);
    XMStoreFloat3(&flightDirection, d);

    temp = DirectX::XMVector3Transform(XMLoadFloat3(&flightRight), rotate);
    XMStoreFloat3(&flightRight, temp);

    temp = DirectX::XMVector3Transform(XMLoadFloat3(&flightUp), rotate);
    XMStoreFloat3(&flightUp, temp);

    pointerPosition = newPointerPosition;
}
translation._41 += position.z*0.01f;
translation._42 += position.y*0.01f;
translation._43 += position.x*0.01f;

world->Transform(XMLoadFloat4x4(&scale), rotate, XMLoadFloat4x4(&translation));

So this is my algorithm. First, position of the mouse sets the angle for rotations (angle is limited -0.5f < angle < 0.5f). Then I create a rotation matrixes around unit vectors that keep aircraft axis. rotate is the R matrix. After that I calculate new unit vectors that keep aircraft axis. Finally I add components of flightDirection to the translation matrix to get new position of an Aircraft.

My problem is that my algorithm doesn't work the way I intended. Something is calculating wrong and my aircraft flies in wrong and random directions after i move the mouse.

Please tell me how to correct this algorithm or give me the link of example how moving an object in 3D space is done.

Best Regards, Antun