The RDP protocol is capable of running directly over the TCP protocol. This is what's happening when you "port forward" an external TCP port to the Terminal Services (TCP port 3389) port of a server. This is cumbersome if you have a number of RDP servers "behind the firewall" that remote users need to access.
The RDP Gateway encapsulates the RDP protocol in HTTP or HTTPS. In this type of configuration you are forwarding an external TCP port to a web server running on either port 80 or 443 (depending on whether you're using SSL or not) on the RDP Gateway machine. The RDP Gateway decapsulates the RDP protocol from the HTTP(S) and makes an RDP connection on behalf of the remote client to the appropriate RDP server. This allows a number of RDP servers to be present with the RDP Gateway brokering access by remote clients to the RDP servers.
I'm not familiar with this client software, but I will speculate that the "Direct Mode" and "Direct SSL Mode" are for connecting directly to the RDP server running on the terminal server. That is, the RDP protocol running directly over TCP with no additional encapsulation. If you've just port-forwarded port 3389 to the Internet you'll want to use one of these modes. Windows 7 and newer versions support SSL within the RDP protocol, so you'd want "Direct SSL Mode" for Windows 7 / Server 2008 R2 and newer RDP servers. For older RDP servers (Windows XP, 2003) you'd want "Direct Mode".
If you've port forwarded an external port to the RDP Gateway running on port 80 or 443 I suspect you'll want to use either "Gateway Mode" or "Gateway SSL Mode". Whether or not you've deployed an SSL certificate onto your RDP Gateway and forwarded SSL connections will determine if you want "Gateway Mode" or "Gateway SSL Mode",