Preface
Please bear with me if I use incorrect terminology or don't express the problem too well since I'm not an expert on system administration/server maintenance. Let me know/correct me if I do so I can learn and clarify my points.
Problem
I have a Linux device, the server, that runs different services on different ports (an HTTP, SSH and FTP server currently, but possibly other servers for other protocols in the future). My friends and I have other devices, the clients, that are running Linux (incl. Android) or Windows, that we'd like to use to access the server's services, as long as both client and server are online on the internet, regardless of whether they're on the same local network.
The other caveat is that we'd like to do this securely such that the server can ensure that it's really only me or my friends accessing it, that we can be sure that the server we're connecting to really is the correct server (not some MITM spoofing their identity as the server), and without third parties being able to obtain (too much) meaningful information by sniffing the exchanged packets.
I'd also like to restrict my friends' access to only specific services (say, only HTTP and FTP for one friend, only SSH for another, etc.).
Possibly relevant information
I have admin access to the server (I can install packages and configure it with unrestricted access) and local network router. The server is running nftables.
Actions considered
I've thought of configuring nftables on the server to only allow inbound packets from specific IP addresses or devices with specific MAC addresses, but I don't think these are appropriate/adequate. First because of the constraint that we'd like to be able to connect from outside the local network, so the client devices' IP addresses can change. Second, because I know that MAC addresses can easily be spoofed so I can't use those to ensure that the client devices really are the allowed ones. Third, because these don't address the constraint that third parties shouldn't be able to obtain meaningful information by snooping on the packets (so ideally, the solution should employ some sort of cryptographic protocols to address this).
Other thoughts
I'm thinking that the solution would involve setting up some sort of accounts-based system where my friends and I each have our own accounts and the server only allows packets of specific protocols from certain accounts.
I'm also thinking I can maybe solve this by setting up a VPN server using a protocol like WireGuard or OpenVPN on the server which would only allow connections from authenticated clients in a whitelist, then route all other traffic like SSH and HTTP through the VPN tunnel, making them accessible only when connected via that tunnel. Would that work? But then I'm not sure how I'd configure that to restrict my friends' access to particular services only. Do these VPN protocols have some feature to restrict the types of traffic allowed per client?
In any case, I don't really have a clue what the optimal solution for this is and how I'd set it up, so I'd really appreciate any useful advice, suggestions and information to solve this.
