I don't mind entering a password for SSH. For added security however, I'd recommend enabling the Ubuntu Firewall ufw, which is installed but disabled on Jaunty by default. It's easy to enable and configure:
sudo ufw enable
EDIT: Don't do this first if you're connecting remotely or you'll lock yourself out! It's safer to enable last once you're sure all your rules are in place. See Olaf's comment below.
Default block everything
sudo ufw default deny
Allow TCP on prt 22 for SSH:
sudo ufw allow 22/tcp
Delete this rule (if necessary down the road):
sudo ufw delete allow 22/tcp
In addition to port 22, you'll want to allow traffic to port 3306 for MySQL, 80 for Apache, and 20 & 21 by default for ProFTPD.
You can check your rules easily too:
sudo ufw status
Lastly, you can create more fine-grained rules to specific hosts or subnets:
ufw allow proto tcp from 192.168.0.0/24 to 192.168.0.1 port 22
After you configure, disable and re-enable ufw to apply.