I've tried these steps to install VirtualMin on my VPS Ubuntu 16.04. I am damn sure it will work for you.
Here are some prerequisites of the VirtualMin on Ubuntu:
-> You must have a new Ubuntu VPS or system with at 1 GB RAM.
-> You must have root access or access to a user with Sudo privileges.
-> A valid hostname set on your server’s primary IP address.
-> Two registered custom nameservers for Virtualmin.
Steps:
Step 1: Update your system with the latest available version:
Linux command: apt-get update
Step 2: Set Hostname and FQDN:
For Virtualmin to work properly, you need to configure the hostname and FQDN on the server itself by editing the /etc/hostname and /etc/hosts files, as well as update your DNS settings so DNS lookups resolve properly.
Linux command: hostname
To change the hostname for your server, open the file /etc/hostname in your editor:
Linux command: sudo nano /etc/hostname
Delete the current hostname and replace it with your hostname:
Use just the hostname, not the entire FQDN, in this file. For example, if your FQDN is demo.yourdomainname.com, enter demo in this file.
Save the file and exit the editor.
Next, add both the hostname and FQDN in the /etc/hosts file:
Linux command: sudo nano /etc/hosts
Modify the line that starts with 127.0.0.1 to use your FQDN and hostname, in that order:
127.0.1.1 your_hostname.your_domain.com your_hostname
127.0.0.1 localhost
...
Remember to replace your_hostname and your_domain.com with your own hostname and domain name. Save the file and exit the editor.
To check if the name has been changed correctly, reboot your server.
Linux command: sudo reboot
Step 3: Download and install Virtualmin
To install Virtualmin, download and run the official Virtualmin installation script, which will install Virtualmin, and Webmin, along with a LAMP stack, BIND, and Postfix.
Use wget to download the script:
Linux command: wget http://software.virtualmin.com/gpl/scripts/install.sh -O /tmp/virtualmin-install.sh
After the Virtualmin script has been downloaded successfully, we will go for the execution of the script. For that, first we will have to make the script file executable by running the following chmod command:
Linux command: chmod +x /tmp/virtualmin-install.sh
Once the file is made executable, we will go further towards running it using a shell. Enter the following command in the terminal to install Virtualmin through script.
Linux command: sh /tmp/virtualmin-install.sh
The script will give a warning message about existing data and compatible operating systems. Press y to confirm that you want to continue the installation.
The installation process may take a few minutes to complete all the steps as it installs various software packages and components.
Step 4: Configuring Webmin Using The Post-Installation Wizard
To configure Webmin, we'll use its web-based Post-Installation Wizard. Open your web browser and navigate to https://your_server_ip:10000. You can also use your fully-qualified domain name to access the site.
Note: Your browser may show a "Your connection is not secure" or "Your connection is not private" warning since Virtualmin uses a self-signed certificate. This warning is normal. You can ignore it.
Next, we'll need to enter the root username and password which is same as of the Linux system's to get access to the Web-based Control Panel of VirtualMin.
Once you have logged in, you'll see the Introduction screen stating that you are going through the steps to configure Virtualmin. Press Next to continue.
On the Memory use screen, select No for Preload Virtual Libraries, because you don't need to run the Virtual User interface all the time. Select Yes for Run email domain lookup server to enable faster mail processing. Press Next to continue.
On the Virus scanning screen, select No for Run ClamAV server scanner so you'll use less RAM. Press Next to continue.
On the Spam filtering screen, select No for Run SpamAssassin server filter and press Next to continue.
On the Database servers screen, select Yes to Run MySQL database server because MySQL server is widely used and supported by most applications, and no to Run PostgreSQL database server. Press Next to continue.
On the MySQL password screen, enter your desired MySQL root password. It should be different from the root password you used to log in to Webmin.
Select Yes if you’d like to remove the test MySQL database
Select Yes if you’d like to remove the anonymous MySQL user
On the MySQL database size screen, select the RAM option that matches the amount of RAM your server has. For a 1GB server, select Large system (1G) on which MySQL is heavily used. Press Next to continue.
On the Nameserver screen, Enter your primary and secondary nameservers which you configured in the prerequisites. If you haven't set these up, check the Skip check for resolvability box to avoid error message and proceed. Press Next to continue.
On the Password storage mode screen, select Store plain-text passwords if you must support password recovery. Otherwise, choose Only store hashed passwords. Here we have chosen plain-text passwords as an example. Press Next to continue.
you will see the All done screen. Click Next to end.
You may see a message stating that Virtualmin's configuration has not been checked. Click the Re-check and refresh the configuration button to check your Virtualmin configuration. Address any errors that the check reports. This process may take a few minutes to complete.
Congratulations! Your server is now configured.