I also had to set up my system to use CUDA 8 and cuDNN 6.
As you have pointed out, you have to first install CUDA. Afterwards, install cuDNN.
You can get the URL of the CUDA Installer that suits your operating system / target platform by visiting CUDA Toolkit Archive - CUDA Toolkit 8.0 - Feb 2017. Make sure you have an NVIDIA developer account before visiting the NVIDIA developer site.
To get the CUDA Installer for Linux Ubuntu 16.04 x86_64 you can use:
!wget https://developer.nvidia.com/compute/cuda/8.0/Prod2/local_installers/cuda-repo-ubuntu1604-8-0-local-ga2_8.0.61-1_amd64-deb
!dpkg -i cuda-repo-ubuntu1604-8-0-local-ga2_8.0.61-1_amd64-deb
!apt-key add /var/cuda-repo-8-0-local-ga2/7fa2af80.pub
!apt-get update
!apt-get install cuda=8.0.61-1
!apt autoremove
To install NVIDIA cuDNN you must have a NVIDIA developer account.
After logging in with you credentials on the NVIDIA developer site, open the NVIDA cuDNN archive, where you can find cuDNN v1 to v7.6.4 at the moment. Now, click on the entry for "Download cuDNN v6.0 (April 27, 2017), for CUDA 8.0". To download cuDNN for Ubuntu 16.04, click on "cuDNN v6.0 Runtime Library for Ubuntu16.04 (Deb)".
You can follow the direct link to cuDNN v6.0 Runtime Library for Ubuntu16.04 (Deb), but make sure you are logged into your NVIDIA developer account first.
Added the following paragraph on 2020-09-18: I do not provide a script to download NVIDIA cuDNN directly using Google Colab here. You could download NVIDIA cuDNN to your local system and upload it to Google Colab via Google Drive, for instance, mount your Google Drive and then install it with the already provided command below.
Here is an example script:
# Download NVIDIA cuDNN after logging in with your NVIDIA developer account
# https://developer.nvidia.com/rdp/cudnn-archive
# Select "Download cuDNN v6.0 (April 27, 2017), for CUDA 8.0"
# Download "cuDNN v6.0 Runtime Library for Ubuntu16.04 (Deb)"
# Directlink (requires NVIDIA developer session):
# https://developer.nvidia.com/compute/machine-learning/cudnn/secure/v6/prod/8.0_20170307/Ubuntu16_04_x64/libcudnn6_6.0.20-1+cuda8.0_amd64-deb
# After downloading, install cuDNN
!dpkg -i "~/Downloads/libcudnn6_6.0.21-1+cuda8.0_amd64.deb"
# Check if package has been installed
!ls -l /usr/lib/x86_64-linux-gnu/libcudnn.so.6*
This should output for example:
Preparing to unpack .../libcudnn6_6.0.21-1+cuda8.0_amd64.deb ...
Unpacking libcudnn6 (6.0.21-1+cuda8.0) ...
Setting up libcudnn6 (6.0.21-1+cuda8.0) ...
Processing triggers for libc-bin (2.27-3ubuntu1) ...
lrwxrwxrwx 1 root root 18 Apr 12 2017 /usr/lib/x86_64-linux-gnu/libcudnn.so.6 -> libcudnn.so.6.0.21
-rw-r--r-- 1 root root 154322864 Apr 12 2017 /usr/lib/x86_64-linux-gnu/libcudnn.so.6.0.21
To clarify my answer, I also added webpage screenshots below for you.

