Importing .py files in Google Colab

Question

Is there any way to upload my code in .py files and import them in colab code cells?

The other way I found is to create a local Jupyter notebook then upload it to Colab, is it the only way?

korakot · Accepted Answer · 2020-12-08T12:42:04.700

161

You can save it first, then import it.

from google.colab import files
src = list(files.upload().values())[0]
open('mylib.py','wb').write(src)
import mylib

Update (nov 2018): Now you can upload easily by

click at [>] to open the left pane
choose file tab
click [upload] and choose your [mylib.py]
import mylib

Update (oct 2019): If you don't want to upload every time, you can store it in S3 and mount it to Colab, as shown in this gist

Update (apr 2020): Now that you can mount your Google Drive automatically. It is easier to just copy it from Drive than upload it.

Store mylib.py in your Drive
Open a new Colab
Open the (left)side pane, select Files view
Click Mount Drive then Connect to Google Drive
Copy it by !cp drive/MyDrive/mylib.py .
import mylib

edited Dec 08 '20 at 12:42

answered Feb 22 '18 at 03:10

korakot

37,818
16
123
144

2

@rsam Then try the second method. I tried it and it worked for me. – korakot Feb 23 '18 at 04:58
files.upload() currently does not work in Firefox. Alternative way is to upload it into Google Drive, then from a notebook use PyDrive to download it. – Alexey Tigarev Mar 28 '18 at 13:20
3

If you use the file tab method you have to repeat the upload each runtime – information_interchange Oct 21 '19 at 00:56
1

I now add the method to store it in S3, so you don't need to upload again. – korakot Oct 21 '19 at 02:55
I am facing the error given below, please let me know anyone of you have any idea "cp: missing destination file operand after '/content/drive/My Drive/Colab Notebooks/attention.py'" – eswara amirthan s Jun 01 '20 at 06:03
8

@eswaraamirthans You forget the dot "." at the end. – korakot Jun 01 '20 at 06:51
where can I insert the command !cp "drive/My Drive/mylib.py" . ? – Wesin Alves Oct 02 '20 at 09:44
@WesinAlves: You can add it into a code cell and execute it the same way as you would execute python code. You also might to have to remove the space in 'My Drive' -> 'MyDrive'. – themenace Dec 08 '20 at 09:52
Thank you for this tutorial. How about opening a .py from Github repo in colab? – HoOman Feb 03 '21 at 17:37
Now you need to add "/content" after the path, so it would be !cp drive/MyDrive/mylib.py /content – kiril_petrov Apr 10 '22 at 12:51
1

It took me a while to find that I forgot to add the `.` at end of `!cp drive/MyDrive/mylib.py .`. Anyhow, this is a clean and nice solution. Solved my big problem. Thanks a ton. Moreover, I really appreciate for updating the answer with colab updating time to time. Really appreciate that. – Mainland Apr 10 '22 at 22:18

score 117 · Answer 2 · answered Aug 08 '19 at 15:35

117

In case anyone else is interested to know how to import files/packages from gdrive inside a google colab. The following procedure worked for me:

1) Mount your google drive in google colab:

from google.colab import drive
drive.mount('/content/gdrive/')

2) Append the directory to your python path using sys:

import sys
sys.path.append('/content/gdrive/mypythondirectory')

Now you should be able to import stuff from that directory!

answered Aug 08 '19 at 15:35

a3VonG

1,279
1
8
4

1

Hello. Thank you. But I am looking for importing python files that depend on other files as well. Can I do that please? – Avv Nov 27 '21 at 21:11
from google.colab import drive import sys drive.mount('/content/gdrive') colab_notebooks_path = "gdrive/MyDrive/Colab Notebooks/" sys.path.append(colab_notebooks_path) – Mahdi Mohammadizadeh Aug 31 '22 at 22:27

score 19 · Answer 3 · answered Mar 23 '18 at 15:13

19

You can upload local files to google colab by using upload() function in google.colab.files
If you have files on github, then clone the repo using !git clone https://github.com/username/repo_name.git. Then just like in jupyter notebook load it using the magic function %load %load filename.py.

answered Mar 23 '18 at 15:13

Aditya Mishra

1,687
2
15
24

1

The second method worked for me, I just had to add the name of the repo: `%load repo_name/filename.py` – xyzjayne May 07 '19 at 23:05
1

@xyzjayne did you try importing it? I can load it fine, but can't use it as a module – information_interchange Oct 21 '19 at 00:58

score 15 · Answer 4 · answered May 11 '18 at 11:27

15

Based on the answer by Korakot Chaovavanich, I created the function below to download all files needed within a Colab instance.

from google.colab import files
def getLocalFiles():
    _files = files.upload()
    if len(_files) >0:
       for k,v in _files.items():
         open(k,'wb').write(v)
getLocalFiles()

You can then use the usual 'import' statement to import your local files in Colab. I hope this helps

answered May 11 '18 at 11:27

Flint Joe

151
1
2

Thanks, I am a noobie and this is the only answer I understood how to implement and it works for me – NeStack Aug 07 '19 at 19:00

score 6 · Answer 5 · answered Jul 05 '20 at 03:05

6

We can do so.

import sys
import os

py_file_location = "/content/drive/My Drive"
sys.path.append(os.path.abspath(py_file_location))

Now you can import it as module in notebook for that location.

import whatever

answered Jul 05 '20 at 03:05

`/content` doesn't contain `drive`. So it is impossible to access my files – Leonid Mednikov Mar 10 '23 at 05:42

score 4 · Answer 6 · answered Aug 07 '19 at 21:23

I face the same problem. After reading numerous posts, I would like to introduce the following solution I finally chose over many other methods (e.g. use urllib, httpimport, clone from GitHub, package the modules for installation, etc). The solution utilizes Google Drive API (official doc) for proper authorization.

Advantages:

Easy and safe (no need for code to handle file operation exceptions and/or additional authorization)
Module files safeguarded by Google account credentials (no one else can view/take/edit them)
You control what to upload/access (you can change/revoke access anytime on a file-by-file basis)
Everything in one place (no need to rely upon or manage another file hosting service)
Freedom to rename/relocate module files (not path-based and won't break your/other's notebook code)

Steps:

Save your .py module file to Google Drive - you should have that since you're already using Colab
Right click on it, "Get shareable link", copy the part after "id=" - the file id assigned by Google Drive
Add and run the following code snippets to your Colab notebook:

!pip install pydrive                             # Package to use Google Drive API - not installed in Colab VM by default
from pydrive.auth import GoogleAuth
from pydrive.drive import GoogleDrive

from google.colab import auth                    # Other necessary packages
from oauth2client.client import GoogleCredentials

auth.authenticate_user()                         # Follow prompt in the authorization process
gauth = GoogleAuth()
gauth.credentials = GoogleCredentials.get_application_default()

drive = GoogleDrive(gauth)
your_module = drive.CreateFile({"id": "your_module_file_id"})   # "your_module_file_id" is the part after "id=" in the shareable link
your_module.GetContentFile("your_module_file_name.py")          # Save the .py module file to Colab VM
import your_module_file_name                                    # Ready to import. Don't include".py" part, of course :)

Side note

Last but not least, I should credit the original contributor of this approach. That post might have some typo in the code as it triggered an error when I tried it. After more reading and troubleshooting my code snippets above worked (as of today on Colab VM OS: Linux 4.14.79).

Umm when i try to call a function in the imported py file this code gives me error — Leothorn, May 23 '20 at 20:37

score 3 · Answer 7 · answered Mar 19 '18 at 11:20

3

Try this way:

I have a package named plant_seedlings. The package is stored in google drive. What I should do is to copy this package in /usr/local/lib/python3.6/dist-packages/.

!cp /content/drive/ai/plant_seedlings.tar.gz /usr/local/lib/python3.6/dist-packages/

!cd /usr/local/lib/python3.6/dist-packages/ && tar -xzf plant_seedlings.tar.gz

!cd /content

!python -m plant_seedlings

answered Mar 19 '18 at 11:20

Feng

559
3
8

Doesn't work for me. "/content" does not have "drive" subdirectory. – Alexey Tigarev Mar 28 '18 at 12:09
I am not sure of this answer but for your problem, you should first click `Mount Drive` button of google drive. Then drive folder will appear under content folder. – Tekin Jun 11 '20 at 19:22

viraj ghorpade · Answer 8 · 2021-01-23T07:40:39.130

Below are the steps that worked for me

Mount your google drive in google colab

from google.colab import drive drive.mount('/content/drive')
Insert the directory

import sys sys.path.insert(0,’/content/drive/My Drive/ColabNotebooks’)
check the current directory path

%cd drive/MyDrive/ColabNotebooks %pwd
Import your module or file

import my_module
If you get the following error 'Name Null is not defined' then do the following

5.1 Download my_module.ipynb from colab as my_module.py file (file->Download .py)

5.2 Upload the *.py file to drive/MyDrive/ColabNotebooks in Google drive

5.3 import my_module will work now

Reference: https://medium.com/analytics-vidhya/importing-your-own-python-module-or-python-file-into-colab-3e365f0a35ec

https://github.com/googlecolab/colabtools/issues/1358

score 2 · Answer 9 · answered Jul 08 '21 at 15:45

2

Here's my process:

import sys 
sys.path.insert(0, '/content/drive/MyDrive/my_folder')
%cd /content/drive/MyDrive/my_folder
%pwd

Now, you can import the module from that path using import my_module easily

answered Jul 08 '21 at 15:45

Kamal

309
2
5

score 1 · Answer 10 · answered May 22 '18 at 12:00

1

You can upload those .py files to Google drive and allow Colab to use to them:

!mkdir -p drive
!google-drive-ocamlfuse drive

All your files and folders in root folder will be in drive.

answered May 22 '18 at 12:00

Huyen

443
4
17

you first have to install the FUSE filesystem for google drive before you can do that – FPcond Jul 06 '18 at 21:03
Actually mine works with only those lines above. Have you tried it? – Huyen Sep 09 '18 at 21:30

score 0 · Answer 11 · answered Jun 27 '19 at 15:44

It's Jun 2019. Make sure in the Python package's __init__.py all related files are imported in order. Push the code to Git or use this code.

for e.g,

from .Boxes import *
from .Circles import *
from .Rectangles import *
...

Don't use Package name in __init__.py file for importing the files.

in Google colab,

! rm -rf SorghumHeadDetection
! git clone https://github.com/user/amazing-repo-name/

score 0 · Answer 12 · answered Aug 26 '21 at 00:12

you can do this by mount your drive to colab and write some code to put the id of your python file you can find code here importing python file from drive to colab

    # Code to read file into colaboratory:
     !pip install -U -q PyDrive
from pydrive.auth import GoogleAuth
from pydrive.drive import GoogleDrive
from google.colab import auth
from oauth2client.client import GoogleCredentials


#Autheticate E-Mail ID
auth.authenticate_user()
gauth = GoogleAuth()
gauth.credentials = GoogleCredentials.get_application_default()
drive = GoogleDrive(gauth)

#2.1 Get the file
your_module = drive.CreateFile({"id": 'write your file id here'})  # "your_module_file_id" is the part after "id=" in the shareable link
your_module.GetContentFile("write the file name here")          # Save the .py module file to Colab VM

import file_name   
from file_name import anything    #as classes or functions from your file

score 0 · Answer 13 · answered Sep 06 '21 at 19:18

0

os.listdir can be used to view all files in the directory

from google.colab import drive
drive.mount("/content/drive")

import os
path="/content/drive/My Drive/Colab Notebooks"
os.chdir(path)
os.listdir(path)`

answered Sep 06 '21 at 19:18

Jemma_Zhou

1
1

Haddock-san · Answer 14 · 2021-11-16T16:18:05.283

In my case, the file I was trying to use was called client.py. This raised a conflict because there's already a library called client in /usr/local/lib/python3.7/dist-packages/.

I solved this by uploading the client.py file to the same Google Drive folder in which the Colab Notebook is saved in and change its name to something unique that doesn't appear in the dist-packages folder.

In my case, I changed the file name to dfsclient.py and then just imported it with

import dfsclient

Then I implemented Kamal's answer:

import sys 
sys.path.insert(0, '/content/drive/MyDrive/my_folder')

score 0 · Answer 15 · edited Mar 04 '22 at 18:39

0

This is how I regularly do it:

Save my module in the directory. Say MyFile.py in MyModules
Define the location of my module:

path_m = '/content/drive/MyDrive/Colab Notebooks/MyModules/'
Then I add the path to sys.path:
```
import sys

sys.path.insert(0,path_m)
```
import the module into my Jupyter/Google Colab notebook.

import MyFile

edited Mar 04 '22 at 18:39

ouflak

2,458
10
44
49

answered Mar 04 '22 at 15:39

Aram

1

score -2 · Answer 16 · edited Mar 13 '20 at 19:17

-2

A easy way is

type in from google.colab import files uploaded = files.upload()
copy the code
paste in colab cell

edited Mar 13 '20 at 19:17

luca.vercelli

898
7
24

answered Mar 13 '20 at 15:21

Sanidhya Tyagi

47
2
11

Importing .py files in Google Colab

16 Answers16

Advantages:

Steps:

Side note

Mount your google drive in google colab

Insert the directory

check the current directory path

Import your module or file

Linked

Related