How can I ignore or remove ".ipynb_checkpoints" in colab?

Question

My code in tf.keras is given below. I want to retrieve a file(Xscale.npy) in each sub_directory(component_0, component_1) of model_cnn folder.

root_dir = '/content/drive/My Drive/DeepCID/model_cnn'

    i=0
    for (root, dirs, files) in os.walk(root_dir):
      for d in dirs:
        print(dirs) 
        os.chdir(os.path.join(root, d))
        print(os.getcwd())
        datafile3 = './Xscale.npy'
        Xscale = np.load(datafile3)

The error message is,

['.ipynb_checkpoints', 'component_0', 'component_1']
/content/drive/My Drive/DeepCID/model_cnn/.ipynb_checkpoints
---------------------------------------------------------------------------
FileNotFoundError                         Traceback (most recent call last)
<ipython-input-1-862f78aebef9> in <module>()
     57     print(os.getcwd())
     58     datafile3 = './Xscale.npy'
---> 59     Xscale = np.load(datafile3)
     60     Xtest = (Xtest0 - Xscale[0])/Xscale[1]
     61 

/usr/local/lib/python3.6/dist-packages/numpy/lib/npyio.py in load(file, mmap_mode, allow_pickle, fix_imports, encoding)
    426         own_fid = False
    427     else:
--> 428         fid = open(os_fspath(file), "rb")
    429         own_fid = True
    430 

FileNotFoundError: [Errno 2] No such file or directory: './Xscale.npy'

I recognize that '.ipynb_checkpoints' is the problem. But, when I look into the folder, there is no .ipynb_checkpoints file or folder.

My drive in Colab is

My questions are

1) How can I ignore .ipynb_checkpoints when accessing a file in sub_directories?

2) Why is .ipynb_checkpoints file not visible in colab disk?

Thanks in advance, D.-H.

the file isn't visible in the GUI because it is a dot file which means it is hidden by default. also no need to switch directories that breaks things, just feed the path to the item... — gold_cy, Apr 14 '20 at 12:01

gold_cy · Accepted Answer · 2020-04-14T13:20:00.507

Change your code to the following.

1) Check if it is a hidden file

2) Not use os.chdir as it is not necessary.

root_dir = '/content/drive/My Drive/DeepCID/model_cnn'
datafile3 = 'Xscale.npy'

i=0
for (root, dirs, files) in os.walk(root_dir):
    for d in dirs:
        if not d.startswith('.'):
            dir_path = os.path.join(root, d)
            file_path = os.path.join(dir_path, datafile3)
            Xscale = np.load(file_path)

There are more elegant ways to do this in terms of getting absolute file paths but I wanted to minimize the amount of code changed.

An alternative method uses pathlib.

from pathlib import Path

root_dir = '/content/drive/My Drive/DeepCID/model_cnn'
datafile3 = 'Xscale.npy'

i=0
for (root, dirs, files) in os.walk(root_dir):
    for d in dirs:
        if not d.startswith('.'):
            fp = Path(root) / d / datafile3
            Xscale = np.load(str(fp))

If possible, could you show me your more elegant way to do this? — Dong-Ho, Apr 14 '20 at 12:58

score 2 · Answer 2 · answered Mar 28 '21 at 23:37

2

I was having the same issue. This might be happening because during the upload, it is also uploading the .ipynb_checkpoint which is a hidden folder. A fix for this is that you can manually delete the folder using

rmdir /content/drive/My Drive/DeepCID/model_cnn/.ipynb_checkpoints in a cell just before executing the code you mention.

answered Mar 28 '21 at 23:37

prashanth

4,197
4
25
42

This helps but you need to write it as follows: rmdir /content/drive/My Drive/DeepCID/model_cnn/".ipynb_checkpoints" – Mariano Stone Sep 27 '22 at 00:22

score 2 · Answer 3 · answered Jan 14 '23 at 15:54

2

You can try the following solution in this link.

Use the command -

rm -rf `find -type d -name .ipynb_checkpoints`

answered Jan 14 '23 at 15:54

Anubhav Santra

21
3

score 0 · Answer 4 · answered Apr 15 '20 at 00:49

You can use pathlib with rglob for a cleaner code.

from pathlib import Path

root_dir = '/content/drive/My Drive/DeepCID/model_cnn'
root = Path(root_dir)
# parent dir must not start with dot
for datafile in root.rglob('[!.]*/Xscale.npy'):
  print(datafile)  # or np.load(datafile)

score 0 · Answer 5 · answered May 16 '23 at 21:26

import os

file_path = '/content/drive/MyDrive/models/dataset2/train/images/.ipynb_checkpoints.csv'

if os.path.exists(file_path):
    os.remove(file_path)
    print("File deleted successfully.")
else:
    print("File not found.")

import shutil
import os

folder_path = '/content/drive/MyDrive/models/dataset2/train/images/.ipynb_checkpoints'

if os.path.exists(folder_path):
    shutil.rmtree(folder_path)
    print("Folder deleted successfully.")
else:
    print("Folder not found.")

How can I ignore or remove ".ipynb_checkpoints" in colab?

5 Answers5