TypeError: file must have 'read' and 'readline' attributes

Question

1st approach: I am trying to make the below code work since morning. I have read many answers here in stackoverflow and tutorials on google about python, I have done 0 progress. Can you help me with this error:

Using TensorFlow backend.
Traceback (most recent call last):
  File "source_code_modified.py", line 65, in <module>
    dict1 = pickle.load(f1,encoding='bytes')
TypeError: file must have 'read' and 'readline' attributes

The code that explodes is this part:

class load_train_data:
    os.open("/home/just_learning/Desktop/CNN/datasets/training_data/images", os.O_RDONLY)
    def __enter__(self):
        return self
    def __exit__(self, type, value, traceback):
        pass


class load_test_data:
    os.open("/home/just_learning/Desktop/CNN/datasets/test_data/images",os.O_RDONLY) 
    def __enter__(self):
        return self
    def __exit__(self, type, value, traceback):
        pass


with load_train_data() as f1:
   dict1 = pickle.load(f1,encoding='bytes')

2nd approach:

Ok, I did that, the error is:

Using TensorFlow backend.
Traceback (most recent call last):
  File "source_code_modified.py", line 74, in <module>
    with open_train_data() as f1:
  File "source_code_modified.py", line 47, in open_train_data
    return open('/home/just_learning/Desktop/CNN/datasets/training_data/images','rb') 
IsADirectoryError: [Errno 21] Is a directory: '/home/just_learning/Desktop/CNN/datasets/training_data/images'

And the code explodes on these points:

def open_train_data():
    return open('/home/just_learning/Desktop/CNN/datasets/training_data/images','rb') <--- explodes here

def open_test_data():
    return open('/home/just_learning/Desktop/CNN/datasets/test_data/images','rb')

with open_train_data() as f1:
   dict1 = pickle.load(f1)   <--- explodes here

3rd approach: I found this: "IsADirectoryError: [Errno 21] Is a directory: " It is a file

and I changed the code to this:

def open_train_data():
    return os.listdir('/home/just_learning/Desktop/CNN/datasets/training_data/images') 

def open_test_data():
    return os.listdir('/home/just_learning/Desktop/CNN/datasets/test_data/images')

with open_train_data() as f1:           <-------------- explodes here
   dict1 = pickle.load(f1) #,encoding='bytes')

and the error is this:

Using TensorFlow backend.
Traceback (most recent call last):
  File "source_code_modified.py", line 74, in <module>
    with open_train_data() as f1:
AttributeError: __enter__

and this error I solved it by searching in github/google and was the result depicted on the first approach, that I have posted above...

Have you tried picke.loadS ignore big S i did that so you can see better. — Jon Nezbit, Jun 10 '20 at 20:12
You have a misunderstanding on how class definitions work. The `os.open` calls are executed only on class definition, not whenever you instantiate the class. The first argument to `pickle.load` is supposed to be a "file-like object"; meaning in this case it needs to have `read` and `readline` methods. — L3viathan, Jun 10 '20 at 20:14
Your 2nd approach gives the error message: `Is a directory`. That is a hint... — Han-Kwang Nienhuys, Jun 10 '20 at 21:17
Hint: `pickle.load` is for loading files that were created by `pickle.dump`. — Han-Kwang Nienhuys, Jun 11 '20 at 09:03

score 8 · Accepted Answer · edited Jul 11 '22 at 16:45

Just sharing some stupid mistake for anyone else affected (by stupidity)^^:

I had the error TypeError: file must have 'read' and 'readline' attributes because I used the plain text file_path instead of the opened f file object as the parameter of the pickle function.

Correct:

file_path = 'path/to/filename'

with open(file_path , 'rb') as f:
    dict1 = pickle.load(f)

Wrong:

file_path = 'path/to/filename'

with open(file_path , 'rb') as f:
    dict1 = pickle.load(file_path)

Han-Kwang Nienhuys · Answer 2 · 2020-06-10T21:21:49.780

4

Use something like:

def open_test_data():
    return open('path/to/filename', 'rb')

with open_test_data() as f:
    dict1 = pickle.load(f)

There is no reason to attempt to define your own context manager for something like this.

Edit: your original first attempt had encoding='bytes'. You may need to add that depending on where the data comes from.

edited Jun 10 '20 at 21:21

answered Jun 10 '20 at 20:30

Han-Kwang Nienhuys

3,084
2
12
31

Because I am searching but I cannot figure out how to make it work. Is there any way to load an image dataset stored in my PC ('png') so that it can be read by: trainX, trainY, testX, testY?? Because most of what I found is like this: (x_train, y_train), (x_test, y_test) = mnist.load_data() or (trainX, trainY), (testX, testY) = cifar10.load_data() – just_learning Jun 11 '20 at 22:08
Another question: Can I transform the folder with the images to a pickle (.P) file? – just_learning Jun 11 '20 at 22:14
1

For that, you can post a new question. – Han-Kwang Nienhuys Jun 11 '20 at 22:24
1

The wording "There is no reason to attempt to define your own context manager for something like this." confused me a bit, I could not surely say whether your `def` is now this "own context manager" since I have not seen it like this before. Admittedly, if I had read the question more thoroughly, I would have clearly seen the point. Anyway, of course, `with open('path/to/filename', 'rb') as f: dict1 = pickle.load(f)` and your code are the same. – questionto42 Sep 17 '21 at 15:37

TypeError: file must have 'read' and 'readline' attributes

2 Answers2