0

I have this pickle file https://github.com/Nilabhra/ethnicity/blob/master/models/ethnicity_classifier_last_name.pkl which is generated from json file https://github.com/Nilabhra/ethnicity/blob/master/json_counts/last_name_ethnicity.json

My question: How to remove old dataset and put new dataset in .pkl file.

import pickle

ethinicity= {"Kumari": {"Hindu,Brahmin": 1.0},"Choopra": {"Jain,Digambar": 1.0}}
pickle.dump(ethinicity, open("ethnicity_classifier_last_name.pkl", "wb"))

however the pickle file generated by above code has a different structure, hence its throwing error when i run this code

Rizwan
  • 95
  • 1
  • 13

2 Answers2

1

remove old pickle file and dump new pickle file with new dataset in it.

Dariusz Krynicki
  • 2,544
  • 1
  • 22
  • 47
0

You need to update the old dict with the new entries before writing to the pickle file

import pickle 
import json

#Loading the old json
old_ethnicity = json.load(open('last_name_ethnicity.json','rb'))
ethinicity= {"Kumari": {"Hindu,Brahmin": 1.0},"Choopra": {"Jain,Digambar": 1.0}}

#Add the changes to old dict
new_ethnicity = dict(old_ethnicity, **ethinicity)
pickle.dump(new_ethnicity, open("ethnicity_classifier_last_name.pkl", "wb"))`
jeweljames
  • 1