How can I build a custom context based Question answering model SQuAD using deeppavlov

Question

I have the following queries

1. Dataset format (is how to split train, test and valid data )
2. Where to place the dataset
3. How to change the path for dataset reader
4. How to save the model in my own directory
5. And How to use the trained model

Edit

my_config['dataset_reader']['data_path'] = '/home/ec2-user/SageMaker/squad/data/'  
my_config['metadata']['variables']['MODELS_PATH'] = '/home/ec2-user/SageMaker/squad/model/'
 

I used this command to change my dataset path and model path in configuration file. My model is saved in this location but It is not using my dataset during training instead of this it is downloading its own dataset in that folder and using it.

Answer 1

1. The example of dataset is https://github.com/deepmipt/DeepPavlov/blob/f5117cd9ad1e64f6c2d970ecaa42fc09ccb23144/deeppavlov/dataset_readers/squad_dataset_reader.py#L46 Your dataset should have the same format.

2-3. The dataset should be placed in the folder https://github.com/deepmipt/DeepPavlov/blob/f5117cd9ad1e64f6c2d970ecaa42fc09ccb23144/deeppavlov/configs/squad/squad_torch_bert.json#L4 (you can change the folder name)

4. Model is saved in the directory https://github.com/deepmipt/DeepPavlov/blob/f5117cd9ad1e64f6c2d970ecaa42fc09ccb23144/deeppavlov/configs/squad/squad_torch_bert.json#L166 (here you can write your own directory)

5. Trained model can be used with the command: python3 -m deeppavlov interact <your_config_name> More detailed tutorial how to launch models is here https://github.com/deepmipt/DeepPavlov