0

I tried to use allennlp.predictors.Predictor.get_gradients to get the gradients of an instance.

allennlp==0.8.5

The main code:

for instance in targeted_dev_data: 
    #model is locally trained on GPU
    #instance is from allennlp.data.dataset_readers.reader 
    predictor = Predictor(model,  
                    StanfordSentimentTreeBankDatasetReader(granularity="2-class",
                                                    token_indexers={"tokens": single_id_indexer},
                                                    use_subtrees=True))
    
    input_gradients = predictor.get_gradients([instance])

But I got the following error:

File "sst_sememe.py", line 436, in main
    input_gradients = predictor.get_gradients([instance])
File "/home/rui.ye/universal-triggers-master/myenv/lib/python3.7/site-packages/allennlp/predictors/predictor.py", line 110, in get_gradients
    outputs = self._model.decode(self._model.forward(**dataset.as_tensor_dict()))
File "sst_sememe.py", line 59, in forward
    embeddings = self.word_embeddings(tokens)
File "/home/rui.ye/universal-triggers-master/myenv/lib/python3.7/site-packages/torch/nn/modules/module.py", line 493, in __call__
    result = self.forward(*input, **kwargs)
File "/home/rui.ye/universal-triggers-master/myenv/lib/python3.7/site-packages/allennlp/modules/text_field_embedders/basic_text_field_embedder.py", line 131, in forward
    token_vectors = embedder(*tensors, **forward_params_values)
File "/home/rui.ye/universal-triggers-master/myenv/lib/python3.7/site-packages/torch/nn/modules/module.py", line 493, in __call__
    result = self.forward(*input, **kwargs)
File "/home/rui.ye/universal-triggers-master/myenv/lib/python3.7/site-packages/allennlp/modules/token_embedders/embedding.py", line 144, in forward
    sparse=self.sparse)
File "/home/rui.ye/universal-triggers-master/myenv/lib/python3.7/site-packages/torch/nn/functional.py", line 1506, in embedding
    return torch.embedding(weight, input, padding_idx, scale_grad_by_freq, sparse)
RuntimeError: Expected object of backend CUDA but got backend CPU for argument #3 'index'

However, I can pass the Predictor.predict_instance test with the following code

for instance in targeted_dev_data: 
    #model is locally trained on GPU
    #instance is from allennlp.data.dataset_readers.reader 
    predictor = Predictor(model,  
                    StanfordSentimentTreeBankDatasetReader(granularity="2-class",
                                                    token_indexers={"tokens": single_id_indexer},
                                                    use_subtrees=True))
    
    input_preds = predictor.predict_instance(instance)
RyleeRuiYe
  • 1
  • 2
  • Can you add what version of AllenNLP you are using? The line numbers in the stack trace don't align with my copy of the code. – Dirk Groeneveld Jan 14 '21 at 23:07
  • In fact, I think I need a more complete example. Can you write something that I can copy and paste into a Python console to reproduce the error? I can't make heads or tails of the information given. Parameter #3 to `torch.embedding` is a Python `int`, so it does not have a device. – Dirk Groeneveld Jan 14 '21 at 23:11
  • Thank you for your quick answer. I solved it (partly) by replacing Predictor(model, *) with Predictor(model.cpu(), *), where the 'model' was locally trained by GPU. It indicated that the error was caused by device unmatch between Predictor(default on GPU in my case) and instance(on CPU). Therefore, here is my followup question, do you know how to put the instance in allennlp to GPU since instance.cuda() did not work? – RyleeRuiYe Jan 15 '21 at 10:26

1 Answers1

0

I solved it (partly) by replacing Predictor(model, *) with Predictor(model.cpu(), *), where the 'model' was locally trained by GPU.

for instance in targeted_dev_data: 
    #model is locally trained on GPU
    #instance is from allennlp.data.dataset_readers.reader 
    predictor = Predictor(model.cpu(),  
                    StanfordSentimentTreeBankDatasetReader(granularity="2-class",
                                                    token_indexers={"tokens": single_id_indexer},
                                                    use_subtrees=True))
    
    input_gradients = predictor.get_gradients([instance])

It indicated that the error was caused by device unmatch between Predictor(default on GPU in my case) and instance(on CPU). Therefore, here is my follow-up question, does anyone know how to put the instance in allennlp to GPU since instance.cuda() did not work?

RyleeRuiYe
  • 1
  • 2