Questions tagged [multihead-attention]

I'm trying to make a hybrid binary text classification model using a multi-head attention mechanism with CNN-LSTM. However, I'm facing an issue when trying to pass the values obtained from CNN-LSTM to the attention layer. This was what I…

I have a NLP task/idea in mind, where the input and output text is structured purely hierarchically, like multi-level bulletpoint lists or a table of content. The question is: Is there any resaetch for this particular type of text for transformer…

How do I quantify the attention between input and output sentences in a sequence-to-sequence language modelling scenario [translation or summarization]? For instance, consider these input and output statements, i.e., document is the input, and…