0

LLMs are forward generating models, generating the next word based on the previous context without having known the future words. This being the case the model shouldn't be able to know the difference between is a and is an grammatically as this involves knowing the second word after is. Please let me know if my knowledge is incorrect.

KaranJ
  • 48
  • 1
  • 10
  • 1
    LLMs generally generate a token instead of a character. Try forcing an LLM to generate some text about `apple` with starting letter `a`. It would try to come up with a clever solution like `a ripe apple ...`. That LLM, based on context knows that it will generate `apple` at some point and if it generated an `an`, the next word propably should be `apple`. That's my intuition but I'm curious about the answer too so I'll stick around. :^) – doneforaiur Jul 02 '23 at 08:39
  • That's my intuition as well. It will need to know what is the subject of the conversation beforehand from the context before it gets the next token to generate after is. However, there might be another solution that I think could be generating next multiple tokens summing up to two words at once. Taking care of 'an apple' or 'a ripe' at once rather than multiple passes. – KaranJ Jul 02 '23 at 09:03
  • Welp! GPT knows and I think it is predicting multiple tokens at once. [link](https://imgur.com/a/aX35th4) **This Response** – KaranJ Jul 02 '23 at 09:08
  • What is the difference between "generating multiple tokens" and "multiple passes"? If the LLM predicted a token, it have to add it to the previous context and predict the token after that. Just found out this; https://ai.stackexchange.com/questions/40086/has-anyone-tried-to-train-a-gpt-model-predicting-the-next-n-tokens-instead-of-th Never thought of the computational cost of the training. – doneforaiur Jul 02 '23 at 09:11
  • 1
    You can't be sure that it did not "understood" that you were referring is an apple based on `doctor, red fruit` etc. What would be the result if you prompted it with `I think it is a`? By the way, you should use OpenAI's playground instead of ChatGPT because there's an `user` and `assistant` dynamic which is not true "completion". – doneforaiur Jul 02 '23 at 09:14
  • @doneforaiur LLMS don't necessarily have to predict only the next token. The way transformers layers are designed is that you can have a different size of window output than the input. so if you carefully train the model you can generate up to n next tokens based on the model architecture. Multiple passes would mean sending the context and adding the new token in the same context. which as you mentioned is computationally more expensive. – KaranJ Jul 02 '23 at 09:14
  • 1
    @doneforaiur thanks for the suggestion will try this out right now. I like tripping GPTs :D – KaranJ Jul 02 '23 at 09:16
  • I think you guys might be correct it is only generating one token at a time and not multiple token. It cannot adjust the context provided based on the output either. Here is the [link](https://imgur.com/a/MFx80lX) *latest one* Also I cannot use the playground, lost my free credits. So will try this as work tomorrow on their APIs :D. Also the completion is on ADA(if I'm not wrong). and chat completions are on 3.5 and 4. Will try to trip them out tomorrow as well. – KaranJ Jul 02 '23 at 09:24

0 Answers0