thank you for your looking for me. I'm going to implement Streaming process in langchain, but I can't display tokenized message in frontend. This is my code:
def generate_message(query, history, behavior, temp, chat):
# load_dotenv()
template = """{behavior}
Training data: {examples}
Chathistory: {history}
Human: {human_input}
Assistant:"""
prompt = PromptTemplate(
input_variables=["history", "examples", "human_input", "behavior"], template=template)
langchain.llm_cache = GPTCache(init_gptcache)
llm = ChatOpenAI(model_name="gpt-3.5-turbo",
temperature=temp,
openai_api_key=OPENAI_API_KEY,
streaming=True,
callbacks=[MyCallbackHander()])
conversation = LLMChain(
llm=llm,
verbose=True,
prompt=prompt
)
embeddings = OpenAIEmbeddings(openai_api_key=OPENAI_API_KEY)
docsearch = Pinecone.from_existing_index(
index_name=PINECONE_INDEX_NAME, namespace = PINECONE_NAMESPACE, embedding=embeddings)
_query = query
docs = docsearch.similarity_search(query=_query, k=10)
examples = ""
for doc in docs:
# if doc.metadata['chat'] == str(chat):
doc.page_content = doc.page_content.replace('\n\n', ' ')
examples += doc.page_content + '\n'
response = conversation.run(
human_input=query,
history=history,
behavior=behavior,
examples=examples
)
token_generator(MyCallbackHander())
Prompt: Write me a song about sparkling water. Answer: Verse 1: Bubbles rising to the top A refreshing drink that never stops Clear and crisp, it's pure delight A taste that's sure to excite
Chorus:
Sparkling water, oh so fine
A drink that's always on my mind
With every sip, I feel alive
Sparkling water, you're my vibe
Verse 2:
No sugar, no calories, just pure bliss
A drink that's hard to resist
It's the perfect way to quench my thirst
A drink that always comes first
Chorus:
Sparkling water, oh so fine
A drink that's always on my mind
With every sip, I feel alive
Sparkling water, you're my vibe
Bridge:
From the mountains to the sea
Sparkling water, you're the key
To a healthy life, a happy soul
A drink that makes me feel whole
Chorus:
Sparkling water, oh so fine
A drink that's always on my mind
With every sip, I feel alive
Sparkling water, you're my vibe
Outro:
Sparkling water, you're the one
A drink that's always so much fun
I'll never let you go, my friend
Sparkling