1

thank you for your looking for me. I'm going to implement Streaming process in langchain, but I can't display tokenized message in frontend. This is my code:

def generate_message(query, history, behavior, temp, chat):
    # load_dotenv()

    template = """{behavior}

    Training data: {examples}

    Chathistory: {history}
    Human: {human_input}
    Assistant:"""

    prompt = PromptTemplate(
        input_variables=["history", "examples", "human_input", "behavior"], template=template)

    langchain.llm_cache = GPTCache(init_gptcache)
    llm = ChatOpenAI(model_name="gpt-3.5-turbo",                  
                     temperature=temp,                    
                     openai_api_key=OPENAI_API_KEY,
                     streaming=True,
                     callbacks=[MyCallbackHander()])

    conversation = LLMChain(
        llm=llm,
        verbose=True,
        prompt=prompt
    )
    embeddings = OpenAIEmbeddings(openai_api_key=OPENAI_API_KEY)
    docsearch = Pinecone.from_existing_index(
        index_name=PINECONE_INDEX_NAME, namespace = PINECONE_NAMESPACE, embedding=embeddings)
    _query = query
    docs = docsearch.similarity_search(query=_query, k=10)

    examples = ""
    for doc in docs:
        # if doc.metadata['chat'] == str(chat):
            doc.page_content = doc.page_content.replace('\n\n', ' ')
            examples += doc.page_content + '\n'

    response = conversation.run(
        human_input=query,
        history=history,
        behavior=behavior,
        examples=examples
    )

    token_generator(MyCallbackHander())

Prompt: Write me a song about sparkling water. Answer: Verse 1: Bubbles rising to the top A refreshing drink that never stops Clear and crisp, it's pure delight A taste that's sure to excite

Chorus:
Sparkling water, oh so fine
A drink that's always on my mind
With every sip, I feel alive
Sparkling water, you're my vibe

Verse 2:
No sugar, no calories, just pure bliss
A drink that's hard to resist
It's the perfect way to quench my thirst
A drink that always comes first

Chorus:
Sparkling water, oh so fine
A drink that's always on my mind
With every sip, I feel alive
Sparkling water, you're my vibe

Bridge:
From the mountains to the sea
Sparkling water, you're the key
To a healthy life, a happy soul
A drink that makes me feel whole

Chorus:
Sparkling water, oh so fine
A drink that's always on my mind
With every sip, I feel alive
Sparkling water, you're my vibe

Outro:
Sparkling water, you're the one
A drink that's always so much fun
I'll never let you go, my friend
Sparkling
Andy Wood
  • 11
  • 4
  • Please clarify your specific problem or provide additional details to highlight exactly what you need. As it's currently written, it's hard to tell exactly what you're asking. – Community Aug 14 '23 at 20:46
  • Maybe this can help [PrivateDocBot](https://github.com/Abhi5h3k/PrivateDocBot) It does stream using langchain. Just like ChatGpt it displays word by word and works locally on PDF data. – Abhi Aug 19 '23 at 07:13

0 Answers0