0

I have used following python code for summarization using Langchain with custom map-reduce method. I have list of chunks of large document. I need to stream final summarization output. But Its streaming summarization of every chunk in map operation as well as reduce operation.

    model_llm = ChatOpenAI(temperature=0.0, model='gpt-4')
    model_llm.streaming = True
    docs = [Document(page_content=txt) for txt in list_chunks]
    chunk_attrib = 'text'
    # replacing attributes in map prompt other than document chunk
    map_prompt = "Summarize document in bullet points in {text}"
    combine_prompt = "Remove redundancy in summary in {text}"

    human_message_prompt = HumanMessagePromptTemplate.from_template(combine_prompt)
    combine_prompt_template = ChatPromptTemplate.from_messages(
        [system_message_prompt, human_message_prompt],
    )
    human_message_prompt_reduce = HumanMessagePromptTemplate.from_template(map_prompt)
    map_prompt_template = ChatPromptTemplate.from_messages(
        [system_message_prompt, human_message_prompt],
    )
    chain = load_summarize_chain(self.model_llm,
                                 chain_type="map_reduce",
                                 map_prompt=map_prompt_template,
                                 combine_prompt=map_prompt_template,
                                 combine_document_variable_name=chunk_attrib,
                                 map_reduce_document_variable_name=chunk_attrib,
                                 input_key=chunk_attrib,
                                 output_key=chunk_attrib
                                 )
    output = chain.run(docs)
CPD
  • 5
  • 2

0 Answers0