Streaming response line chatgpt

Question

Does anyone know if I can display chatgpt-like streaming response in Streamlit using streamlit_chat message?

I need something like message(streaming=True) or any other alternative for this. my code segment is as below:

from streamlit_chat import message
import streamlit as st

for i in range(len(st.session_state['generated']) - 1, -1, -1):
      message(st.session_state['past'][i], is_user=True, key=str(i) + '_user')
      message(st.session_state["generated"][i], key=str(i))`

I expect the response streaming like chatgpt on steamlit app

score 0 · Answer 1 · answered Jun 07 '23 at 22:06

I am not sure about the streaming strategy within Streamlit but here is a generic hack logic for client-side streaming in case helpful,

import asyncio, shlex, subprocess, sys
async def subprocess_async(cmd, **kwargs):
    cmd_list = shlex.split(cmd)
    proc = await asyncio.create_subprocess_exec(
        *cmd_list,
        stdout=asyncio.subprocess.PIPE,
        stderr=asyncio.subprocess.STDOUT,
        **kwargs)

    answer = ""
    while proc.returncode is None:
        buf = await proc.stdout.read(4)
        if not buf:
            return
        buf = buf.decode("utf-8")
        print(buf, end="")
        sys.stdout.flush()
        answer += buf
    res = subprocess.CompletedProcess(cmd, proc.returncode,  stdout=answer, stderr=b'')
    return res

X = """Python is a high-level, interpreted, general-purpose programming language. It is a powerful and versatile language that is used for a wide range of applications, from web development to data science. Python is known for its readability and ease of use, making it a great language for beginners. It also has a large and active community of developers contributing to the language and its libraries. """

asyncio.create_task(subprocess_async(f"echo {X}"))

Streaming response line chatgpt

1 Answers1