How to stub/mock a streamed response from ChatGPT?

Question

In my testing environment I'm trying to intercept my request to ChatGPT (shown below) and replace it with a mock response. The fetch api returns a Response.body as a ReadableStream of content. In my implementation, I use the ReadableStream.getReader() which creates a reader and locks the stream to it. While "some-condition" is true, we await the reader.read() method which returns a series of objects (see the console.log() screenshot below). Their values are decoded and then passed into parser which handles the parsed content which eventually shows up in the UI.

Since directly interacting with the ChatGPT api results in flakey e2e tests, I'm trying to figure out is how to correctly stub/mock the response from /api/chat-stream in a way that is consumable and shaped correctly as to render a fake response in my integration tests. I've shared the snippet of code that ultimately will be used to intercept the request, but I'm stumped on how to correctly implement it.

// this should intercept the request and provide the mock/stub
await page.route(/.*\/api\/chat-stream/, async (route) => {
  await route.fulfill({
    status: 200,
    headers: {
      "some-headers"
    },
    body: someBody,
  });
});

request to chatgpt

// localhost:3000/api/chat-stream

export async function POST(request: Request) {
  const { messages } = await request.json();

  const completion = await fetch("https://api.openai.com/v1/chat/completions", {
    method: "POST",
    body: JSON.stringify({
      model: "gpt-3.5-turbo",
      messages: messages,
      stream: true,
    }),
    headers: {
      "Content-Type": "application/json",
      Authorization: `Bearer ${process.env.NEXT_PUBLIC_OPENAI_API_KEY}`,
    },
  });

  return new Response(completion.body, {
    // completion.body is a readable stream as detailed above
    // ReadableStream { locked: false, state: 'readable', supportsBYOB: false }
    status: 200,
    headers: {
      "Content-Type": "application/json; charset=utf-8",
    },
  });
}

api call that triggers and handles above request

export const handleStreamMessage = async (messages: any) => {
  const response = await fetch("/api/chat-stream", {
    method: "POST",
    body: JSON.stringify({
      messages: messages,
    }),
    headers: {
      "Content-Type": "application/json",
    },
  });

  const reader = response.body?.getReader();
  const decoder = new TextDecoder();

  const onParse: EventSourceParseCallback = (event) => {
    if (event.type === "event") {
      try {
        const data: { choices: { delta: { content: string } }[] } = JSON.parse(
          event.data
        );

        // filter for chatgpt "deltas" with content
        data.choices
          .filter(({ delta }) => !!delta.content)
          .forEach(({ delta }) => {
            // do something in react
            setCurrentMessage((prev) => {
              return `${prev || ""}${delta.content}`;
            });
          });
      } catch (error) {
        console.log("error", error);
      }
    }
  };

  const parser = createParser(onParse);

  if (reader) {
    while (some-condition) {
      const readOperation = await reader.read();
      console.log("readOperation", readOperation);

      const dataString = decoder.decode(readOperation.value);

      if (readOperation.done || dataString.includes("[DONE]")) {
        break;
      }

      parser.feed(dataString);
    }
  }
};

Answer 1

Great news, I solved my problem.

What I ended up doing was copying one of the console logged responses (see above), parsed the object capturing just the values, passed those values into a Uint8Array, and then created a Buffer from that source. That Buffer was then passed into the body of my route.fulfill method and hooray I've now intercepted the chatgpt api response with a valid buffer (that I sourced from them).

// transformation
const rawObject = {
  "0": 100,
  "1": 97,
  "2": 116,
  "3": 97,
  ...
};

const parsedArray = Object.values(rawObject); 
const myBuffer: Buffer = Buffer.from(new Uint8Array(parsedArray));

// playwright route interception
await page.route(/.*\/api\/chat-stream/, async (route) => {
  await route.fulfill({
    status: 200,
    headers: {
      "Content-Type": "application/json; charset=utf-8",
    },
    body: myBuffer,
  });
});