0

I've seen examples using the Vercel ai library to quickly create a streamable OpenAI response type (e.g. https://stackoverflow.com/a/76634615/5348578), however, I'd like to do data modification prior to sending a response.

Is this possible using the libraries? If not, what is the correct approach?

Currently, I'm attempting to do the following without the use of the aforementioned libraries:

const completionMessage = await openai.createChatCompletion({
        model: 'gpt-4',
        stream: true,
        messages: conversation,
    });


const transformStream = new Transform({
    transform(chunk, encoding, callback) {
        const stringData = chunk.toString();

        // Parse the Server-Sent Event
        const eventData = stringData.replace('data: ', '');
        /*
            If `stream` is set, partial message deltas will be sent. Tokens will be sent as data- 
            only server-sent events as they become available, with the stream terminated by a data: 
            [DONE] message.
        */
        if (eventData === '[DONE]') {
            return callback();
        }

        // Parse the data
        const data = JSON.parse(eventData);
        const message = data.choices[0].message?.content;

        const newData = {
            type: 'chat',
            data: {
                role: 'assistant',
                content: message,
            },
            myProperty: false,
        };

        this.push(JSON.stringify(newData));
        callback();
    },
});

const responseStream = completionMessage.body;

if (responseStream === null) {
    throw new Error('Response stream is null');
}

const writableStream = new WritableStream({
    write(chunk) {
        transformStream.write(chunk);
    },
    close() {
        transformStream.end();
    },
    abort(err) {
        transformStream.destroy(err);
    },
});

responseStream.pipeTo(writableStream);

return new NextResponse(transformStream);

However, this approach does not seem to work as the transformStream is not a valid type to assign to NextResponse.

DeveloperRyan
  • 837
  • 2
  • 9
  • 22

0 Answers0