Background:
I am creating a personal music bot for Discord, mainly using discord.js, and one of the features I'd like to add is having the bot play audio directly from Voicemeeter. This solution is optimal because bots are able to stream in much better quality than Discord users are, and they are able to play stereo sound instead of mono, and I can choose what sources are streamed with Voicemeeter.
Voicemeeter comes with a feature called VBAN which allows you to send audio data over your network using the VBAN protocol to another computer running a VBAN receiver.
Jacob Bower made a VBAN receiver script in Node JS that takes the raw PCM audio data and plays it through your speakers using node-speaker.
Problem:
Using Jacob's script, if you try to pipe the PCM audio directly into the Discord bot, it gives the error "Error [VOICE_PLAY_INTERFACE_BAD_TYPE]: Unknown stream type." I believe this is because the PCM audio received from the script is contained in a Buffer, which the discord bot cannot play natively. I need to find a way to get that PCM audio in a readable format for the discord bot to play. (Readable formats I believe include opus and wav, (and possibly other streaming formats that aren't PCM))
What I have tried:
- As stated above, I directly piped the PCM audio buffers into the bot, and it gives that error.
- I attempted putting the buffers into a PassThrough Stream, and piping that stream into the bot. This is met with no errors in console, the bot joining the voice channel like it should, and emitting no sound. I think this is because discord.js does not support playing raw PCM audio which I learned after the fact.
- The Discord.js guide mentions that PCM needs to be encoded with opus to be able to play over discord. So I tried using the opus encoder built into discord.js. First I tried encoding each PCM buffer and writing that into a PassThrough stream.
const { OpusEncoder } = require('@discordjs/opus');
const { PassThrough } = require('stream');
const EncodedStream = new PassThrough();
const encoder = new OpusEncoder(48000, 2); //48000 hz, 2 channels
. . .
if (data.header.sp == 0) { //This code executes each time there is a new frame sent from VBAN
EncodedStream.write(encoder.encode(data.audio));
}
. . .
connection.play(EncodedStream); //Connection is just the Bot being inside a voice channel.
That was met with the error "Cannot create a Buffer larger than 0xffffffff bytes." I'm not sure why that error came up, because running Buffer.byteLength on each of the buffers yielded 1024.
- I then moved on to trying to use prism-media to encode the stream where found more errors and more of the bot remaining silent.
- I installed some npm modules that claimed they could convert streams to AudioBuffers, and another module that would convert the PCM AudioBuffer to a usable wav format, but I found more errors there too.
I am at my wits end right now trying to solve this problem, and any help pointing me in the right direction would be much appreciated.
