The Situation
I have a video file that I want to process in nodejs, but I want to run it through ffmpeg first in order to normalize the encoding. As I get the data / for a given data event I would like to be able to know how "far" into the video the piped stream has progressed, at as close to frame-level granularity as possible.
0.1 second granularity would be fine.
The Code
I am using Nodejs to invoke ffmpeg to take the video file path and then output the data to stdout:
const ffmpegSettings = [
'-i', './path/to/video.mp4', // Ingest a file
'-f', 'mpegts', // Specify the wrapper
'-' // Output to stdout
]
const ffmpegProcess = spawn('ffmpeg', ffmpegSettings)
const readableStream = ffmpegProcess.stdout
readableStream.on('data', async (data) => {
readableStream.pause() // Pause the stream to make sure we process in order.
// Process the data.
// This is where I want to know the "timestamp" for the data.
readableStream.resume() // Restart the stream.
})
The Question
How can I efficiently and accurately keep track of how far into the video a given 'data' event represents? Keeping in mind that a given on
event in this context might not even have enough data to represent even a full frame.
Temporary Edit
(I will delete this section once a conclusive answer is identified)
Since posting this question I've done some exploration using ffprobe.
start = () => {
this.ffmpegProcess = spawn('ffmpeg', this.getFfmpegSettings())
this.ffprobeProcess = spawn('ffprobe', this.getFfprobeSettings())
this.inputStream = this.getInputStream()
this.inputStream.on('data', (rawData) => {
this.inputStream.pause()
if(rawData) {
this.ffmpegProcess.stdin.write(rawData)
}
this.inputStream.resume()
})
this.ffmpegProcess.stdout.on('data', (mpegtsData) => {
this.ffmpegProcess.stdout.pause()
this.ffprobeProcess.stdin.write(mpegtsData)
console.log(this.currentPts)
this.ffmpegProcess.stdout.resume()
})
this.ffprobeProcess.stdout.on('data', (probeData) => {
this.ffprobeProcess.stdout.pause()
const lastLine = probeData.toString().trim().split('\n').slice(-1).pop()
const lastPacket = lastLine.split(',')
const pts = lastPacket[4]
console.log(`FFPROBE: ${pts}`)
this.currentPts = pts
this.ffprobeProcess.stdout.resume()
})
logger.info(`Starting ingestion from ${this.constructor.name}...`)
}
/**
* Returns an ffmpeg settings array for this ingestion engine.
*
* @return {String[]} A list of ffmpeg command line parameters.
*/
getFfmpegSettings = () => [
'-loglevel', 'info',
'-i', '-',
'-f', 'mpegts',
// '-vf', 'fps=fps=30,signalstats,metadata=print:key=lavfi.signalstats.YDIF:file=\'pipe\\:3\'',
'-',
]
/**
* Returns an ffprobe settings array for this ingestion engine.
*
* @return {String[]} A list of ffprobe command line parameters.
*/
getFfprobeSettings = () => [
'-f', 'mpegts',
'-i', '-',
'-print_format', 'csv',
'-show_packets',
]
This pipes the ffmpeg output into ffprobe and uses that to "estimate" where in the stream the processing has gotten. It starts off wildly inaccurate, but after about 5 seconds of processed-video ffprobe and ffmpeg are producing data at a similar pace.
This is a hack, but a step towards the granularity I want. It may be that I need an mpegts parser / chunker that can run on the ffmpeg output directly in NodeJS.
The output of the above is something along the lines of:
undefined (repeated around 100x as I assume ffprobe needs more data to start)
FFPROBE: 1.422456 // Note that this represents around 60 ffprobe messages sent at once.
FFPROBE: 1.933867
1.933867 // These lines represent mpegts data sent from ffmpeg, and the latest pts reported by ffprobe
FFPROBE: 2.388989
2.388989
FFPROBE: 2.728578
FFPROBE: 2.989811
FFPROBE: 3.146544
3.146544
FFPROBE: 3.433889
FFPROBE: 3.668989
FFPROBE: 3.802400
FFPROBE: 3.956333
FFPROBE: 4.069333
4.069333
FFPROBE: 4.426544
FFPROBE: 4.609400
FFPROBE: 4.870622
FFPROBE: 5.184089
FFPROBE: 5.337267
5.337267
FFPROBE: 5.915522
FFPROBE: 6.104700
FFPROBE: 6.333478
6.333478
FFPROBE: 6.571833
FFPROBE: 6.705300
6.705300
FFPROBE: 6.738667
6.738667
FFPROBE: 6.777567
FFPROBE: 6.772033
6.772033
FFPROBE: 6.805400
6.805400
FFPROBE: 6.829811
FFPROBE: 6.838767
6.838767
FFPROBE: 6.872133
6.872133
FFPROBE: 6.882056
FFPROBE: 6.905500
6.905500