1

The Situation

I have a video file that I want to process in nodejs, but I want to run it through ffmpeg first in order to normalize the encoding. As I get the data / for a given data event I would like to be able to know how "far" into the video the piped stream has progressed, at as close to frame-level granularity as possible.

0.1 second granularity would be fine.

The Code

I am using Nodejs to invoke ffmpeg to take the video file path and then output the data to stdout:

const ffmpegSettings = [
    '-i', './path/to/video.mp4',       // Ingest a file
    '-f', 'mpegts',                    // Specify the wrapper
    '-'                                // Output to stdout
]
const ffmpegProcess = spawn('ffmpeg', ffmpegSettings)
const readableStream = ffmpegProcess.stdout
readableStream.on('data', async (data) => {
    readableStream.pause() // Pause the stream to make sure we process in order.

    // Process the data.
    // This is where I want to know the "timestamp" for the data.

    readableStream.resume() // Restart the stream.
})

The Question

How can I efficiently and accurately keep track of how far into the video a given 'data' event represents? Keeping in mind that a given on event in this context might not even have enough data to represent even a full frame.


Temporary Edit

(I will delete this section once a conclusive answer is identified)

Since posting this question I've done some exploration using ffprobe.

    start = () => {
        this.ffmpegProcess = spawn('ffmpeg', this.getFfmpegSettings())
        this.ffprobeProcess = spawn('ffprobe', this.getFfprobeSettings())
        this.inputStream = this.getInputStream()

        this.inputStream.on('data', (rawData) => {
            this.inputStream.pause()
            if(rawData) {
                this.ffmpegProcess.stdin.write(rawData)
            }
            this.inputStream.resume()
        })

        this.ffmpegProcess.stdout.on('data', (mpegtsData) => {
            this.ffmpegProcess.stdout.pause()
            this.ffprobeProcess.stdin.write(mpegtsData)
            console.log(this.currentPts)
            this.ffmpegProcess.stdout.resume()
        })

        this.ffprobeProcess.stdout.on('data', (probeData) => {
            this.ffprobeProcess.stdout.pause()
            const lastLine = probeData.toString().trim().split('\n').slice(-1).pop()
            const lastPacket = lastLine.split(',')
            const pts = lastPacket[4]
            console.log(`FFPROBE: ${pts}`)
            this.currentPts = pts
            this.ffprobeProcess.stdout.resume()
        })

        logger.info(`Starting ingestion from ${this.constructor.name}...`)
    }

    /**
     * Returns an ffmpeg settings array for this ingestion engine.
     *
     * @return {String[]} A list of ffmpeg command line parameters.
     */
    getFfmpegSettings = () => [
        '-loglevel', 'info',
        '-i', '-',
        '-f', 'mpegts',
        // '-vf', 'fps=fps=30,signalstats,metadata=print:key=lavfi.signalstats.YDIF:file=\'pipe\\:3\'',
        '-',
    ]

    /**
     * Returns an ffprobe settings array for this ingestion engine.
     *
     * @return {String[]} A list of ffprobe command line parameters.
     */
    getFfprobeSettings = () => [
        '-f', 'mpegts',
        '-i', '-',
        '-print_format', 'csv',
        '-show_packets',
    ]

This pipes the ffmpeg output into ffprobe and uses that to "estimate" where in the stream the processing has gotten. It starts off wildly inaccurate, but after about 5 seconds of processed-video ffprobe and ffmpeg are producing data at a similar pace.

This is a hack, but a step towards the granularity I want. It may be that I need an mpegts parser / chunker that can run on the ffmpeg output directly in NodeJS.

The output of the above is something along the lines of:

undefined (repeated around 100x as I assume ffprobe needs more data to start)
FFPROBE: 1.422456 // Note that this represents around 60 ffprobe messages sent at once.
FFPROBE: 1.933867
1.933867 // These lines represent mpegts data sent from ffmpeg, and the latest pts reported by ffprobe
FFPROBE: 2.388989
2.388989
FFPROBE: 2.728578
FFPROBE: 2.989811
FFPROBE: 3.146544
3.146544
FFPROBE: 3.433889
FFPROBE: 3.668989
FFPROBE: 3.802400
FFPROBE: 3.956333
FFPROBE: 4.069333
4.069333
FFPROBE: 4.426544
FFPROBE: 4.609400
FFPROBE: 4.870622
FFPROBE: 5.184089
FFPROBE: 5.337267
5.337267
FFPROBE: 5.915522
FFPROBE: 6.104700
FFPROBE: 6.333478
6.333478
FFPROBE: 6.571833
FFPROBE: 6.705300
6.705300
FFPROBE: 6.738667
6.738667
FFPROBE: 6.777567
FFPROBE: 6.772033
6.772033
FFPROBE: 6.805400
6.805400
FFPROBE: 6.829811
FFPROBE: 6.838767
6.838767
FFPROBE: 6.872133
6.872133
FFPROBE: 6.882056
FFPROBE: 6.905500
6.905500
slifty
  • 13,062
  • 13
  • 71
  • 109
  • 1
    a) your command is transcoding, since codec option `-c` is *not* specified. b) see the ffmpeg option `-progress` to track output stats in a file which you can then `tail -F`. – Gyan May 19 '20 at 19:30
  • Re: transcoding whoops (I'll correct the question)! Re: progress, unfortunately -progress does not provide enough granularity for my purposes -- e.g. I'm looking for an update every 1 to 3 frames. – slifty May 20 '20 at 00:54
  • @Gyan I updated the question to show an (ugly hackish) example of the kind of granularity I'm trying to get at! Thank you so much for engaging by the way. – slifty May 20 '20 at 01:29
  • ffmpeg updates its stats line 0.5 seconds (same as -progress) and ffmpeg holds output in a buffer and flushes to output periodically so depending on output bitrate you still have a interval floor on when you can expect output to update. – Gyan May 20 '20 at 05:16

0 Answers0