Issues with parallel and transcoding video

Question

Hey guys I'm getting a weird error even though set -x isn't showing me anything noticeable. It looks like my cmds aren't actually getting processed by parallel, or at least that is the assumption. the script below should do the following

find all the contents
push them all through a switch to ensure no rewriting of same formats happen
re-encode them into html5 video formats (webm ogg and mp4 via h264)
move those files

Can someone briefly run this code and let me know what is going on? I'm sure its something simple, but it's escaping me atm...

#!/bin/bash
# Video transcoder

# CPR : Jd Daniel
# MOD : 2014-02-20 @ 10:51:25
# VER : Beta 1

# proto: cd drop ; types=(asf asx avi flv m4v mov mp4 mpg rm swf vob wmv); for i in "${types[@]}"; do touch "video.${i}"; done;

inp="drop"
out="export"

## test for reqs of exit
for requires in ffmpeg parallel; do
    hash $requires 2>/dev/null || { 
        echo >&2 "I require $requires to run but it's not installed.  Aborting."; exit 1; 
    }
done


## dectypes
declare -r fpath=$( cd "$(dirname "$0")" ; pwd -P )
declare -r types="asf\|asx\|avi\|flv\|m4v\|mov\|mp4\|mpg\|ogg\|rm\|swf\|vob\|webm\|wmv"
declare -r allow=( mp4 ogg webm )
##

    for dir in "$inp" "$out"; do
        [ -d "$fpath/$dir" ] || { mkdir -p "$fpath/$dir"; } # create if not available
    done

        [ -z "$(ls $fpath/$inp)" ] && { "$fpath is empty, finished..."; exit 0; } # exit when empty


    ## start video conversion
    cd $fpath ; for video in $(find . -type f -iregex ".*\(${types}\)" -printf '%P\0 '); do

        skip=0 #reset skip flag 
        name=$(echo $video |awk -NF '.' '{print $1}')
        exts=$(echo $video |awk -NF '.' '{print $2}')

        # assign predetermined pos
        for (( i = 0; i < ${#allow[@]}; i++ )); do
           if [ "${allow[$i]}" = "${exts}" ]; then
               skip=$(($i + 1));
           fi
        done

        cmds=() # empty
        [[ $skip = 1 ]] || { cmds+=("'$video' -b 1500k -vcodec libx264   -vpre   slow      -vpre baseline -g 30          '$name.mp4' ") ; }
        [[ $skip = 2 ]] || { cmds+=("'$video' -b 1500k -vcodec libvpx    -acodec libvorbis -ab   160000   -f webm -g 30  '$name.webm'") ; }
        [[ $skip = 3 ]] || { cmds+=("'$video' -b 1500k -vcodec libtheora -acodec libvorbis -ab   160000   -g 30          '$name.ogg' ") ; }

        echo "${cmds[@]}" | parallel --gnu -j10 ffmpeg -i $fpath/$inp/{/}\;

        for ext in "${allow[@]}"; do
            [ -a "${fpath}/${inp}/${name}.{ext}" ] && mv "${fpath}/${inp}/${name}.{ext}" "${fpath}/${out}/"
        done

    done

A link to the repo can be found here transcoder. Please feel free to use this on YOU OWN PROJECTS!!!! Who knows how it can come in handy, but i'd love to see it shared ;)

@NevikRehnel sorry about the GIT tag, i think it asked due to me specifying the repo in there =P silly me for not paying attention to my tagging — ehime, Feb 24 '14 at 17:21
Are you sure that you wanted variables like `$video` and `$name.ext` within __single-quotes__? — devnull, Feb 24 '14 at 17:37
Yeah that was a sticky point I saw that I was getting something like `ffmpeg -i /path/to/'drop.mp4'` after removal though I am still getting the missing video shenanigans — ehime, Feb 24 '14 at 18:02

score 2 · Accepted Answer · answered Feb 25 '14 at 00:36

Ended up with this, no longer uses parallel but still does background tasks

#!/bin/bash
# Video transcoder

# CPR : Jd Daniel :: Ehime-ken
# MOD : 2014-02-24 @ 16:34:29
# VER : Beta 3


inp="drop"
out="export"

clear # set -x

## test for reqs of exit
for requires in ffmpeg parallel; do
    hash $requires 2>/dev/null || {
        echo >&2 "I require $requires to run but it's not installed.  Aborting."; exit 1;
    }
done


## dectypes
declare -r fpath=$( cd "$(dirname "$0")" ; pwd -P )
declare -r types="asf\|asx\|avi\|flv\|m4v\|mkv\|mov\|mp4\|mpg\|ogg\|rm\|swf\|vob\|webm\|wmv"
declare -r allow=( mp4 ogg webm )
##

    for dir in "$inp" "$out"; do
        [ -d "$fpath/$dir" ] || { mkdir -p "$fpath/$dir"; } # create if not available
    done

    [ -z "$(ls $fpath/$inp)" ] && { "$fpath is empty, finished..."; exit 0; } # exit when empty


    ## start video conversion
    cd $fpath ; while IFS= read -r video; do

        skip=0 #reset skip flag
        name="${video%.*}"
        exts="${video##*.}"

        # assign predetermined pos
        for (( i = 0; i < ${#allow[@]}; i++ )); do
           if [ "${allow[$i]}" = "${exts}" ]; then
               skip=$(($i + 1));
           fi
        done


        pids=() # pid track if we want to use it later for something
        [[ $skip = 1 ]] || { ffmpeg -y -i "$inp/$video" -vb 1500k -vcodec libx264 -vpre slow -vpre baseline -g 30 "$out/${name}.mp4" 2> /dev/null & } # \n pid+="$! " # if we're going to do post proc kill work later
        [[ $skip = 2 ]] || { ffmpeg -y -i "$inp/$video" -vb 1500k -vcodec libvpx -acodec libvorbis -ab 160000 -f webm -g 30 "$out/${name}.webm" 2> /dev/null & }
        [[ $skip = 3 ]] || { ffmpeg -y -i "$inp/$video" -vb 1500k -vcodec libtheora -acodec libvorbis -ab 160000 -g 30 "$out/${name}.ogg" 2> /dev/null & }


        # move out of path
        for ext in "${allow[@]}"; do
            [ -a "$inp/$name.$ext" ] && cp -f "$inp/$name.$ext" "$out/" &
        done

        for job in `jobs -p`; do
            echo "Job: $job"
            wait $job || let "FAIL+=1"
        done

        [ "0" == "$FAIL" ] && echo "YAY!" || echo "FAIL! ($FAIL)"

    done <<< $(find "$fpath/$inp" -type f -iregex ".*\(${types}\)" -printf '%P\0 ')

Issues with parallel and transcoding video

1 Answers1