bash script to install / update ffmpeg static builds

Question

I'm trying to make my first "real" bash script to do some real work than can be executed both manually and through cron jobs.

The script should download and install the ffmpeg-static-build from https://johnvansickle.com/ffmpeg

Here is what I've got so far:

#!/bin/bash
# Still to come, see if the script runs with root privileges else exit
#Download FFMPEG static daily build and it's md5
cd ~
wget https://johnvansickle.com/ffmpeg/builds/ffmpeg-git-64bit-static.tar.xz | tar -xf
#wget https://johnvansickle.com/ffmpeg/builds/ffmpeg-git-64bit-static.tar.xz.md5
curl -L https://johnvansickle.com/ffmpeg/builds/ffmpeg-git-64bit-static.tar.xz.md5 | tar -xf | tee -a https://johnvansickle.com/ffmpeg/builds/ffmpeg-git-64bit-static.tar.xz | cut -d\  -f1 | md5sum > md5sum.txt


#Chech the md5 is currect
#md5sum -c MD5SUMS
file1="md5sum.txt"
file2="ffmpeg-git-64bit-static.tar.xz.md5"

if ! cmp --silent "$file1" "$file2"; then
    echo "md5sum doesn't match...\n exit" >&2
    exit 1
fi


#tar -xf ffmpeg-*.tar.xz
cp ffmpeg-*-static/ff* /usr/bin/
cp ffmpeg-*-static/ff* /usr/local/bin/
cp ffmpeg-*-static/qt-faststart /usr/bin/
cp ffmpeg-*-static/qt-faststart /usr/local/bin/
# Consider change second location to use ln -s
# Remove downloads and do some clean up

rm -fr ffmpeg-*

#EOF

As you can see in the script i would like to add the md5sum check but it fails (got the md5sum check part from Compare md5 sums in bash script)

If i remove the md5sum part the script is working, but right now the script fails at the | tar -xf | part with this code

2017-02-28 00:10:24 (617 KB/s) - ‘ffmpeg-git-64bit-static.tar.xz’ saved [17564756/17564756]

tar: option requires an argument -- 'f'
Try 'tar --help' or 'tar --usage' for more information.
tee: 'https://johnvansickle.com/ffmpeg/builds/ffmpeg-git-64bit-static.tar.xz': No such file or directory
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100    65  100    65    0     0     87      0 --:--:-- --:--:-- --:--:--    87
(23) Failed writing body
md5sum doesn't match...
 exit

As this is my first, I would appreciate any "I'm 4 years old" advices and why

update

Have made some changes to the script based on the suggestions in this thread, but my problem is I have no output what so ever at this state :( So I think it's time to ask for the next clue

#!/bin/bash

# version 0.3z

## Download FFMPEG static daily build and it's md5
## 
## To make this script working you might need to change the below values
## based on whether you are using a 32bit or 64 bit OS
## to obtain the right links you have to have a look @ https://johnvansickle.com/ffmpeg/
## and then change those below
## 
## If you are running on a shared server or dowsn't have root priviliges you might need to uncomment
## point 5.


# a "~" means running users home folder :) and should be different from destination dir
download_dir=~

# as this can change if the ffmpeg is to be stored on ex. shared server
dest_dir=/usr/bin/

# The version it equal the filename from above link
version=ffmpeg-git-64bit-static.tar.xz

## Do not change anything below here!!
source_url=https://johnvansickle.com/ffmpeg/builds/${version}
md5_url=https://johnvansickle.com/ffmpeg/builds/${version}.md5

# Still to come, see if the script runs with write privileges else exit

# 1. Can we enter download directory else exit
cd ${download_dir} ||

        printf "%s\n" "You can't enter this folder.\nPlease chage download_dir in this script..." >&2
        exit 1

# 2. Check the md5 is correct or have changed from last check
## instead of downloading ffmpeg-git-64bit-static.tar.xz every time,
## regardless if it is new or not, I recommend only downloading it
## if the md5 does not match. Would save John some bandwidth at least
## thx for the idea to @LordNeckbeard



## So somehow i'll guess some sed or grep only first part is nessesary :(
## This is getting more advance than expected for a first time script :/

if ! diff <(md5sum ${version}) <(curl -s ${md5_url})

    then
        printf "%s\n" "md5sum doesn't match...\n
                        Downloading new version" >&2
        rm -f ${version} >&2
        curl ${source_url} -o ${download_dir}/${version} >&2
        #exit 2

    elif
        diff <(md5sum ${version}) <(curl -s ${md5_url})
        printf "%s\n" "Nothing new to download" >&2
      exit 3
fi

# 3. untar
tar -xf ffmpeg-git-*-static.tar.xz

# 4. Move builds to destination directories

mv ${download_dir}/ffmpeg-*-static/ff* ${dest_dir}/
mv ${download_dir}/ffmpeg-*-static/qt-faststart ${dest_dir}/

# 5. Make soft links to static builds
ln -sfn ${dest_dir}/qt-faststart /usr/local/bin/qt-faststart
ln -sfn ${dest_dir}/ffmpeg /usr/local/bin/ffmpeg
ln -sfn ${dest_dir}/ffmpeg-10bit /usr/local/bin/ffmpeg-10bit
ln -sfn ${dest_dir}/ffprobe /usr/local/bin/ffprobe
ln -sfn ${dest_dir}/ffserver /usr/local/bin/ffserver

# Remove unzipped folder to do some clean up

rm -fr ffmpeg-git-*-static/

#EOF

note: do to some more in depth answering of why not compile from source: 1. This precompiled is usable on all Linux variations, despite distro and version 2. It usable on shared hosting servers with ssh access

update 2

#!/bin/bash

# version 0.4z

## Download FFMPEG static daily build and it's md5
## 
## To make this script working you might need to change the below values
## based on whether you are using a 32bit or 64 bit OS
## to obtain the right links you have to have a look @ https://johnvansickle.com/ffmpeg/
## and then change those below
## 
## Finished and working script should be distributed under GPLv3: free as in freedom
## 
## If you are running on a shared server or dowsn't have root priviliges you might need to uncomment
## point 5.


# a "~" means running users home folder :) and should be different from destination dir
download_dir=~

# as this can change if the ffmpeg is to be stored on ex. shared server
dest_dir=/usr/bin/

# The version it equal the filename from above link
version=ffmpeg-git-64bit-static.tar.xz

## Do not change anything below here!!
source_url=https://johnvansickle.com/ffmpeg/builds/${version}
md5="curl -s https://johnvansickle.com/ffmpeg/builds/${version}.md5"

# Still to come, see if the script runs with write privileges else exit

# 1. Can we enter download directory else exit
cd ${download_dir} || 
        printf "%s\n" "You can't enter this folder.\nPlease chage download_dir in this script..." >&2
        exit 1

# 2. Check the md5 is correct or have changed from last check
## instead of downloading ffmpeg-git-64bit-static.tar.xz every time,
## regardless if it is new or not, I recommend only downloading it
## if the md5 does not match. Would save John some bandwidth at least
## thx for the idea to @LordNeckbeard

## This is getting more advance than expected for a first time script :/

if diff <(md5sum ${version}) <(${md5})

    then
        printf "%s\n" "No new version availeble" >&2
        exit 1

elif ! diff <(md5sum ${version}) <(${md5})
    then
        rm -f ${version}
        curl ${source_url} > ${version}
        exit 0

        #only proceed if downloaded version match it's md5
        if ! diff <(md5sum ${version}) <(${md5})
            then
            rm -f ${version}
            printf "%s\n" "Downloaded version is damaged, try later\ndamaged version have been deleted" >&2
            exit 1
        fi

            # 3. untar
            tar -xf ffmpeg-git-*-static.tar.xz

            # 4. Move builds to destination directories
            mv ${download_dir}/ffmpeg-*-static/ff* ${dest_dir}/
            mv ${download_dir}/ffmpeg-*-static/qt-faststart ${dest_dir}/

            # 5. Make soft links to static builds
            ln -sfn ${dest_dir}/qt-faststart /usr/local/bin/qt-faststart
            ln -sfn ${dest_dir}/ffmpeg /usr/local/bin/ffmpeg
            ln -sfn ${dest_dir}/ffmpeg-10bit /usr/local/bin/ffmpeg-10bit
            ln -sfn ${dest_dir}/ffprobe /usr/local/bin/ffprobe
            ln -sfn ${dest_dir}/ffserver /usr/local/bin/ffserver

            # Remove unzipped folder to do some clean up
            rm -fr ffmpeg-git-*-static/
            printf "%s\n" "Going to install new version" >&2
            exit 1
fi
#EOF

But still having some issues :(

Running this script returns: a blanc shell, but I've expected one of the printf statements
When I'm trying to narrow down the problem and going back to basic and trying to run the script with only the "if" part it fails with 44: Syntax error: "(" unexpected

running the very same "if" part typed directly into the shell/terminal itself it's all happy!!

if diff <(md5sum ffmpeg-git-64bit-static.tar.xz) <(curl -s "https://johnvansickle.com/ffmpeg/builds/ffmpeg-git-64bit-static.tar.xz.md5"); then printf "%s\n" "No new version availeble" >&2; elif ! diff <(md5sum ffmpeg-git-64bit-static.tar.xz) <(curl -s "https://johnvansickle.com/ffmpeg/builds/ffmpeg-git-64bit-static.tar.xz.md5"); then rm -f ffmpeg-git-64bit-static.tar.xz; curl https://johnvansickle.com/ffmpeg/builds/ffmpeg-git-64bit-static.tar.xz > ffmpeg-git-64bit-static.tar.xz; printf "%s\n" "Going to install new version" >&2; fi

I'm confused

Update 3 It has come to my attention also by @tomas that question 3 was running the script by sh update_ffmpeg.sh ofcurse was wrong it should just be executed from it's locaation by typing the full path to script ex. /home/username/update_ffmpeg.sh

So to share the current working edition (but still incomplete):

#!/bin/bash

# version 0.4z

## Download FFMPEG static daily build and it's md5
## 
## To make this script working you might need to change the below values
## based on whether you are using a 32bit or 64 bit OS
## to obtain the right links you have to have a look @ https://johnvansickle.com/ffmpeg/
## and then change those below
## 
## Finished and working script should be distributed under GPLv3: free as in freedom
## 
## If you are running on a shared server or dowsn't have root priviliges you might need to uncomment
## point 5.


# a "~" means running users home folder :) and should be different from destination dir
download_dir=~

# as this can change if the ffmpeg is to be stored on ex. shared server
dest_dir=/usr/bin/

# The version it equal the filename from above link
version=ffmpeg-git-64bit-static.tar.xz

## Do not change anything below here!!
source_url=https://johnvansickle.com/ffmpeg/builds/${version}
md5_url=https://johnvansickle.com/ffmpeg/builds/${version}.md5

# Still to come, see if the script runs with write privileges else exit

# 1. Can we enter download directory else exit
cd ${download_dir} || {
    printf "%s\n" "You can't enter this folder." "Please change download_dir in this script..." >&2
    exit 1
}

# 2. Check the md5 is correct or have changed from last check
## instead of downloading ffmpeg-git-64bit-static.tar.xz every time,
## regardless if it is new or not, I recommend only downloading it
## if the md5 does not match. Would save John some bandwidth at least
## thx for the idea to @LordNeckbeard

## This is getting more advance than expected for a first time script :/

if ! diff <(md5sum ${version}) <(curl -s ${md5_url})
then
    # Sum doesn't match is not really an error... I comment out the redirection.
    printf "%s\n" "md5sum doesn't match..." "Downloading new version"
    rm -f ${version}
    curl ${source_url} -o ${download_dir}/${version}

        # 3. untar
        tar -xf ffmpeg-git-*-static.tar.xz

        # 4. Move builds to destination directories
        mv ${download_dir}/ffmpeg-*-static/ff* ${dest_dir}/
        mv ${download_dir}/ffmpeg-*-static/qt-faststart ${dest_dir}/

        # 5. Make soft links to static builds
        ln -sfn ${dest_dir}/qt-faststart /usr/local/bin/qt-faststart >&2
        ln -sfn ${dest_dir}/ffmpeg /usr/local/bin/ffmpeg >&2
        ln -sfn ${dest_dir}/ffmpeg-10bit /usr/local/bin/ffmpeg-10bit >&2
        ln -sfn ${dest_dir}/ffprobe /usr/local/bin/ffprobe >&2
        ln -sfn ${dest_dir}/ffserver /usr/local/bin/ffserver >&2

        # Remove unzipped folder to do some clean up
        rm -fr ffmpeg-git-*-static/
        printf "%s\n" "Installing new version" >&2
else
    printf "%s\n" "Nothing new to download" # >&2 -- Why is this an error?
    exit # 3 -- I don't think this is any error. You checked and it's fine.

fi

#EOF

Next task:

Getting the script to check if current user have write access to download_dir and dest_dir else exit with prompt asking for new location or elevate user rights

Once again, I'm so happy for all the help I've received so far :)

Just an alternative solution if you're open to other ideas, but at that frequency I would download the source code, then update weekly and [compile](http://trac.ffmpeg.org/#CompilingFFmpeg). You could customize it and wouldn't be relying on someone else for updates. — llogan, Feb 28 '17 at 18:05
@LordNeckbeard that idear I've had previously, but for my usage Johns precompiled solution and keep sings simple (kiss) this turns out to be a very good solution, also for the common sharing and usage of ffmpeg and this script to others — Joakim, Mar 01 '17 at 12:22
@Joakim Welcome to StackOverflow! For future reference, note that your code is very long and this will cause people to not want to answer your question. For future reference on this site, I recommend you read this page, because it'll help you a lot in the future: https://stackoverflow.com/help/mcve — Leo Izen, Mar 01 '17 at 19:28
Another thing worth mentioning - when you do something like `<(curl URL)` this is something called "process substitution," where Bash will create a (character device) file that reads from the output of that command. If you want to use just a string like `${md5}` as the input, you need to use `<(echo -n "${md5}")` or something like that, because `${md5}` is not a command, but `echo -n ${md5}` is. — Leo Izen, Mar 01 '17 at 19:32

score 3 · Answer 1 · 2017-03-01T19:19:38.203

3

You got lost with what you're doing. Compare your script with this one.

#!/bin/bash
# Still to come, see if the script runs with root privileges else exit

# Download FFMPEG static daily build and it's md5

# or exit if not
cd ~ || exit 2

# 1. get the file
wget https://johnvansickle.com/ffmpeg/builds/ffmpeg-git-64bit-static.tar.xz

# 2. Check the md5 is correct
if ! diff <(md5sum ffmpeg-git-64bit-static.tar.xz) \
          <(curl -L https://johnvansickle.com/ffmpeg/builds/ffmpeg-git-64bit-static.tar.xz.md5)
then 
    printf "%s\n" "md5sum doesn't match..." >&2 
    exit 1
fi

# 3. untar
tar -xf ffmpeg-git-64bit-static.tar.xz

# 4. and so on..
cp ffmpeg-*-static/ff* /usr/bin/
cp ffmpeg-*-static/ff* /usr/local/bin/
cp ffmpeg-*-static/qt-faststart /usr/bin/
cp ffmpeg-*-static/qt-faststart /usr/local/bin/
# Consider change second location to use ln -s
# Remove downloads and do some clean up

rm -fr ffmpeg-*

#EOF

A few fixes to your updated version. I removed your comments to make my own visible.

#!/bin/bash
download_dir=~
dest_dir=/usr/bin/
version=ffmpeg-git-64bit-static.tar.xz
source_url=https://johnvansickle.com/ffmpeg/builds/${version}
md5_url=https://johnvansickle.com/ffmpeg/builds/${version}.md5

# You need braces to build blocks.
# As it was, your script terminated at the exit. Regardless. End of story.
cd ${download_dir} || {
    printf "%s\n" "You can't enter this folder." "Please change download_dir in this script..." >&2
    exit 1
}


if ! diff <(md5sum ${version}) <(curl -s ${md5_url})
then
    # Sum doesn't match is not really an error... I comment out the redirection.
    printf "%s\n" "md5sum doesn't match..." "Downloading new version" # >&2
    rm -f ${version} # >&2 -- Why would you redirect any output to stderr?
    curl ${source_url} -o ${download_dir}/${version} # >&2 -- Same as above.
else
    # You've done this already.
    # diff <(md5sum ${version}) <(curl -s ${md5_url})
    printf "%s\n" "Nothing new to download" # >&2 -- Why is this an error?
    exit # 3 -- I don't think this is any error. You checked and it's fine.
    # It might stay if you really MEAN it.
fi

# I'm not checking further.

tar -xf ffmpeg-git-*-static.tar.xz
mv ${download_dir}/ffmpeg-*-static/ff* "${dest_dir}"
mv ${download_dir}/ffmpeg-*-static/qt-faststart "${dest_dir}"

ln -sfn ${dest_dir}/qt-faststart /usr/local/bin/qt-faststart
ln -sfn ${dest_dir}/ffmpeg /usr/local/bin/ffmpeg
ln -sfn ${dest_dir}/ffmpeg-10bit /usr/local/bin/ffmpeg-10bit
ln -sfn ${dest_dir}/ffprobe /usr/local/bin/ffprobe
ln -sfn ${dest_dir}/ffserver /usr/local/bin/ffserver

rm -fr ffmpeg-git-*-static

Answers to your comment questions.

When is something to be determined as blocks?

Here you can get a wider picture of blocks. In Bash they're called lists. Run this:
```
man bash | grep -A 30 'Compound Commands'
```
and see what Bash's man has to say about them for a good start. This guide should be more approachable.
How to make it output the current running command to the console ex. the tar -xf etc.?

The only ready solution I know for this is to run Bash in the debugging mode, if that's what you want. You can turn it on anywhere in the script with
```
 set -x
```
Then you turn it off with:
```
set +x
```
You can do the same in the command line.

You can also set the whole script to run in this mode in the shebang.
```
#!/bin/bash -x
```
Or you can tell the interpreter from the command line to run in this mode,
```
bash -x ~/bin/your_script.bash
```

edited Mar 01 '17 at 19:19

answered Feb 28 '17 at 10:15

I'm aware that you basically cleaned up the script basically as is, but instead of downloading `ffmpeg-git-64bit-static.tar.xz` every time, regardless if it is new or not, I recommend only downloading it if the md5 does not match. Would save John some bandwidth at least. Also, no need to use `curl` **and** `wget`. Just one or the other should suffice. – llogan Feb 28 '17 at 18:09
1

@LordNeckbeard Good idea with checking first. But what's the point of not using both? – Feb 28 '17 at 23:37
3

Simply to reduce the number of required dependencies. For example, you can use `curl -O https://johnvansickle.com/ffmpeg/builds/ffmpeg-git-64bit-static.tar.xz` instead of `wget https://johnvansickle.com/ffmpeg/builds/ffmpeg-git-64bit-static.tar.xz` and now you just need `curl` and not both `curl` and `wget`. – llogan Mar 01 '17 at 01:13
@LordNeckbeard Fair enough. – Mar 01 '17 at 05:06
LordNeckbeard and @tomas these ideas are both pretty good both first I stroggle with this cleaned up suggestion. update_ffmpeg.sh: 13: update_ffmpeg.sh: Syntax error: "(" unexpected and used my entire day yesterday figure out how to fix this :( with no luck – Joakim Mar 01 '17 at 12:25
@Joakim I don't know how that error might have slipped in. http://www.shellcheck.net/ says my script is legit, and it's quite reliable. Check for typos, and also if you modify anything, run your script through that webpage. It is very helpful. – Mar 01 '17 at 12:39
thx for the link @tomas that's pretty cool :) and tells me that it's the if ! diff
if ! diff <(md5sum ${version}) <(curl -s ${md5_url}) ^-- SC1009: The mentioned parser error was in this if expression. – Joakim Mar 01 '17 at 14:51
@Joakim Welcome. There's a link on that site to its source on GitHub, in case you want to have it on your PC. It's in Ubuntu repos too. – Mar 01 '17 at 15:00
@tomas that's cheating :) and not learning :( but the fun is that http://www.shellcheck.net/ says there's no errors, but running it local parse the 44: /update_ffmpeg.sh: Syntax error: "(" unexpected of course tested it manually and then it worked fine if diff <(md5sum ffmpeg-git-64bit-static.tar.xz) <(curl -s https://johnvansickle.com/ffmpeg/builds/ffmpeg-git-64bit-static.tar.xz.md5); then printf "%s\n" "No new version availeble" >&2; fi No new version availeble – Joakim Mar 01 '17 at 15:43
@tomas I can see there are some text in previous comment that have been lost, the code diff code only works if i past the if part directly into my shell, but it fails with the Syntax error: "(" when i try to run it through bash script. Anybody who might know why this occurs? – Joakim Mar 01 '17 at 16:53
1

@Joakim I've seen your update1 to the question. Take a look at my update to the answer. – Mar 01 '17 at 17:45
@tomas I'm so happy for your time and effort, but even copied and pated your update into a hole new script nano -c update_ffmpeg.sh; chmod +x update_ffmpeg.sh; sh update_ffmpeg.sh; but update_ffmpeg.sh: 13: update_ffmpeg.sh: Syntax error: "(" unexpected Any body who could help me why this is such a different? and how to solve this diffrent? 2. it seems to me that elif exist in bash according to this post http://stackoverflow.com/a/32941761/1847653 – Joakim Mar 01 '17 at 18:11
@Joakim Yes, it exists, sorry, I corrected that. :-) Please take another try. – Mar 01 '17 at 18:25
1

@Joakim And DON'T run it with sh! Either `./update_ffmpeg.sh` or `bash update_ffmpeg.sh`. That's why Bash scripts should be suffixed `.bash` not `.sh`. They are two different things. – Mar 01 '17 at 18:33
@Joakim `./update_ffmpeg.sh` or another path. Then the first line in the script will tell the system what it should be run with. And it says `#!/bin/bash`, right? – Mar 01 '17 at 18:36
absolutely @tomas and it's working now :) I'm very happy. To my learning curve your comment # You need braces to build blocks., when is something to be determined as blocks? and next part how to make it output the current running command to the console ex. the tar -xf etc.? – Joakim Mar 01 '17 at 18:52
cool with the -x set -x it's easier to follow what the script do while you are in dev mode :) thx for sharing, i'll read the block info now :) – Joakim Mar 01 '17 at 20:13

bash script to install / update ffmpeg static builds

1 Answers1