Bash : Check if file contains other file contents

Question

So I'm trying to append the contents of one file to another file, if it's not already included. This is how I try:

catAndAppendIfMissing(){
    [[ ! -s $2 ]] && touch "$2" || [[ ! -s $2 ]] && sudo touch "$2"
    if grep $1 $2; then
        echo "found"
    else
        catAndAppend $1 $2       #this appends file $1 contents to file $2 (and takes care of adding newlines if needed and uses sudo if needed, thus the separate function)
    fi
}

With if grep $1 $2 I'm trying to see if file $1 contents are present in file $2. That's the part that doesn't work as intended:
When I run this twice on the same file, it will simply append the same text twice to the destination file.

How can I solve that?

---

Precisions:

• I'm on OSX 10.11.5 (but a solution for Linux / cross-platform could also be relevant both for me at home or for someone else reading this)
• My choice of using catAndAppend over cat $file1 >> $file2 is to handle cases where sudo is needed and separate the appended stuff from what's already there by adding newlines as needed.
• I don't wish to append if file $1 is anywhere in file $2 (not only at the beginning or the end)
• For info, here's one of the files $1 contents that I tried against:

.

alias ls='ls -a'
alias mkdir="mkdir -pv"
alias wget="wget -c"
alias histg="history | grep"
alias echopath='echo $PATH | tr -s ":" "\n"'
alias myip="curl -sSL http://ipecho.net/plain | xargs echo"
alias webpic="mogrify -resize 690\> *.png"

alias cddog='cd ~/dev/go/src/github.com/dogtools/dog'
alias xp='cd ~/dev/go/src/experiments'

• but I will need to use it with other files containing var exports, code, commands, configs, any kind of text basically

Answer 1

Don't append if file $1 is anywhere in file $2:

catAndAppendIfMissing(){
    f1=$(wc -c < "$1")
    diff  -y <(od -An -tx1 -w1 -v "$1") <(od -An -tx1 -w1 -v "$2") | \
    rev | cut -f2 | uniq -c | grep -v '[>|]' | numgrep /${f1}../ | \
    grep -q -m1 '.+*' || cat "$1" >> "$2";     }

---

How it works:

1. Count chars in file $1 using wc.
2. Use od to produce a one byte per line hex dump of both files, and using a bashism, obtain a diff file, which is piped to...
3. rev, then cut the 2nd field, and do a uniq count of the consecutive lines that have blanks instead of '>'s.
4. If one of those counts is equal to or greater than $f1, it's OK to append. This could be checked with variables, but numgrep was convenient and helps avoid variables.

Notes. Good: works with binary files too. Bad: inefficient, od reads the whole of both files, and diff reads the whole of od's output. If file1 was a one line string, which was in the first line of a 1TB file2, much time would be wasted.

---

(Old version). Don't append if file $1 is already appended to file $2:

catAndAppendIfMissing(){
    f1=$(wc -c < "$1")
    f2=$(wc -c < "$2")
    [ $f1 -le $f2 ] &&  cmp -s "$1" "$2" 0 $(( $f2 - $f1 )) && return 1 
    cat "$1" >> "$2"
    }

---

How it works:

1. Get file lengths with wc, store in $f1 and $f2.
2. If the first file is longer than the second file, (or if shorter, if cmp shows the the first file isn't already appended to the second file), then append it to the second file with cat. Otherwise return with an error code.

Answer 2

It probably isn't worth trying to conditionally update the file; just source each file to make sure all the aliases are defined, then unconditionally store the output of alias to the file you would otherwise be appending to.

source "$1"   # Original aliases
source "$2"   # New aliases
alias > "$1"  # Combined aliases

Answer 3

Line:

if grep $1 $2 

should be:

if grep `cat $1` $2

OR

file1_Content=`cat $1`

if grep ${file1_Content} $2

OR

file1_Content=`cat $1`
grep ${file1_Content} $2

if [ $? == 0 ];then
  echo "found"
else
  #catAndAppend
fi

Answer 4

So I did my homework and came up with a solution which (almost) fits the bill, with the only difference that it's done in python instead of bash. My python script is then called from bash.

So here's the code :

import re, os, subprocess, mmap, sys, pprint 

def isFile1InFile2(file1Path, file2Path): 
    with open(file2Path) as file2: 
        file2Access = mmap.mmap(file2.fileno(), 0, access=mmap.ACCESS_READ) 
        file1Contents = open(file1Path).read() 
        if file2Access.find(file1Contents) != -1: 
            return True 
        else: 
            return False 

def appendIfMissing(source, dest): 
    destFullPath = os.path.expanduser(dest) 
    if os.path.isfile(destFullPath): 
        if isFile1InFile2(source, destFullPath): 
            print ('Source\'s contents found in dest file, no need to append') 
        else: 
            print('Source\'s contents cannot be found in dest file, appending...') 
            # append source file to destfile 
            command = ' '.join(['source', './common/configs/.shell-functions', '&&', 'catAndAppend', source, destFullPath]) 
            os.system(command) 

    else: 
        print "destfile not a file yet, copying sourcefile to destfile..." 
        # copy source file to destfile 
        command = ' '.join(['source', './common/configs/.shell-functions', '&&', 'catAndAppend', source, destFullPath]) 
        print command 
        os.system(command)

if len(sys.argv) != 3:
    sys.exit('[ERROR] appendIfMissing.py, line 31: number of arguments passed is not 3')
else:
    appendIfMissing(sys.argv[1], sys.argv[2])

And then to call it from bash:

appendIfMissing(){ 
    python ./common/configs/appendIfMissing.py $1 $2 
} 

With the bash function (the one called from python) staying the same:

createFileIfMissing(){
    # create file if doesn't exist, with right permission
    [[ ! -s $1 ]] && touch "$1" || [[ ! -s $1 ]] && sudo touch "$1"
}

addNewLineToFile(){
    [[ ! -e $1 ]] || [[ -w $1 ]] && printf "\n" >> $1 || [[ -e $1 ]] && [[ ! -w $1 ]] && sudo bash -c "printf \"\n\" >> $1"
}

catAndAppend(){ 
    createFileIfMissing $2 
    # append stuff to it 
    [[ ! -e $2 ]] || [[ -w $2 ]] && cat $1 >> $2 || [[ -e $2 ]] && [[ ! -w $2 ]] && sudo bash -c "cat $1 >> $2" 
    addNewLineTo $2 
} 

---

Cons:

• It's not bash. I asked for a bash solution in my question (but really all I care about is to have a solution)
• It's not bash. And as it's intended for system setup scripts, I'd have to install python first for this to work. But I want that eventually installed anyway.

Pros:

• It's more readable / maintainable / customizable than bash IMHO (being a newbie to both of those languages, python is more intuitive to start with)
• It's cross platform