How to create a hidden hash file in Linux for each file recursively?

Question

I could manage to generate a hash file for each file not yet hashed recursively:

find . -type f ! -name '*.sha256' -execdir sh -c '[ ! -f "$1.sha256" ] && sha256sum "$1" > "$1.sha256"' _ {} \;

I can´t get it to produce hidden files prefixed with a dot. Adding a . like so

find . -type f ! -name '*.sha256' -execdir sh -c '[ ! -f "$1.sha256" ] && sha256sum "$1" > ".$1.sha256"' _ {} \;

This produces no output.

I can´t figure out how to add the . prefix or how to escape it.

I tried to isolate the variable like ${1} or {$1}. This also didn´t help.

I got the current generation for non-hidden files extrapolated from this question and its solution.

I would be perfectly happy to have a bash script to call to do the job if it is easier to produce. I would need the solution to work on Debian and Ubuntu Linux.

should a file called `.foo` receive a hidden hash called `..foo.sha256` ? — jhnc, Jul 22 '23 at 14:02

score 2 · Answer 1 · answered Jul 22 '23 at 09:35

Firstly, you check for "$1.sha256" but then try to create ".$1.sha256".

Secondly and more importantly, when a command is invoked from execdir, its argument will always start with ./. Prepending a dot to it won't do you much good. You can use basename to get rid of the ./.

score 0 · Answer 2 · answered Jul 23 '23 at 19:55

As I was unable to handle it in a shell call I resorted to a python script. I could not wrap my head around how to make use of suggested basename neither had success with using exec instead of execdir.

I'd still be interested out of curiosity if some one could provide me a working line of shell code.

Until that I will work with the following python code

import hashlib, os, sys


for root, dirs, files in os.walk(".", topdown=False):
    for fileName in files:
        if(not fileName.endswith('.sha256')):
            # TODO: check if has file already present to only add if not yet computed
            absPath = os.path.join(root, fileName)
            hashFileName = "." + fileName + ".sha256"
            absPathPossibleHash = os.path.join(root, hashFileName)
            if(os.path.isfile(absPathPossibleHash)):
                print ("hash file already exists: " + absPathPossibleHash)
                continue
            print("fileName : " + fileName)
            print("root : " + root)
            print("creating hash of " + absPath)
            sha256_hash = hashlib.sha256()
            with open(absPath,"rb") as f:
                # Read and update hash string value in blocks of 4K
                for byte_block in iter(lambda: f.read(4096),b""):
                    sha256_hash.update(byte_block)
                hashLine = sha256_hash.hexdigest() +"  " + fileName
                print(hashLine)
                
                with open(os.path.join(root,hashFileName), "w") as hashFile:
                    hashFile.write(hashLine)

At least I'm still able to use find to verify my hashes ;)

find . -type f -name '.*.sha256' -execdir sh -c 'sha256sum -c "$1"' _ {} \;

score 0 · Answer 3 · answered Jul 25 '23 at 15:35

For each file found, find prepends the path by which the file was reached. -execdir cd's to that same directory for each file, so the prefix is always ./.

For example, if a file foo is found, ".$1.sha256" becomes "../foo.sha256", which is not what you want.

So the ./ prefix must be removed. Assuming your shell is POSIX-compatible and relatively modern, this can be done with parameter expansion.

# Print the contents of the variable `$1` with the largest match
# of the pattern `*/` at the beginning of the expansion removed
echo "${1##*/}"

In your case, the full command could be:

find . -type f ! -name '*.sha256' -execdir \
    sh -c '[ ! -f ".${1##*/}.sha256" ] && sha256sum "$1" > ".${1##*/}.sha256"' _ {} \;

Or, to keep things more DRY:

find . -type f ! -name '*.sha256' -execdir \
    sh -c 'hf=".${1##*/}.sha256"; [ ! -f "$hf" ] && sha256sum "$1" > "$hf"' _ {} \;

How to create a hidden hash file in Linux for each file recursively?

3 Answers3