Linux piping find and md5sum not sending output

Question

Trying to loop every file, do some cutting, extract the first 4 characters of the MD5.

Here's what I got so far:

find . -name *.jpg | cut -f4 -d/ | cut -f1 -d. | md5sum | head -c 4

Problem is, I don't see any more output at this point. How can I send output to md5sum and continue sending the result?

Are you trying to compute the md5sums of the files? That's not happening here. — William Pursell, May 31 '17 at 01:17
the `find` command will take hours to complete. do i have to wait for it to finish? i want to get md5 line by line and perform additional operations — user2914191, May 31 '17 at 01:23
The command you wrote computes a single MD5 hash of the whole `find`/`cut` result. — melpomene, May 31 '17 at 01:55
i need md5 of just the file name one at a time not whole stream. looks like ill have to python this one — user2914191, May 31 '17 at 02:06

score 3 · Accepted Answer · answered May 31 '17 at 02:46

md5sum reads everything from stdin till end of file (eof) and outputs md5 sum of full file. You should separate input into lines and run md5sum per line, for example with while read var loop:

find . -name *.jpg | cut -f4 -d/ | cut -f1 -d. | 
  while read -r a; 
   do   echo -n $a| md5sum | head -c 4; 
  done

read builtin bash command will read one line from input into shell variable $a; while loop will run loop body (commands between do and done) for every return from read, and $a will be the current line. -r option of read is to not convert backslash; -n option of echo command will not add newline (if you want newline, remove -n option of echo).

This will be slow for thousands of files and more, as there are several forks/execs for every file inside loop. Faster will be some scripting with perl or python or nodejs or any other scripting language with builtin md5 hash computing (or with some library).

Same loop was used in https://stackoverflow.com/a/5890196, similar in https://stackoverflow.com/a/31955350 — osgx, May 31 '17 at 02:51

Muhammad Muntaza · Answer 2 · 2017-06-02T02:04:27.853

1

How to find all .jpg file then execute md5sum then cut first 4 caracters:

find . -name '*.jpg'  -exec md5sum {} \; | cut -b 1-4

edited Jun 02 '17 at 02:04

answered May 31 '17 at 02:03

Muhammad Muntaza

80
1
6

i need md5 of the file name (no extension), not the whole path to file – user2914191 May 31 '17 at 02:07
While this code may answer the question, providing additional context regarding how and/or why it solves the problem would improve the answer's long-term value. – Badacadabra May 31 '17 at 12:37

score 1 · Answer 3 · answered May 31 '17 at 02:51

You can do what you are attempting to do with a short "helper" script that you call from find. For example, you could create a short script to find the basename of each file passed as an argument, remove the '.jpg' extension, and then provide the remaining name w/o extension as input to md5sum on stdin to get the md5sum of the name itself. Call the script anything you like, say namemd5.sh. Example:

#!/bin/bash

[ -z "$1" ] && exit 1           ## validate single argument

fname=$(basename "$1")          ## get the filename alone
fname="${fname%.jpg}"           ## remove .jpg extension

fnsum=$(md5sum - <<<"$fname")   ## get md5sum of name w/o .jpg
fnsum=${fnsum%% *}              ## remove trailing ' -'

echo "$fnsum - $fname"          ## output md5sum - name
                                ## (remove ' - $fname' for md5sum alone)

(note: the name is provided as part of the output for example purposes, remove if you want the md5sum alone as shown in the comment above)

Example Files

$ find /home/david/img/wp/ -type f -name "*.jpg"
/home/david/img/wp/hacker_manifesto_1200x900.jpg
/home/david/img/wp/hacker_manifesto_by_otalicus.jpg
/home/david/img/wp/reflections-triple-1920x1200.jpg
/home/david/img/wp/hacker_wallpaper_1600x900.jpg
/home/david/img/wp/Zen.jpg
/home/david/img/wp/hacker_wallpaper_by_vanilla23-dot254.jpg
/home/david/img/wp/hacker_manifesto_1600x900.jpg

Example Use/Output

$ find /home/david/img/wp/ -type f -name "*.jpg" -exec ./namemd5.sh '{}' \;
0f7d2aac158eb9f7842215e14ff6573c - hacker_manifesto_1200x900
604bc695a0bb70b8db0352267caf226f - hacker_manifesto_by_otalicus
5decea0e306f185bf988ac9934ec0e2c - reflections-triple-1920x1200
82bd8e1ad3df588eb0e0848c5f764812 - hacker_wallpaper_1600x900
0f4daba431a22c03f28977f087e4c695 - Zen
0c55cd3ebd2a847e10c20d86e80e6ceb - hacker_wallpaper_by_vanilla23-dot254
e5c1da0c2db3827d2bf81c306633cc56 - hacker_manifesto_1600x900

You can also call the script with the -execdir version within find as well, e.g.

$ find /home/david/img/wp/ -type f -name "*.jpg" -execdir \
/full/path/to/namemd5.sh '{}' \;

(note: the use of the /full/path to your helper script above)

Linux piping find and md5sum not sending output

3 Answers3