Returning the MD5-hash of all files in a directory to a .txt file with file name

Question

I have a directory of pictures.

I need the MD5-hash of each file in that directory and the name of the file placed into a .txt document. This file is to be read at a later time to reference the MD5 and file name together.

Note:
I would like this to just pull all directory files with me specifying them.
I have tried playing with tar and find and I just cannot seem to find a good solution...

This is a directory example:

/Desktop/testing
RandomFilename1.png
RandomFilename2.png
RandomFilename3.png

The .txt output is:

RandomFilename1,da39a3ee5e6b4b0d3255bfef95601890afd80709
RandomFilename2,da39a3ee5e6b4b0d3255bfef95601890afd80709
RandomFilename3,da39a3ee5e6b4b0d3255bfef95601890afd80709

I've looked everywhere online with no luck.
This will be on a Linux terminal.

Do you really want the file names without the extension in the output? If so, what if two file names only differ in their extension? — dessert, Nov 06 '18 at 22:09
See answer by marcolino in particular on yhe linked duplicate — Sergiy Kolodyazhnyy, Nov 06 '18 at 23:39

score 6 · Accepted Answer · edited Nov 06 '18 at 22:03

6

You can use the following bash command:

for file in testing/*; do md5sum $file; done > res.txt

Of course you have to adjust the destination directory of res.txt.
The result could look similar to this:

8b1500ea6fe42927891fd1831bb955ba  testing/Pic1.gif
73f3222889a14411a3b78675ed0bc269  testing/Pic2.gif
c5b18ef1ea1346f13f668a7ead4e4c92  testing/Pic3.gif

So the MD5 hash is followed by the filename and path.

edited Nov 06 '18 at 22:03

dessert

39,982

answered Nov 06 '18 at 19:29

zx485

2,426

4

The benefit of this solution is that you can later check the md5's with md5sum -c res.txt. I'd also suggest a simple md5sum *.gif > res.txt instead of a loop. – PerlDuck Nov 06 '18 at 19:54
Also, you should move the redirect outside the loop: for file ... done > res.txt, instead of opening and closing that file for each run of the loop. – muru Nov 06 '18 at 21:41

score 1 · Answer 2 · edited Nov 06 '18 at 22:04

For what you want to do, execute md5sum and use sed or awk to transform the output

md5sum *.png | sed 's/^\([0-9a-f]\+\) \+\(.\+\)/\2,\1/' >images.md5sums
md5sum *.png | awk '{print $2","$1}' >image.md5sums

However, the plain output of md5sum is hash filename and is equivalent to your format. Using that standard md5sum output format is often more convenient, since you can use it directly with md5sum -c to check the hash of the files (after a copy or network transfer). Some other utilities may also expect this particular format.

Kulfy · Answer 3 · 2018-11-07T08:16:23.013

-1

I have written a small script as per the parameters defined:

cd /home/username/Desktop/testing
for F in *
do
    echo "$F", $(md5sum "$F" | sed -e 's/'"$F"'//g')
done

Example output:

1.txt, c8ef7cca2ceb3f878921b1ff163dbf91
demo, e9470a3c74b32036fa148dfabce70929
File with spaces.txt, c691b3d2fc2678839a9c141b6ee1524e
line.cpp, e0732d2781c6b2b57d413f79f9c1fd28
test, 3734699610e8d322767cf65f90ed43ce
test.cpp, afbe5b90cf002e1eb6df4cf98dfe4c0e

Note: This script would show the output in terminal. To save it in file you can use > 1.txt while executing the file, for e.g.:

./test.sh > 1.txt

edited Nov 07 '18 at 08:16

answered Nov 06 '18 at 19:34

Kulfy

17,696

Nice, although I'd suggest couple improvements. Use for F in ./* instead of plain * there. Filenames with leading dashes will be interpreted as flags to command and not filename. I literally had spend 5 minutes figuring out why for f in * loop was hanging, and that was because I had - file , so expansion made md5sum - read from stdin. Second, you could do echo "$F", $(md5sum < "$F" | tr -d '-' ) and that also reduces complexity in parsing md5sum output. – Sergiy Kolodyazhnyy Nov 07 '18 at 08:35
What you can also do is md5sum "$F" | awk '{printf "%s,%s\n",substr($0,length($1)+3),$1 }' This is just 2 pipelines and should be safe against filenames with spaces. – Sergiy Kolodyazhnyy Nov 07 '18 at 08:41
@SergiyKolodyazhnyy Yeah you are right. It is unresponsive with filenames starting with special characters. I tried using ./* instead of simply * but I'm getting errors like: sed: -e expression #1, char 14: unknown option to 's' ./test.cpp. I don't know why. While the second suggestion echo "$F", $(md5sum < "$F" | tr -d '-' ) is working fine. Again with third suggestion md5sum "$F" | awk '{printf "%s,%s\n",substr($0,length($1)+3),$1 }' error is being generated invalid option with filenames having - in starting. – Kulfy Nov 07 '18 at 09:17
@Kulfy It probably complains about the / part because / is separator in sed, so you're doing something like sed -e 's/./foobar//g'. Not a big deal, GNU sed can use different separators so something like sed -e 's#./test##g' could work – Sergiy Kolodyazhnyy Nov 07 '18 at 09:26
@SergiyKolodyazhnyy Thank you for letting me know this. But the output is like ./<filename>, <mda5sum> ./ and sometimes ./<filename>, <mda5sum> – Kulfy Nov 07 '18 at 09:38

Returning the MD5-hash of all files in a directory to a .txt file with file name

3 Answers3