MD5 hashing all files within a usb and saving output file on a usb

Question

so I'm trying to work out how to hash all files within a USB directory. I came across another post similar to mine including this script below:

find "$PWD" -type d | sort | while read dir; do [ ! -f "${dir}"/@md5Sum.md5 ] && echo "Processing " "${dir}" || echo "Skipped " "${dir}" " @md5Sum.md5 already present" ; [ ! -f "${dir}"/@md5Sum.md5 ] && md5sum "${dir}"/* > "${dir}"/@md5Sum.md5 ; chmod a=r "${dir}"/@md5Sum.md5;done

It works really well and does everything I'd want it to do however I can't figure out how to alter it to target the USB. I've already got the user to input the USB device by them inputting /dev/sdf or /dev/sdg etc. But I can't figure out how to carry this information into the command above. Any advice or suggestions on this would be appreciated.

You should check that the partition on the USB drive is [auto]mounted (and mount it if it not). Then change directory cd to the mountpoint and run your script. — sudodus, Dec 29 '21 at 10:12
This link and this link may be helpful if automounting of a Microsoft file system (FAT32, exFAT, NTFS) does not work. — sudodus, Dec 29 '21 at 10:19
@sudodus so it does auto mount no problem and can access everything. The command I've been writing is to clone a whole usb including a bitlocker encryption onto a identical usb. I'm wanting to create an md5 hash of the source usb which will then be copied across to the cloned drive and then do another md5 hash to check if anything has been altered during the copying. — cameron.g, Dec 29 '21 at 10:21
and the reason i've got them to input /dev/sdf etc is to make sure its the whole directory of the USB not just an individual partition — cameron.g, Dec 29 '21 at 10:23
Well, that is different to what I thought, when you wrote 'hashing all files'. You can use lsblk -f and lsblk -m to get a list of all drives and partitions and that way identify the USB drive's device letter. You can clone it with a crude method using dd to another drive or to an image file, or more efficiently with Clonezilla. It is difficult to get a reproducible md5sum of the cloned drive because the target drive might be bigger than the original one. Instead test that the cloned copy works correctly. — sudodus, Dec 29 '21 at 10:35
I use dd as that appeared to be quickest. Clonezilla didn't work correctly for me for whatever reason and wasn't really user friendly... I've got everything sorted within my command and the end users know how to find the usb device name. I've been making clones for the past month or so using my command so i know it works. It's just purely trying to get this command I originally commented about to work with what i have. — cameron.g, Dec 29 '21 at 10:41
Then you should be able to printout the used size of the source (original) USB drive (to the end of the last partition). Then run dd using bs and count and pipe the result to md5sum. After cloning you should check that you get the same result on the target drive. Please notice that drives are usually not exactly the same size (in bytes), so it is important to check with the same numbers for bs and count. (The target drive must be big enough to copy until the end of the last partition of the source drive.) — sudodus, Dec 29 '21 at 11:06
So I haven't tried what you've suggested yet (@sudodus). But I tried altering the above to this:
find "/dev/$source" -type d | sort | while read source; do [ ! -f "${source}"/@md5Sum.md5 ] && echo "Processing " "${source}" || echo "Skipped " "${source}" " @md5Sum.md5 already present" ; [ ! -f "${source}"/@md5Sum.md5 ] && md5sum "${source}"/* > "${dir}"/@md5Sum.md5 ; chmod a=r "${source}"/@md5Sum.md5;done

After doing this, no progress is appeared to made and nothing is created in said USB... Soooo im a little bit stumped now — cameron.g, Dec 29 '21 at 12:04
Instead of trying to modify your script, I wrote an answer where I focus on what I think are important steps in order to check the 'whole' USB drive. I hope it will help you to reach your goal. — sudodus, Dec 29 '21 at 17:23

sudodus · Accepted Answer · 2021-12-29T17:35:55.450

I made a test with the Ubuntu Server iso file ubuntu-20.04.3-live-server-amd64.iso

from https://releases.ubuntu.com/20.04.3/ with the sha256sum

f8e3086f3cea0fb3fefb29937ab5ed9d19e767079633960ccb50e76153effc98

The iso file was cloned to USB drive. (You can do it with any cloning tool.)
The used size on the USB drive was determined with
```
sudo fdisk -lu /dev/sdx
```
where x is the drive letter (identified with fdisk and for example lsblk -f) and the number of sectors (from 0 to 'end_sector') is end_sector + 1, so in my test example 2463616 and the sector size is 512 bytes.
Check that no file system in the drive is mounted, and unmount the partition(s) if mounted (but do not eject).
Check on the USB drive

Calculate a checksum on the whole used part of the USB drive,
```
sudo dd if=/dev/sdx bs=512 count=2463616|sha256sum
```
The output of this check is the same as the uploaded checksum, which verifies that the cloning operation was successful. You may prefer to pipe the output from dd to md5sum.
If the checksum does not match for you, please check first that the number of sectors and number of bytes are correct (that you have checked exactly the correct part of the USB drive).

This same method works also when you clone from one USB drive to a another drive that is big enough, so check on the new drive (sha256sum or md5sum according to your choice).

Thanks for this so much sudo.... Lastly I'd prefer a txt file containing the md5 inside of it which i can create. But it always saves to my desktop how can i save it to other places like my usb? — cameron.g, Dec 29 '21 at 17:10
You can save the result to a file in a mounted file system, but not into the drive that you are checking. In order to avoid problems, you should also check that no file system in the drive to check is mounted, so unmount if necessary (but do not eject because that turns off the power to the drive). So use the full path /mountpoint/directory-path/file-name to where you want to save the result. — sudodus, Dec 29 '21 at 17:41

MD5 hashing all files within a usb and saving output file on a usb

1 Answers1