36

I use the command

find . -type f -exec sha256sum {} \; > sha256SumOutput

to hash every file in a folder hierarchy. Unfortunately, sha256sum doesn't get the file names from find in alphabetical oder. How can this be fixed?

I'd like to have them ordered before they are hashed so they are hashed in alphabetical order (this has a reason).

A.B.
  • 90,397
UTF-8
  • 5,710
  • 10
  • 31
  • 67

2 Answers2

52

Using some pipes and sort

find . -type f -print0 | sort -z | xargs -r0 sha256sum > sha256SumOutput

Explanation

From man find

   -print0
        True; print the full file name on the standard output, followed
        by a null character (instead of the newline character that -print
        uses). This allows file names that contain newlines or other
        types of white space to be  correctly  interpreted by programs
        that process the find output.  This option corresponds to the -0
        option of xargs.

From man sort

   -z, --zero-terminated
        line delimiter is NUL, not newline

From man xargs

   -r
        If the standard input does not contain any nonblanks, do not run
        the command. Normally, the command is run once even if there is
        no input. This option is a GNU extension.

-0

        Input items are terminated by a null character instead of by
        whitespace, and the quotes and backslash are not special (every
        character is taken literally).  Disables the end of file string,
        which is treated like any  other  argument. Useful when input
        items might contain white space, quote marks, or backslashes.
        The GNU find -print0 option produces input suitable for this mode.

Example

% ls -laog
total 4288
drwxrwxr-x  2 4329472 Aug 17 08:20 .
drwx------ 57   20480 Aug 17 08:20 ..
-rw-rw-r--  1       0 Aug 17 08:16 a
-rw-rw-r--  1       0 Aug 17 08:16 a b
-rw-rw-r--  1       0 Aug 17 08:16 b
-rw-rw-r--  1       0 Aug 17 08:16 c

% find . -type f -print0 | sort -z | xargs -r0 sha256sum

e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855  ./a
e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855  ./a b
e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855  ./b
e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855  ./c

The values in the first column are the same, as the files don't have any content in my test.

marktani
  • 155
  • 10
A.B.
  • 90,397
  • this works with sha256sum, but not with echo... why? – Michael Aug 22 '21 at 02:33
  • @Michael If your command ends with ... | xargs -r0 echo what exactly do you mean "not working"? By design, echo will echo multiple parameters on a single line. If you want each file on a separate line, you can use ... | xargs -r0 printf "%s\n" instead. However, note that filenames can include line breaks, too. – Mikko Rantalainen Nov 16 '22 at 09:05
3

You should be able to just pipe your output from find to sort.

user3591723
  • 179
  • Yeah, but then there is no -exec switch. – UTF-8 Aug 17 '15 at 03:31
  • 2
    I don't believe find has any way to alphabetize the output, but piping to sort and then using xargs would give the expected output. find . -type f | sort | xargs sha256sum. Although it would have issues with subdirectories.. – user3591723 Aug 17 '15 at 03:44
  • Hacky way to deal with sub directories would be find . -type f | awk -F/ '{print $NF, $0}' | sort | awk '{print $2}' | xargs sha256sum – user3591723 Aug 17 '15 at 03:50
  • This prints the error xargs: unmatched single quote; by default quotes are special to xargs unless you use the -0 option sha256sum: invalid option -- 'l' Try 'sha256sum --help' for more information.. – UTF-8 Aug 17 '15 at 04:29
  • My guess is one of your files has a single quote in the name – user3591723 Aug 17 '15 at 04:52
  • If that is the case this should work: sha256sum $(find . -type f | awk -F/ '{print $NF, $0}' | sort | awk '{print $2}') – user3591723 Aug 17 '15 at 05:04
  • It prints No such file or directory errors when there are spaces in file names. Also, I have some files beginning with a date like 2015-08-17 someFileName and it seems to have special problems with that. – UTF-8 Aug 17 '15 at 05:35
  • This method is too hacky to be honest, there are tons of legal file names and it will fail on most of the strange ones. I can add a way to deal with the spaces, but then maybe something else comes up. I am sure there is a way to use find and xargs but I am kind drawing a blank right now. – user3591723 Aug 17 '15 at 06:24
  • "Unfortunately, sha256sum doesn't get the file names from find in alphabetical oder. How can this be fixed?" "Pipe find to sort" is an answer to that question, hold your fire. – user3591723 Aug 17 '15 at 10:21
  • See answer https://askubuntu.com/a/662383/50254 instead to see how to correctly handle all special characters in filenames. – Mikko Rantalainen Nov 16 '22 at 09:07