1

I've been beating my head against this for a while, but I'm really not a good script writer. Apologies...

I'm running Ubuntu/gnome 18.10, and have a large set of pictures exported from my wifey's mac. The directory structure is:

year1
  (login dir name with spaces) - Month
    Image names
year2
  ...

as in:

2013
  May 4, 2013
    Image1.jpg
    Image2.jpg
  May 5, 2013
    Image 1.jpg
    Image 3.jpg
  June 22, 2013

What I would like, is:

2013
   January
     All the "january" images...
   February
     All the...

I can create the directories easily enough mkdir {January..December} suffices. But I cannot figure out how to walk the ugly directory tree (exported from Mac), move the images, and then delete the ugly directory.

pa4080
  • 29,831
Charles Green
  • 21,339
  • I'd want to rename the images first, probably adding the full date, otherwise it looks like they'll overwrite each other if moved to a common directory. If the file's dates match their folder's dates that might be easier than using the file's path.. – Xen2050 Jan 16 '19 at 23:14
  • @Xen2050 Thanks - the file naming has changed over the years as cameras were bought and destroyed... but I think I have a fairly elegant solution. It unfortunately does not advance my knowledge of scripting... exiftool '-FileName<DateTimeOriginal' -d %Y/%m-%B/%f%%-c.%%e . -r -> it's almost perfect, except for some .mov files which do not have exif data – Charles Green Jan 17 '19 at 00:28

3 Answers3

2

There is an application which can move and rename the files in a single command line - exiftool

sudo apt install libimage-exiftool-perl

In my case, the specific command line used was

exiftool  '-FileName<DateTimeOriginal' -d <path-to-output-dir>/%Y/%m-%B/%Y%m%d_%H%M%S%%-c.%%e . -r

or

exiftool  '-FileName<CreateDate' -d <path-to-output-dir>/%Y/%m-%B/%Y%m%d_%H%M%S%%-c.%%e . -r

depending upon which tag is present in the images. If the requested tag is not present, the application throws a warning and does not move that file, but continues processing. This application also handles multiple images with the same date/time by appending a numeric to the end of the filename.

I did have several images without exif data, and I was able to add exif data to the images with the command

exiftool -createdate='2011:12:04 12:00:00' * -overwrite_original

As an example, I placed some images in a directory ~/aa/test1 and ran the tool, placing the output in ~/aa/test2. The results are below:

chick@dad:~/aa$ tree .
.
├── test1
│   ├── DSC00018.JPG
│   ├── DSC00022.JPG
│   ├── DSC00024.JPG
│   ├── DSC00025.JPG
│   ├── DSC00026.JPG
│   ├── DSC00028.JPG
│   ├── DSC00031.JPG
│   ├── DSC00033.JPG
│   └── Thumbs.db
└── test2
    └── 2000
        └── 12-December
            ├── 20001222_185523.JPG
            ├── 20001222_200726.JPG
            ├── 20001222_200819.JPG
            ├── 20001222_201205.JPG
            ├── 20001222_201223.JPG
            ├── 20001222_210536.JPG
            ├── 20001222_211858.JPG
            └── 20001222_215950.JPG
Charles Green
  • 21,339
  • Superb! You may accept this answer, because of its simplicity :) Yesterday when I made the last edit on my answer, I realized rename can be used in similar way as it is made here, but I do not have time to elaborate. – pa4080 Jan 18 '19 at 17:33
  • 1
    @pa4080 But I still like your answer, because it will cause me to work seeper into scripting...! – Charles Green Jan 18 '19 at 17:36
1

Here is such script:

#!/bin/bash

# The destination where the new directory structure will be created
DEST="/tmp/new-order-pictures/"

MONTHS=('Jan' 'Feb' 'Mar' 'Apr' 'May' 'Jun' 'Jul' 'Aug' 'Sep' 'Oct' 'Nov' 'Dec')

# Walk through the first level directories, located in the current directory and go inside
for year in */
do
    cd "$year"
    # Walk through the months of the year
    for month in "${MONTHS[@]}"
    do
        # Walk through the second level directories
        for dir in */
        do
            # If we have coincidence between the name of the directory and the month
            # go inside, make a new destination directory; ignore character cases^^
            if [[ ${dir^^} =~ ${month^^} ]]
            then
                cd "$dir"
                dest="${DEST}${year}${month}"
                mkdir -p "$dest"
                find . -type f | while IFS= read -r item
                do
                    # Copy the files to the new destination and
                    # add the file's md5sum to its name to prevent files lose
                    filename=$(basename -- "$item")
                    extn="${filename##*.}"
                    name="${filename%.*}"
                    cp "$item" "${dest}/${name}-$(md5sum "$item" | cut -f1 -d' ').${extn}"
                done
                cd ..
            fi
        done
    done
    cd ..
done

The script should be executed in the first level directory where your images are located. You should tweak the destination directory - DEST="/tmp/new-order-pictures/". This version of the script relies that all files are in directories that contain the name of a month in one way or another. Example of usage:

user@host:~/Pictures$ tree .
.
├── 2013
│   ├── January 17, 2013
│   │   ├── Image1.jpg
│   │   └── Image 3.jpg
│   ├── January 24, 2013
│   │   └── Image2.jpg
│   ├── January 25, 2013
│   │   └── Image 3.jpg
│   ├── June 22, 2013
│   │   └── image1.jpg
│   ├── May 4, 2013
│   │   └── Image1.jpg
│   └── May 5, 2013
│       ├── Image1.jpg
│       └── Image 2.jpg
└── 2014
    ├── January 17, 2014
    │   ├── Image1.jpg
    │   └── Image 3.jpg
    ├── January 24, 2014
    │   └── Image2.jpg
    ├── January 25, 2014
    │   └── Image 3.jpg
    └── May 5
        ├── Image1.jpg
        └── Image 2.jpg

12 directories, 14 files


user@host:~/Pictures$ order.sh 


user@host:~/Pictures$ tree /tmp/new-order-pictures/
/tmp/new-order-pictures/
├── 2013
│   ├── Jan
│   │   ├── Image1-7b71d9fdfe5b15a2d1a4968c195f93ae.jpg
│   │   ├── Image2-cbf4d36ff84e7ec24c05f8181236e6b8.jpg
│   │   ├── Image 3-0bca5188fd3f3eb470533fdaf0630633.jpg
│   │   └── Image 3-6a83880cae1aa57e19a7c45de7759e68.jpg
│   ├── Jun
│   │   └── image1-adb3bf995f1a25d008f758a7266d7be5.jpg
│   └── May
│       ├── Image1-511d541b35fcb38af8ada18d7961268c.jpg
│       ├── Image1-a66c5863e6986605cb2ca6d622ae72a0.jpg
│       └── Image 2-c34ffc32ce5d3901e1ad89b9fd15a877.jpg
└── 2014
    ├── Jan
    │   ├── Image1-7b71d9fdfe5b15a2d1a4968c195f93ae.jpg
    │   ├── Image2-cbf4d36ff84e7ec24c05f8181236e6b8.jpg
    │   ├── Image 3-0bca5188fd3f3eb470533fdaf0630633.jpg
    │   └── Image 3-6a83880cae1aa57e19a7c45de7759e68.jpg
    └── May
        ├── Image1-511d541b35fcb38af8ada18d7961268c.jpg
        └── Image 2-c34ffc32ce5d3901e1ad89b9fd15a877.jpg

7 directories, 14 files

In my case the script is named order.sh and it is located in ~/bin, thus I can use it as shell command. In the example you can see the directory structure is changed but the number of files is 14 in both structures.


Here is another version of the script that use mv instead of cp and will deal also with the files which are not in directories that contain the name of a month. Before running this script it is a good idea to create a backup copy of the original directory structure.

#!/bin/bash

# The destination where the new directory structure will be created
DEST="/tmp/new-order-pictures/"

MONTHS=('Jan' 'Feb' 'Mar' 'Apr' 'May' 'Jun' 'Jul' 'Aug' 'Sep' 'Oct' 'Nov' 'Dec')

# Walk through the first level directories, located in the current directory and go inside
for year in */
do

    cd "$year"

    # Walk through the months of the year
    for month in "${MONTHS[@]}"
    do
        # Walk through the second level directories
        for dir in */
        do
            # If we have coincidence between the name of the directory and the month
            # go inside, make a new destination directory; ignore character cases^^
            if [[ ${dir^^} =~ ${month^^} ]]
            then

                cd "$dir"
                dest="${DEST}${year}${month}"
                mkdir -p "$dest"

                while IFS= read -r item
                do
                    # Copy the files to the new destination and
                    # add the file's md5sum to its name to prevent files lose
                    filename=$(basename -- "$item")
                    extn="${filename##*.}"
                    name="${filename%.*}"
                    mv "$item" "${dest}/${name}-$(md5sum "$item" | cut -f1 -d' ').${extn}"
                done < <(find . -type f)

                cd ..

            fi

        done

    done

    # Dial with the rest of the files for that $year

    dest="${DEST}${year}other"

    while IFS= read -r item
    do
        mkdir -p "$dest"
        filename=$(basename -- "$item")
        extn="${filename##*.}"
        name="${filename%.*}"
        mv "$item" "${dest}/${name}-$(md5sum "$item" | cut -f1 -d' ').${extn}"
    done < <(find . -type f)

    cd ..

done

Example of usage:

user@host:~/Pictures$ tree .
.
├── 2013
│   ├── January 17, 2013
│   │   ├── Image1.jpg
│   │   ├── Image 3.jpg
│   │   └── video 7.mpg
│   ├── January 25, 2013
│   │   └── Image 3.jpg
│   ├── June 22, 2013
│   │   └── image1.jpg
│   └── May 5, 2013
│       ├── Image1.jpg
│       └── Image 2.jpg
└── 2014
    ├── Apr 7
    │   ├── Image1.jpg
    │   └── Image 2.jpg
    ├── Image 2.jpg
    ├── January 11, 2014
    │   ├── Image1.jpg
    │   └── Image 3.jpg
    ├── some other name
    │   └── some other name file inside.jpg
    ├── some other name file inside.jpg
    └── video 1.avi

9 directories, 15 files

user@host:~/Pictures$ order.sh 

user@host:~/Pictures$ tree /tmp/new-order-pictures/
/tmp/new-order-pictures/
├── 2013
│   ├── Jan
│   │   ├── Image1-7b71d9fdfe5b15a2d1a4968c195f93ae.jpg
│   │   ├── Image 3-0bca5188fd3f3eb470533fdaf0630633.jpg
│   │   ├── Image 3-6a83880cae1aa57e19a7c45de7759e68.jpg
│   │   └── video 7-86764d9565469adfb22c8ef4f0b9c04f.mpg
│   ├── Jun
│   │   └── image1-adb3bf995f1a25d008f758a7266d7be5.jpg
│   └── May
│       ├── Image1-511d541b35fcb38af8ada18d7961268c.jpg
│       └── Image 2-c34ffc32ce5d3901e1ad89b9fd15a877.jpg
└── 2014
    ├── Apr
    │   ├── Image1-3c19da25e0e56ef0fc752a9e4f75b190.jpg
    │   └── Image 2-dcc35e86de393a014ac62e8c4390c7e6.jpg
    ├── Jan
    │   ├── Image1-ae34289b0bc5258f286165745ff3c258.jpg
    │   └── Image 3-1724adf2dfcc1d4a0dc50cb38ad2c510.jpg
    └── other
        ├── Image 2-eff5208f7eee6a536e48f9982b918dfb.jpg
        ├── some other name file inside-7d0a68e0b4e9cc3928744cb83f4d1136.jpg
        ├── some other name file inside-c2dd637e94a9025c3e1004d66f59539c.jpg
        └── video 1-c277d93a2427bedf3f0b8ae07427edb9.avi

8 directories, 15 files

After that you can go inside the destination directory and use the rename command within for loop to deal with the long names:

# For each directory on the second level
for dir in */*
do
    cd "$dir"
    rename 's/^.*(\.[0-9a-zA-Z]+)$/our $i; sprintf("Image-%03d$1", 1+$i++)/e' *
    cd ..
    cd ..
done

Example:

user@host:~/Pictures$ cd /tmp/new-order-pictures/

user@host:/tmp/new-order-pictures$ for dir in */*; do cd "$dir"; rename 's/^.*(\.[0-9a-zA-Z]+)$/our $i; sprintf("Image-%03d$1", 1+$i++)/e' *; cd ..; cd ..; done

user@host:/tmp/new-order-pictures$ tree .
.
├── 2013
│   ├── Jan
│   │   ├── Image-001.jpg
│   │   ├── Image-002.jpg
│   │   ├── Image-003.jpg
│   │   └── Image-004.mpg
│   ├── Jun
│   │   └── Image-001.jpg
│   └── May
│       ├── Image-001.jpg
│       └── Image-002.jpg
└── 2014
    ├── Apr
    │   ├── Image-001.jpg
    │   └── Image-002.jpg
    ├── Jan
    │   ├── Image-001.jpg
    │   └── Image-002.jpg
    └── other
        ├── Image-001.jpg
        ├── Image-002.jpg
        ├── Image-003.jpg
        └── Image-004.avi

8 directories, 15 files

Or you can change (\.[0-9a-zA-Z]+) with (\.jpg), then on next iteration with (\.mpg) (respectively Image- with Video-), etc. References about this usage of rename:

pa4080
  • 29,831
  • Thanks! I think this is really close to what I asked for, and I'll have to scour the web looking for shell expansions et al... – Charles Green Jan 17 '19 at 15:05
  • Hi, @CharlesGreen, I"ve updated the answer with few more ideas. Sorry for the long answer and the lack of explanations inside... if you need clarifications about some parts, please ping me :) – pa4080 Jan 17 '19 at 18:19
  • No problem at all - I really am looking to use this more as a reason to learn some shell scripting. I did in fact find a single line command which does exactly what I need! – Charles Green Jan 17 '19 at 18:22
  • @CharlesGreen: Nice! It will be worth to write an answer. – pa4080 Jan 17 '19 at 18:34
  • I can certainly share the command, and will add it later. But your command answered a deeper question in prompting me to learn more about scripting – Charles Green Jan 17 '19 at 18:59
  • Added an answer with the command I mentioned in earlier comments. – Charles Green Jan 18 '19 at 16:49
0

So, I assume you want a step-by-step and simple solution. First, I would address the problem of Whitespace and comma in the folders. I would first cd into the year folder and use rename to first remove Whitespace

rename "s/ //g" *

Similarly, remove comma

rename "s/,//g" * 

Now that I have all the folders as desired I would create a month based list using

ls | grep "January" > January.txt

Now make folder "January"

mkdir January

Then loop thru the list using xargs to copy their contents

cat January.txt | xargs -I {} cp -R {}/. ./January/
puneet
  • 1
  • 1