merge files in different directories running a script on terminal

Question

I am new to UBUNTU. How can I merge files in different directories? Using cat to merge them all is a meticulous task. It is the only way I know. Thanks in advance, Saikat

Answer 1

When having simple text files, you can merge them without difficulties by using the >> operator. It will add the output of the previous command to the end of the file specified after.

Basic Procedure (not for many files)

Example:

Imagine a directory structure like this:

parent
 |--a
 |  \--first.txt
 \--b
    \--second.txt

Now the contentents of a/first.txt are

Hello! This is my first Textfile.
   ~~some lines~~
end

and b/second.txt contains

    ///////////////////
   //               //
  //  second file  //
 //               //
///////////////////

So we can merge it by typing:

$ touch merge.txt
$ cat a/first.txt >> merge.txt
$ cat b/second.txt >> merge.txt

A new empty file merge.txt in the current directory will be created and the files contents will be added without overwriting, like > would do.

merge.txt now contains:

Hello! This is my first Textfile.
   ~~some lines~~
end
    ///////////////////
   //               //
  //  second file  //
 //               //
///////////////////

Done!

Advanced Scripting (to cover many files in different directories)

If you have more than ten files, maybe even in ten different directories, you can make a bash script and let the computer do all the selecting and copying on its own. If you're completely new to bash or shell scripting, I'd recommend you to read a little beforehand about what it is capable of and getting a basic understanding of the appearence of variables and loops, so you don't get too confused.

I'm using the top of http://www.cyberciti.biz/faq/bash-loop-over-file/ as a reference now. Our script will look basically like this:

#!/bin/bash
FILES="file1
/path/to/file2"
OUTPUT="merge.txt"
for f in $FILES
do
    cat $f >> $OUTPUT
done

(Oh, I hope this will work)

If all your files share something that differenciate them from other files we don't want to merge, that's exactly what we need. Ideally they share a common parent or anchestor folder where you can't find any other files but ours. Then we can simply use /the/path/to/that/folder/* as selector when defining our $FILES variable. As you see in the model, we can set multiple selectors. I hope you don't care too much about the order the files will be selected automatically, else we'd have to think about some way to specify this... For our example above, we could set:

FILES="*/*" or FILES="*/*.txt" or FILES="a/* b/*" or so and OUTPUT="merge.txt" of course.

Some last thoughts

Maybe we'll need to pay attention to not selecting our merging file with "*", that may end awkward... Above I therefore included one / everytime. You also could simply put it somewhere else.

If your files follow some pattern, like p_some-thing-axy-1,p_some-thing-axy-1,p_some-thing-bxz-1, you can choose some selector like p_some-thing-* or so.

Now I hope you can make some use of that, because I'm out.

Good luck! :D