Automating file transfer using Bash scripts in SSH on my new server from old server

Question

How can I connect to a remote host using SSH and create one Bash script to copy all files and folder from my old server to my new server for backup every day?

Answer 1

Set up key-based ssh authentication

First, you need to generate a ssh key. On the machine you're connecting from, run:

$ ssh-keygen
Generating public/private rsa key pair.
Enter file in which to save the key (/home/vidarlo/.ssh/id_rsa):
Enter passphrase (empty for no passphrase): 
Enter same passphrase again: 
Your identification has been saved in /home/vidarlo/.ssh/id_rsa.
Your public key has been saved in /home/vidarlo/.ssh/id_rsa.
The key fingerprint is:
SHA256:/jxfxiWiao0m7YG9MiHgXBFKoo7kJcgTOrPtAZNtpVg vidarlo@hannah.bitsex.no
The key's randomart image is:
+---[RSA 2048]----+
|..E o.           |
|=B.+.            |
|@==. .           |
|=O= .            |
|o=oo    S   . . .|
| .o.. .+   . o o |
|  .  ..o+o.   +  |
|      + =*o  o   |
|       B+ oo.    |
+----[SHA256]-----+
[~]$ 

Just press enter when asked; the default locations and no passphrase is OK.

This will generate a private and public key. Next step is to copy the public key to the remote server, so that it can be used. ssh-copy-id can be used for this:

$ ssh-copy-id user@host
/usr/bin/ssh-copy-id: INFO: Source of key(s) to be installed: "/home/vidarlo/.ssh/id_rsa.pub"
/usr/bin/ssh-copy-id: INFO: attempting to log in with the new key(s), to filter out any that are already installed
/usr/bin/ssh-copy-id: INFO: 1 key(s) remain to be installed -- if you are prompted now it is to install the new keys
user@host's password: 

Number of key(s) added: 1

Now try logging into the machine, with:   "ssh 'user@host'"
and check to make sure that only the key(s) you wanted were added.

At this stage you should be able to run ssh user@host, and get logged in without entering a password.

Backup job

You want a simple scp. This is bad for several reasons:

1. You don't get any history. If a file is overwritten by mistake, and you do not discover it before the next backup job, scp will happily overwrite the content.
2. You have to copy all the content every night.
3. You don't get a status report.
4. If a backup job does not finish in time, you risk having two backup jobs writing to the same content.

But, anyway. This can be done, as long as you're aware of the caveats. Use crontab -e to edit your users crontab. Insert a line like this:

0 5 * * * /usr/bin/scp "/path/to/backup" "user@remote:/path/to/store/backups"

This command will run nightly at 05:00. This can be changed if you desire. The explanation of the fields is as follows:

1. minutes, 0-60. 0 means run it at xx:00, * means run it every minute
2. hours, 0-23. 02 means 02:xx. * means every hour.
3. Day of month, 1-31. * means every day.
4. Month, 1-12. * is every month
5. Day of Week, 1-7.