0

I have a directory on my Ubuntu server that has many subdirectories and files. I want to download entire folder, so I used this command:

scp -r [email protected]:/path/to/target_directory /path/to/local_directory

So target_directory ends up in /path/to/local_directory/target_directory. This is fine so far.

The problem: target_directory is not so big (around 50MB) but has many tiny files. scp download files one by one, which takes too long. So I'm thinking may be there's a way to compress the directory and download it (optionally, without keeping the compressed file on the server). I think it's achievable but not sure how.

Note: I use Mac for my local machine.

evilReiko
  • 155

2 Answers2

3

Thanks to @muru for answer and explanation. I'm posting this to remind myself and anyone else who might need this.

rsync is really fast! This is how to do it with rsync:

rsync -azP [email protected]:/path/to/target_directory/ /path/to/local_directory
  • -a will (recursively) copy everything
  • -z will compress the files,
  • -P will show you progress bar for each file

rsync allows you to resume downloading in case of interruption or network failure just by running same command.

Note: the / in the right end of the source /path/to/target_directory/ means it will copy the content of target_directory (not the directory itself). If you want to copy the directory itself then remove this trailing slash.

Read more: https://www.digitalocean.com/community/tutorials/how-to-use-rsync-to-sync-local-and-remote-directories-on-a-vps

muru
  • 197,895
  • 55
  • 485
  • 740
evilReiko
  • 155
1

You don't really need compression, if the problem is that there are many small files. Use tar to make an archive on Ubuntu, and instead of saving the archive to a file, send it to stdout (the default for Ubuntu's GNU tar when there's no file specified) and read the tar file on the other end:

ssh ubuntu-server tar c -C /path/to/target_directory . | tar xvf - -C /path/to/local_directory

Or use a smarter tool like rsync.

muru
  • 197,895
  • 55
  • 485
  • 740
  • This is what I've happen looking for! Would be great if you can provide rsync example like the one you wrote for ssh. – evilReiko May 13 '20 at 08:02
  • 1
    @evilReiko pretty much what you'd do with scp, just that the options are different: rsync -aP [email protected]:/path/to/target_directory /path/to/local_directory (-a ~ archive - preserve owner, mode, etc., -P show progress) – muru May 13 '20 at 08:11