1

I want to schedule a large download to occur at night, between 12AM and 5AM. Everyday at 12AM wget should automatically pick up where it left the night before, and at 5AM, it should automatically pause the download, so it can resume the following night.

How can I script this in Ubuntu? What tools do I need?

jonvon
  • 118
a06e
  • 13,223
  • 26
  • 70
  • 104

2 Answers2

2

A combination of an Upstart job and a cron job:

First, create an Upstart job. Create a file called ~/.config/upstart/my-download.conf containing:

description "My download job."
start on resume-download
stop on stop-download

respawn
respawn limit 10 5

script   
     cd /path/to/download/folder
     wget --continue uri://link/to/download    
end script

Now, add a cron job to start and stop this Upstart job. Run EDITOR=gedit crontab -e and add to the file:

0 0 * * * /sbin/initctl emit resume-download
0 5 * * * /sbin/initctl emit stop-download

You can manually start the download any time you want using:

initctl emit resume-download
# or
start my-download

And stop it any time using:

initctl emit stop-download
# or
stop my-download
  1. Due to the respawn in the job, Upstart will try to restart wget if it exits. This can be useful if it timed out and exited at two in the morning. A limit is placed so that we don't have Upstart trying to start wget infinitely (e.g., the network is facing extended problems, the download was completed, etc.).
  2. We added initctl emit commands in the cron job. This way, if we create more such Upstart jobs, all of them starting on this event, this single cron entry will be enough to start all of them.
Community
  • 1
muru
  • 197,895
  • 55
  • 485
  • 740
0

You will want to use the --continue flag of wget and cron jobs.

You can have a cron entry to start a script at 12AM with using timeout. See https://serverfault.com/questions/257345/can-i-limit-the-runtime-of-a-cronjob. Or you could run another script that kills/interrupts it at 5AM.

Example for a daily script: 0 0 * * * timeout -s KILL 5h /path/to/command

The above example will run timeout on /path/to/command at 12:00am on every day of every month and send it a KILL signal after 5 hours.

This is how a cron job is laid out:

minute (0-59), hour (0-23, 0 = midnight), day (1-31), month (1-12), weekday (0-6, 0 = Sunday) /path/to/command

An asterisk (*) can be used so that every instance (every hour, every weekday, every month, etc.) of a time period is used.

Community
  • 1
jonvon
  • 118