youtube-dl file download progress with zenity progress bar

Question

How to add youtube-dl file download progress percentage to zenity progress bar

sample code (just an example, not a working one)

#!/bin/sh
(
   progress=$(youtube-dl --extract-audio --audio-quality 0 --newline --audio-format mp3 https://www.youtube.com/playlist?list=PL1C815DB73EC2678E)
per=$(awk '{print perc}' <<<$progress)
time=$(awk '{print time}' <<<$progress)
file_no=$(awk '{print file_no}' <<<$progress) #only for playlist, example=Downloading video 1 of 4 

echo "$per" ; sleep 1
echo "# $file_no \n Time Left: $time" ; sleep 1

) |
zenity --progress \
  --title="Download" \
  --text="Downloading..." \
  --percentage=0

if [ "$?" = -1 ] ; then
        zenity --error \
          --text="Download cancelled."
fi

i have used this code to get download progress

youtube-dl --extract-audio --audio-quality 0 --newline --audio-format mp3 https://www.youtube.com/playlist?list=PL1C815DB73EC2678E

This is the out put

[youtube:playlist] PL1C815DB73EC2678E: Downloading webpage
[download] Downloading playlist: Less than 1 minute
[youtube:playlist] playlist Less than 1 minute: Collected 4 video ids (downloading 4 of them)
[download] Downloading video 1 of 4
[youtube] KNLwsqzFfNg: Downloading webpage
[youtube] KNLwsqzFfNg: Extracting video information
[youtube] KNLwsqzFfNg: Downloading DASH manifest
download] Destination: _1 min. - Amendes pour les particules du LHC-KNLwsqzFfNg.m4a

[download]   0.4% of 231.51KiB at  6.10KiB/s ETA 00:30
[download]   1.1% of 231.51KiB at 27.07KiB/s ETA 00:10
[download]   4.0% of 231.51KiB at 19.24KiB/s ETA 00:04
[download]   6.5% of 231.51KiB at 75.06KiB/s ETA 00:03
[download]  13.4% of 231.51KiB at 98.22KiB/s ETA 00:03
[download]  28.7% of 231.51KiB at 81.40KiB/s ETA 00:02
[download]  61.7% of 231.51KiB at 91.56KiB/s ETA 00:01
[download]  86.2% of 231.51KiB at 82.96KiB/s ETA 00:00
[download] 100.0% of 231.51KiB at 73.21KiB/s ETA 00:00
[download] 100% of 231.51KiB in 00:02
[ffmpeg] Correcting container in "_1 min. - Amendes pour les particules du LHC-KNLwsqzFfNg.m4a"
WARNING: Your copy of avconv is outdated, update avconv to version 10-0 or newer if you encounter any errors.
[avconv] Destination: _1 min. - Amendes pour les particules du LHC-KNLwsqzFfNg.mp3
WARNING: Your copy of avconv is outdated, update avconv to version 10-0 or newer if you encounter any errors.
Deleting original file _1 min. - Amendes pour les particules du LHC-KNLwsqzFfNg.m4a (pass -k to keep)
[download] Downloading video 2 of 4
[youtube] wTvXkMpJflk: Downloading webpage
[youtube] wTvXkMpJflk: Extracting video information
[youtube] wTvXkMpJflk: Downloading DASH manifest
etc..
etc..
.
.

and i want only

Downloading video 1 of 4 [download] Downloading video 2 of 4

as $files_no

FIRST FILE

file_no= Downloading video 1 of 4

per      time             rate
0.40%   00:30:00    6.10KiB/s
1.10%   00:10:00    27.07KiB/s
4.00%   00:04:00    19.24KiB/s
6.50%   00:03:00    75.06KiB/s
13.40%  00:03:00    98.22KiB/s
28.70%  00:02:00    81.40KiB/s
61.70%  00:01:00    91.56KiB/s
86.20%  00:00:00    82.96KiB/s
100.00% 00:00:00    231.51KiB/s

SECOND, THIRD...FILES

As separate variable $file, $per, $time i know we can use awk but for this complicated output how should i use it. if all parameters are not possible, can at least percentage and file_no be extracted.

Answer 1

Yes, it is possible. You need to

1. Make sure that output is unbuffered, that it is printed as soon as it's received. Pipes are buffered by default.
2. Parse the downloader's output so that only the percentages are printed;
3. Parse the output so that the file number is printed with a # at the beginning of the line. Zenity will automatically update the text of its dialog with lines starting with #.

Combining the above, and implementing a little regex magic, we get:

#!/bin/bash
youtube-dl --extract-audio --audio-quality 0 --newline --audio-format mp3 \
           https://www.youtube.com/playlist?list=PL1C815DB73EC2678E | 
 grep --line-buffered -oP '^\[download\].*?\K([0-9.]+\%|#\d+ of \d)' |
    zenity --progress \
  --title="Download" \
  --text="Downloading..." \
  --percentage=0 

Explanation

The --line-buffered option makes grep print its output immediately, turning off the default buffering. The -o makes it print only the matched portion of the line and the -P turns on Perl Compatible Regular Expressions.

The regex is a little complex, so let's break it down:

• ^\[download\] : matches lines that start with [download].
• .*? : 0 or more characters, but the ? makes it stop at the shortest possible match.
• \K : this is basically a lookbehind, it means "ignore anything matched so far".
• (...|...) : the | means OR. Therefore, (A|B) wil match either A or B.
• [0-9.]+\% : 1 or more numbers or . followed by a %. This prints the percentage.
• #\d+ of \d : a # followed by one or more digits, of and then one or more digits again. This matches the "Video X of Y" line.

Taken together, that grep command will print:

#1 of 4
0.1%
0.3%
0.8%
1.7%
3.4%
7.0%
14.0%
28.2%
56.5%
99.5%
100.0%
100%
#2 of 4
0.1%
0.3%
0.8%
1.6%
3.4%
6.9%
13.9%
27.8%
55.8%
[...]

etc, and that's precisely the output that zenity needs. Finally, you can make the whole thing more useful by implementing the ability to specify multiple URLs from the command line:

#!/bin/bash
for url in "$@"
do
  youtube-dl --extract-audio --audio-quality 0 --newline --audio-format mp3 \
           https://www.youtube.com/playlist?list=PL1C815DB73EC2678E | 
   grep --line-buffered -oP '^\[download\].*?\K([0-9.]+\%|#\d+ of \d)' |
    zenity --progress \
  --title="Download" \
  --text="Downloading..." \
  --percentage=0 
done

Then, you can call your script like this:

myscript.sh "http://url1.com" "http://url2.com" "http://urlN.com