I am using jupyter notebook on virtual environment to extract data using selenium. I have used a shell script to run the jupyter notebook on cronjob as recommended.
#!/bin/bash
/home/username/environments/venv/bin/jupyter nbconvert
--execute
--to notebook /home/username/environments/venv/jupyter_scripts/Untitled1.ipynb
--output /home/username/environments/venv/jupyter_scripts/Untitled2.ipynb \
I am able to run this script on terminal and it is giving the output csv file. But when I run the same script on cronjob, it’s not giving me the output csv file. Following is the log from cronlog:
Jan 28 06:57:01 username cron[23859]: (root) RELOAD (crontabs/root)
Jan 28 06:57:02 username CRON[25475]: (root) CMD (/home/username/environments/project/script.sh >/dev/null 2>&1)
Which means the shell script is getting executed. Yet I am not getting the output csv file. Can anyone please help me understand what is the issue here? How can I resolve this issue?
Edit: Following is the cronjob
57 06 * * * /home/username/environments/project/script.sh >/dev/null 2>&1
Then change the cron cmd as follows to get the log in a separate log file than the cronlog as follows.
/home/username/environments/project/script.sh > /home/username/environments/project/script.log 2>&1
From script.log I found the following error
[NbConvertApp] ERROR | Error while converting '/home/username/environments/venv/jupyter_scripts/Untitled1.ipynb'
But when running ./script.sh I got the following log on terminal
username@username:~/environments/project$ ./script.sh
[NbConvertApp] Converting notebook /home/username/environments/venv/jupyter_scripts/Untitled1.ipynb to notebook
[NbConvertApp] Executing notebook with kernel: python3
[NbConvertApp] Writing 3543 bytes to /home/username/environments/venv/jupyter_scripts/Untitled2.ipynb
And the desired csv output was also created while running the script on terminal.
