1

I have a script that begins with

#!/usr/bin/env python3

Should it use the "python3" virtual environment with it having to be explicitly activated? I'm pretty sure this used to work on previous versions of Ubuntu.

I am calling the script from the crontab, but it can't find the Python dependencies (which are definitely installed).

57 10 * * * root /opt/cleanup.py >> /var/log/cleanup.log

Traceback (most recent call last):
  File "/opt/cleanup.py", line 3, in <module>
    import boto3
ModuleNotFoundError: No module named 'boto3'

My dependencies:

(python3) root@ip[prod]:/opt/virtualenv/python3$ pip list
Package            Version
------------------ ---------
boto3              1.26.8
botocore           1.29.8
certifi            2022.12.7
charset-normalizer 2.1.1
idna               3.4
jmespath           1.0.1
pip                23.0.1
python-dateutil    2.8.2
requests           2.28.1
s3transfer         0.6.0
setuptools         59.6.0
six                1.16.0
urllib3            1.26.12
wheel              0.37.1

The cleanup.py script:

#!/usr/bin/env python3
import requests
import boto3
import sys
import urllib3
import time

I am on Ubuntu 22.04.1 LTS.

crmpicco
  • 213
  • 1
  • 3
  • 8

1 Answers1

0

You shouldn't rely on /usr/bin/env python3 pointing to the same path/environment when running it as root(or any other different user for that matter) ... Under your user, usually it will first look in /home/user/.local/bin while under root, it usually won't ... See the difference with e.g.:

# For your user … "ubuntu" is my test user
$ echo "$PATH"
/home/ubuntu/.local/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/snap/bin

For "root"


$ sudo su root -c 'echo "$PATH"'
/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin

And you are clearly asking cron to run your script as root i.e.:

57 10 * * * root /opt/cleanup.py >> /var/log/cleanup.log

/usr/bin/env is actually a command/executable (Not to be mixed with Python virtual environments) ... It finds the passed executable based on the search path of the invoking user ... Hence with root as in the example above /usr/bin/env python3 will actually translate to /usr/bin/python3(as for the part of interpreting the passed executable path) as the first match in the search path.

You can see and test which Python environment your shebang will invoke when run as root with e.g.:

sudo su root -c '/usr/bin/env python3'

Part of activating a Python virtual environment, in fact, is to modify the default user's path so it begins with the path to the virtual environment itself i.e. in the venv/bin/activate file with something like:

VIRTUAL_ENV='/full/path/to/venv'
export VIRTUAL_ENV
PATH="$VIRTUAL_ENV/bin:$PATH"
export PATH

It's not advised, however, to manually modify the default user's path/environment for the sole purpose of running some Python script file in a certain Python virtual environment as it can cause issues for other scripts/applications or even render them unusable and might break your system as well ... You can simply do that by setting the shebang in your script to the path of the Python executable in the virtual environment you want e.g.:

#!/opt/virtualenv/python3

Activating a virtual environment prior to using it is not necessary in this case ... Please see: How venvs work.

And, to verify, you can list installed modules in that virtual environment with e.g.:

/opt/virtualenv/python3 -m pip list

It might be worth noting as well that the /opt directory is reserved/used for add-on application software packages as per the Linux FHS(Filesystem Hierarchy Standard) … So, you might want to comply with that for good practice.

Raffa
  • 32,237