4

For background: my end goal is to add a clean install of Python 3.11 onto my Ubuntu 20.04 machine so that I can test some code I wrote that uses some of Python 3.11's new features. I'm not trying to replace my system Python or anything like that, I just want Python 3.11 for testing purposes.

What I've tried: First, I tried installing Python 3.11 directly through apt, but when I did apt-get install python3.11; python3.11 --version, the output Python version was Python 3.11.0rc1. But I want to install the final, stable version of Python 3.11, not a release candidate, so I uninstalled the Python 3.11rc I'd just installed.

A quick search led me to the deadsnakes PPA, so I added the deadsnakes PPA, updated apt, and successfully installed Python 3.11.0, this time not a release candidate (verified by python3.11 --version). I then tried to verify that I had a clean install by python3.11 -m pip list, but it listed almost 70 packages. I tried removing all these packages using python3.11 -m pip remove <packages>, but this failed because some of the packages in the newly-installed Python 3.11 were built using distutils. I tried to check if there was a workaround to uninstall Python packages that were built using distutils, but I quickly determined that that is too far beyond my knowledge for me to comfortably tinker, if not outright impossible.

Thus, here's what I did up until now:

# first attempt at installing Python 3.11

> python3.11 --version #to ensure that I don't already have Python 3.11 installed
Command 'python3.11' not found, but can be installed with:
apt install python3.11
> python3 --version #to double ensure that my system Python version isn't 3.11
Python 3.10.6
> apt-get install python3.11 -y
> python3.11 --version
Python 3.11.0rc1
> apt-get remove python3.11 -y #removing unwanted rc version
> apt autoremove #removing additional python3.11-minimal and other packages that came when installing python3.11


second attempt at installing Python 3.11


> add-apt-repository ppa:deadsnakes/ppa --yes
> apt update; apt-get update;
> apt-get install python3.11 -y
> python3.11 --version
Python 3.11.0
> python3.11 -m pip list
blinker                1.4
certifi                2020.6.20
chardet                4.0.0
<66 more packages cut for brevity...>
> python3.11 -m pip uninstall blinker certifi chardet <66 more packages cut for brevity> --yes
Found existing installation: blinker 1.4
ERROR: Cannot uninstall 'blinker'. It is a distutils installed project and thus we cannot accurately determine which files belong to it which would lead to only a partial uninstall.

Not sure what to do, I read the deadsnakes main page more closely, and I saw this:

The packages provided here are loosely based on the debian upstream packages with some modifications to make them more usable as non-default pythons and on ubuntu.

Perhaps I wildly misinterpreted this, but I read this to mean "When you install Python from deadsnakes, we include the same set of packages that Ubuntu's system Python would include, plus a few extras." That appears to explain my problem, or at least, it would explain why the Python 3.11 I installed has all the same packages as my system Python, plus a few extra.

So I tried finding a way to install just a clean, empty version of Python 3.11. I saw during my earlier attempts that I was actually installing multiple packages, including python3.11-minimal. I figured maybe that would be what I needed, so I uninstalled Python3.11 again and installed python3.11-minimal. This installed Python3.11, but python3.11 -m pip list again returned a list of 69 packages, and like before, I couldn't remove them because some or all of them were built with distutils.

I thought it might be an issue specifically with Python 3.11, so I tried installing older versions of python to see if they also came with extra packages, but when I installed Python 3.8 and Python 3.9, they came with a bunch of extra packages just like Python 3.11 did.

I tried to find any reference online to installing a truly empty version of Python, or at least one that you can remove the extra packages from after install, but I couldn't find anything. I can't imagine it's a terribly uncommon thing to want though, because surely someone else has wanted to install a clean copy of non-system Python that doesn't come with an additional 60+ packages at some point, right?

So how can I install a clean copy of Python in addition to my system Python, that doesn't come with a bunch of unremovable packages bundled? Is it possible?

Is it possible, in Ubuntu, to install a version of python in addition to your system python, and then create a virtual environment with that secondary python, and get that virtual environment to a state such that, when you issue a pip list from within that new virtual environment, pip lists no packages outside of the stdlib (except maybe pip/setuptools)?

Raffa
  • 32,237
Nick Muise
  • 143
  • First create a playground(Virtual Environment) for your specific python version then play ... Please see https://askubuntu.com/a/1348136 and https://askubuntu.com/a/1321433 and https://askubuntu.com/a/1226562 – Raffa Dec 02 '22 at 08:13
  • @Nmath I agree - I don't want to change my system Python, I want to add Python 3.11 to test code. I wanted to create a Python 3.11 virtual environment for each app I want to test, but since every attempt at installing Python 3.11 has given me a Python 3.11 with a bunch of unremovable packages installed into that Python, any Python 3.11 virtual environments I create likewise inherit those unremovable packages. That is, apt-get install python3.11; python3.11 -m venv venv; . venv/bin/activate; pip list shows 60+ packages. Is this just how it works on Ubuntu? – Nick Muise Dec 02 '22 at 20:25
  • @Nmath I guess my confusion is, I expected that installing Python 3.11 would require some additional packages to be installed at the OS level, but I didn't expect that it would also require additional packages to be installed into the new Python 3.11 itself. – Nick Muise Dec 02 '22 at 20:30
  • I would like to emphasize again that I am not altering my system python. And while I appreciate the good information Raffa shared in the other questions, they don't answer my question. The first answer explains how to install a version of python separate from your system python and create a venv with it, both of which I've already done. The second explains that there's no way to "restore your system python to a clean state as it would have shipped with Ubuntu", but I'm not trying to restore my system python. The third explains that Python comes with the stdlib, (...) – Nick Muise Dec 02 '22 at 23:29
  • but Twisted, Jinja2, lazr, PyJWT, and the other nearly 70 packages I see when I pip list from within a freshly-created virtual environment created within the freshly-installed Python 3.11, are NOT part of the stdlib, so the stdlib has no bearing on why I'm seeing these packages. – Nick Muise Dec 02 '22 at 23:34
  • We're definitely having some miscommunication, so I will try asking a slightly different question - forget everything else I've said, even: is it possible, in Ubuntu, to install a version of python in addition to your system python, and then create a virtual environment with that secondary python, and get that virtual environment to a state such that, when you issue a pip list from within that new virtual environment, pip lists no* packages outside of the stdlib (except maybe pip/setuptools)?* If no, then I guess Python fundamentally does not work the same as it does on Windows. – Nick Muise Dec 02 '22 at 23:36

1 Answers1

1

Is it possible, in Ubuntu, to install a version of python in addition to your system python, and then create a virtual environment with that secondary python, and get that virtual environment to a state such that, when you issue a pip list from within that new virtual environment, pip lists no packages outside of the stdlib (except maybe pip/setuptools)?

Yes, Please see the demonstration below assuming you have already installed python3.11 ... For reference here is system-wide python3.11 -m pip list:

ubuntu@Lenovo:~$ python3.11 -m pip list
Package                Version
---------------------- ---------------
agate                  1.6.3
agate-dbf              0.2.2
agate-excel            0.2.5
agate-sql              0.5.8
apturl                 0.5.2
attrs                  21.2.0
Babel                  2.8.0
bcc                    0.18.0
bcrypt                 3.2.0
beautifulsoup4         4.10.0
blinker                1.4
Brlapi                 0.8.3
certifi                2020.6.20
chardet                4.0.0
chrome-gnome-shell     0.0.0
click                  8.0.3
colorama               0.4.4
command-not-found      0.3
cryptography           3.4.8
csvkit                 1.0.6
cupshelpers            1.0
dbfread                2.0.7
dbus-python            1.2.18
defer                  1.0.6
distro                 1.7.0
distro-info            1.1build1
duplicity              0.8.21
et-xmlfile             1.0.1
fasteners              0.14.1
future                 0.18.2
greenlet               1.1.2
html5lib               1.1
httplib2               0.20.2
idna                   3.3
importlib-metadata     4.6.4
iniconfig              1.1.1
isodate                0.6.1
jdcal                  1.0
jeepney                0.7.1
keyring                23.5.0
language-selector      0.1
launchpadlib           1.10.16
lazr.restfulclient     0.14.4
lazr.uri               1.0.6
leather                0.3.4
libevdev               0.5
libvirt-python         8.0.0
lockfile               0.12.2
louis                  3.20.0
lxml                   4.8.0
macaroonbakery         1.3.1
Mako                   1.1.3
MarkupSafe             2.0.1
monotonic              1.6
more-itertools         8.10.0
netaddr                0.8.0
netifaces              0.11.0
oauthlib               3.2.0
olefile                0.46
openpyxl               3.0.9
packaging              21.3
paramiko               2.9.3
parsedatetime          2.6
pexpect                4.8.0
Pillow                 9.0.1
pip                    22.0.2
pluggy                 0.13.0
protobuf               3.12.4
ptyprocess             0.7.0
py                     1.10.0
pycairo                1.20.1
pycups                 2.0.1
Pygments               2.11.2
PyGObject              3.42.1
PyICU                  2.8.1
PyJWT                  2.3.0
pymacaroons            0.13.0
PyNaCl                 1.5.0
pyparsing              2.4.7
pyRFC3339              1.1
pytest                 6.2.5
python-apt             2.3.0+ubuntu2.1
python-dateutil        2.8.1
python-debian          0.1.43ubuntu1
python-slugify         4.0.0
pytimeparse            1.1.5
pytz                   2022.1
pyudev                 0.22.0
pyxdg                  0.27
PyYAML                 5.4.1
reportlab              3.6.8
requests               2.25.1
SecretStorage          3.3.1
setuptools             59.6.0
six                    1.16.0
soupsieve              2.3.1
SQLAlchemy             1.4.31
ssh-import-id          5.11
systemd-python         234
toml                   0.10.2
ubuntu-advantage-tools 27.11.3
ubuntu-drivers-common  0.0.0
ufw                    0.36.1
unattended-upgrades    0.1
Unidecode              1.3.3
urllib3                1.26.5
usb-creator            0.3.7
vboxapi                1.0
wadllib                1.3.6
webencodings           0.5.1
wheel                  0.37.1
xdg                    5
xkit                   0.0.0
xlrd                   1.2.0
zipp                   1.0.0

First method(With python3.11-venv ... Jammy and above):

ubuntu@Lenovo:~$ sudo apt install python3.11-venv
.
.
.
ubuntu@Lenovo:~$ mkdir venv_1 && cd venv_1 
ubuntu@Lenovo:~/venv_1$ python3.11 -m venv env
ubuntu@Lenovo:~/venv_1$ source env/bin/activate
(env) ubuntu@Lenovo:~/venv_1$ python3.11 -m pip list
Package    Version
---------- -------
pip        22.0.2
setuptools 59.6.0
(env) ubuntu@Lenovo:~/venv_1$

Second method(With virtualenv -p [ path to python3.11 executable ] ... Find the executable with e.g. which python3.11):

ubuntu@Lenovo:~$ pip3 install virtualenv
.
.
.

ubuntu@Lenovo:~$ mkdir venv_2 && cd venv_2 
ubuntu@Lenovo:~/venv_2$ virtualenv env -p /bin/python3.11
ubuntu@Lenovo:~/venv_2$ source env/bin/activate
(env) ubuntu@Lenovo:~/venv_2$ python3.11 -m pip list
Package    Version




pip        22.3.1
setuptools 65.6.3
wheel      0.38.4
(env) ubuntu@Lenovo:~/venv_2$
Raffa
  • 32,237
  • Installing python3.11-venv instead of python3.11 did it! I feel silly for not catching that before. Interestingly, the Python I get from installing python3.11-venv has the same set of packages installed into it as the Python I got from installing python3.11 before, including even virtualenv, but of course only the python3.11-venv python creates virtual environments that are clean. Do you know what the difference is between these two Pythons that makes them work totally differently despite having the same set of Python modules bundled in? – Nick Muise Dec 05 '22 at 14:52
  • @NickMuise The thing here is that you are installing a python3 version that is newer than the default Ubuntu one and you need to install the correct version venv module for it ... hence you need python3.11-venv which contains pyvenv-3.11 ... It will install python3.11 as well as a dependency ... Please compare apt show python3.11-venv vs. apt show python3-venv ... :-) – Raffa Dec 05 '22 at 15:30