2

apt-mark showmanual lists a lot of packages for me:

root@hcompneu:~# apt-mark showmanual | wc -l
4013
root@hcompneu:~# apt-mark showmanual | head
a11y-profile-manager-indicator
abiword
abiword-common
abiword-plugin-grammar
account-plugin-facebook
account-plugin-flickr
account-plugin-google
account-plugin-identica
account-plugin-twitter
accountsservice
root@hcompneu:~# apt-mark showmanual | tail
yelp-xsl
zeitgeist
zeitgeist-core
zeitgeist-datahub
zenity
zenity-common
zip
zlib1g
zlib1g-dev
zlib1g:i386

As I understand, it should only list those packages which I manually installed. I don't remember in each case anymore but I'm pretty sure I did not install all of those manually. Actually, I rarely installed any software on this system, despite Chrome, OpenOffice and some other very basic things. So I would expect a list of maybe 10-100 packages but not more. Maybe it also includes ubuntu-desktop, grub and linux or so from the base Ubuntu installation.

Maybe in the past though I wrote some scripts to fix up a broken system by automatically calling apt-get install XXX on various packages. Or maybe I did other things. I don't really remember anymore.

Could that be the case here? So I somehow messed it up? If that is the case, what would you suggest? Mark all to automatically installed, and then by hand add some selected packages like ubuntu-desktop?

Or is such output correct and I understand it wrongly? I read this and I'm not sure. It looks like it is much more complicated to get the list of manually installed packages? Or why is the answer so complicated?

Or is apt-mark showmanual buggy?

root@hcompneu:~# ls -ld /var/log/installer/
drwxr-xr-x 2 root root 4096 Dez 27 11:27 /var/log/installer/
root@hcompneu:~# grep -r cdrom /etc/apt/ --include=*.list
root@hcompneu:~# dpkg -l | grep ^ii | wc -l
3812
Albert
  • 2,577
  • Any chance you use this update-applet (i never know the name) to update your system? – nobody Dec 27 '21 at 11:15
  • 3
    How old is your system? Please show output from ls -ld /var/log/installer/ and grep -r cdrom /etc/apt/ --include=*.list and dpkg -l | grep ^ii | wc -l. On my 7 year system I have 3953 manually installed packages of 5680 in total; on my 11 year system I have 3947 manually installed packages of 6395 in total. Some of them were installed by me, some - as dependencies. So there is no reason to be afraid. – N0rbert Dec 27 '21 at 13:10
  • @N0rbert But those installed as dependencies should not be in that list. Only those which you directly installed via apt-get install XXX, not the dependencies. Or actually that is my question, if this is the case. That is my understanding at least. – Albert Dec 27 '21 at 14:13
  • I was mistaken about dependencies, they are not marked as manually installed. But you do not answer the questions and do not provide the requested command output. We both share a love to Gentoo, but please provide additional info. – N0rbert Dec 27 '21 at 14:34
  • @N0rbert I added the outputs of the commands. I don't really know how old the system is but I think it is old. Maybe started with Ubuntu 8 or so and went through 10, 12, 14 and 16, while I recently updated to 18. However, despite Chrome, OpenOffice and some other very basic things, I never installed anything manually. So definitely most things in the list were not manually explicitly installed by me. So maybe the answer to my question is simply yes, it is messed up? – Albert Dec 27 '21 at 15:15

4 Answers4

4

TL;DR No, your system is not messed up. Moreover, you can leave all things as they are, so in the current state.

Let's consider three systems:

Current System Installation media Age dpkg -l | grep ^ii | wc -l apt-mark showmanual | wc -l
Ubuntu 18.04.6 LTS with MATE desktop(*) Ubuntu 12.04.4 LTS 7 y.o. 5680 3953
Ubuntu 18.04.6 LTS with MATE desktop(*) Ubuntu 10.04 LTS 11 y.o. 6395 3947
Yours Ubuntu 18.04 LTS (**) ~Ubuntu 8.04 LTS ~13 y.o. 3812 4013

Notes:

  • (*) my systems were originally installed from Ubuntu desktop installation disks, thus shipped with GNOME and/or Unity (not a flavour). Then I have installed MATE desktop using official PPAs for 12.04 LTS, 14.04 LTS and their meta-packages; then when 16.04 LTS were released I have re-installed MATE desktop as task package using commands like sudo apt-get install ubuntu-mate-desktop^. This installation of task package causes all such packages to be marked as manually installed and thus protects them from removal. All other software were installed manually during life-span of the both systems.
  • (**) the strange thing about your system is that you have total number of installed packages 3812 less than manually installed packages 4013. You have to investigate this using Synaptic.

I tried to conduct some research with fresh installations of old and new systems. Below is the table about clean installations of ordinary Ubuntu desktop ISOs.

Ubuntu version dpkg -l | grep ^ii | wc -l aptitude search '~i!~M' | wc -l apt-mark showmanual Synaptic: Status → Installed (manual)
6.06 LTS 1046 1046 n/a n/a
8.04 LTS 1102 1087 n/a n/a
10.04 LTS 1296 1286 n/a 1286
12.04 LTS 1392 1367 1367 1342
14.04 LTS 1722 1690 1690 1666
16.04 LTS 1745 220 220 195
18.04 LTS 1602 1557 1557 1529
20.04 LTS 1607 49 49 49
22.04 LTS (dev) 1700 44 44 31

We can understand here that functionality of Ubiquity installer and APT by itself have changes over time.

From the above table we can see that old systems have huge numbers of manually installed packages from their day 0 of installation.
Newer Ubuntu versions have smaller numbers, this ease their maintenance. They probably have more meta-packages installed which prevent package removal because of dependencies.

Another experiment I have done: I upgraded fresh Ubuntu installation from 8.04 LTS to 10.04 LTS → 12.04 LTS → 14.04 LTS → 18.04 LTS → 20.04 LTS and in the resulting 20.04.3 LTS system I have 1867 manually installed packages of 2913 total packages.

The "manual" installation status of package prevents its removal, so you should not set package as auto-installed.

Good resources to read:

  • Chapter 2. Debian package management of Debian Reference.

  • man apt-mark locally or online, the below quote is essential:

    apt-mark - show, set and unset various settings for a package
    [...]
    showmanual can be used in the same way as showauto except that it will print a list of manually installed packages instead.

N0rbert
  • 99,918
  • 1
    I wonder, do you really have installed all those almost 4k packages manually? Or is it maybe also messed up in your case? As said before, dependencies should not be in that list. – Albert Dec 27 '21 at 16:06
  • I did some experiments on fresh Ubuntu MATE 22.04 LTS daily VM - it has the following numbers apt-mark showmanual | wc -l returns 30 (I suppose that these packages were installed by Ubiquity to support needed locales and so on), while total number of packages is 1971. Installation of each next package increase the list of manually installed packages by 1 (as expected). For 18.04 LTS it needs further investigation. – N0rbert Dec 27 '21 at 16:24
  • @Albert Yes, it is possible to have more than 4k manually installed packages. For eg:- If N0rbert installs kde desktop (or any desktop) they will also install various apps with it (around 100!) And all those apps will also be shown in the manually installed list. So it doesn't mean that having 4k manually installed packages means, you have run apt install 4k times! (The suggested application packages and other extra packages (not including dependencies) are also manually installed packages ;) – Error404 Dec 28 '21 at 05:13
  • @N0rbert How did you installed Ubuntu Mate 22.04? Is it released? – Error404 Dec 28 '21 at 05:15
  • @someone I have installed Ubuntu MATE 22.04 LTS for testing purposes from http://cdimage.ubuntu.com/ubuntu-mate/daily-live/current/ using daily ISO. – N0rbert Dec 28 '21 at 07:31
  • 1
    @someone No, if you do apt-get install kubuntu-desktop, that will install a lot of packages, but only one single package will be added to the manual list, namely kubuntu-desktop. All the others will be marked as auto, as they are only dependencies. – Albert Dec 28 '21 at 12:55
  • @Albert Is Konsole a dependency for Kubuntu desktop? – Error404 Dec 28 '21 at 13:28
  • @someone: I don't really know. Why is that relevant? See here for direct dependencies but there are much more indirect dependencies. It is for example a dependency of the kde-full package. – Albert Dec 28 '21 at 17:05
  • @N0rbert Can you look at https://askubuntu.com/a/1384019/1460940. Is it true? Please see the comments also. Am I missing something? Aren't all packages installed during the installation of OS marked "non-auto" or "manual" by the installer, so that they don't get removed by sudo apt autoremove? – Error404 Dec 29 '21 at 03:21
  • During Ubuntu 18.04 LTS installation many packages are marked as manual. In newer versions only bunch are marked as manual. For me it seems that there some changes occur in the internals of APT for last 3 years. – N0rbert Dec 29 '21 at 08:16
  • You now changed the answer quite a bit, and reversed the meaning. Now from your description, it sounds like the system is messed up, but somehow you get to the conclusion it is not messed up? It also looks like the fresh installation of Ubuntu <=18 is messed up, as it should not list such a big list of manual installed packages. If almost all packages are listed as manual installed, that should not be, i.e. is messed up. But somehow you conclude just because that is how it is, that it is not messed up? – Albert Dec 30 '21 at 13:18
  • To add, I just found this, and there is indeed a bug report about this behavior, bug/424643. – Albert Dec 30 '21 at 13:20
  • I conducted comprehensive analysis to understand why it goes this way. I use such a messed system for years. There is no reason to worry. – N0rbert Dec 30 '21 at 13:22
  • Well, the whole point of this functionality is that autoremove works, also in case of further updates where some packages might not need some other dependencies anymore. So you say, I should not care that autoremove will not work, at least for those packages by the base installation? Also, I fear that it could break some distribution upgrade later. I already had a couple of problems due to packages which had problems in the upgrade which I did not really need at all. For example it wanted to have upstart together with systemd. – Albert Dec 30 '21 at 13:36
  • Then you have to describe exact "problems in the upgrade" first. Upstart and Systemd may stay in the system if there are packages depending on them. You can run aptitude why upstart to understand why this package is installed. Currently I do not understand exact problem in your system. – N0rbert Dec 30 '21 at 14:07
  • Upstart also stays in the system when it is wrongly flagged as manual, which was the case, which is the exact problem I'm describing here. Only because Ubuntu 8 or whatever flagged all packages as manually installed does not make it correct, or not messed up. – Albert Jan 04 '22 at 07:46
1

The output of the showmaual option includes any package that ever got updates, so not really what you want. The below command I copied from some other answer (don't recall where or I'd attribute it), and compares the output against the initial install list, producing the packages you installed yourself. Put it into a file in your bin area for convenience.

comm -23 <(apt-mark showmanual | sort -u) <(gzip -dc /var/log/installer/initial-status.gz | sed -n 's/^Package: //p' | sort -u)

The first time I ran apt-mark showmanual, I knew the list was too big because things like "tar" were included, which I knew I had never manually touched. "tar" however had had an update applied, which caused it to be output. Since "tar" was included in the initial system packages, it gets removed by the script, just leaving things that "showed up" after the initial system installation.

ubfan1
  • 17,838
  • "showmaual option includes any package that ever got updates" - I don't really understand what that means. As I understood it, it ends up in this list exactly iff I did apt install X, which is thus exactly what I want. – Albert Dec 30 '21 at 13:09
0

The system is not "messed up", but basically your understanding of what manually installed means is incorrect. A package is never actually marked manual or not-manual. It can only ever be marked auto or not-auto.

A package is marked auto when it is installed automatically to satisfy a dependency. This means that it can (and implies that it should) be removed automatically when no other package depends on it any longer.

On a normal desktop system the majority of packages are installed when the OS is first installed by the installer. These packages have to be installed not-auto, or else they would be removed because nothing depends on them.

Take for example 'abiword' near the top of your list. This is a word processor that is probably quite normal to have installed on a desktop system. I assume that you used one of many flavours of desktop installers. It probably didn't ask you specifically if you wanted abiword. You probably didn't install abiword yourself by name. You still chose to have it by choosing the install medium that you used.

Since there is no other package on your system that depends on you having abiword installed in order to work, it cannot be marked auto or else it would be removed. This means it is not marked auto. apt-mark showmanual lists the packages that are not marked auto.

Note also that many installers install everything non-auto even if they could mark them auto because something depends on them. Even the important core packages that hundreds of things depend on get marked non-auto. In this sense, maybe the installer is "messsed up", but it is nothing special about your system.

My best attempt at getting a list of things I might have installed myself is:

aptitude search -F '%p' --disable-columns '~i !~M !~E !~prequired !~pimportant !~pstandard !~tminimal !~tstandard'

To exclude packages that are depended on, use:

aptitude search -F '%p' --disable-columns '~i !~M !~R~i !~E !~prequired !~pimportant !~pstandard !~tminimal !~tstandard'
Tom V
  • 117
  • 1
    I don't exactly understand. Your description matches my understanding. Yes, abiword in that list is maybe sth I installed manually. However, definitely not abiword-common, yelp-xsl or zeitgeist or so. Those should all be marked as auto because they were probably installed to satisfy a dependency. – Albert Dec 27 '21 at 19:28
  • 1
    The point I am making is that if you run apt-mark showmanual on a brand new freshly installed system it will list many hundreds of packages because the system install media mark everything as non-auto so that it doesn't get removed. – Tom V Dec 27 '21 at 21:04
  • No, that is not true. On a fresh install, showmanual only lists about 30 packages, not more. Not more are needed. You just need ubuntu-desktop, linux, and a few others. All the remaining ones are then dependencies. – Albert Dec 27 '21 at 21:46
  • @Albert No, that is not true. On a fresh VM of Ubuntu 21.10, I can see almost 150 packages. Also, all the applications such as firefox and gedit are on the list. So, preinstalled applications are also manually installed. I agree with TomV. +1 – Error404 Dec 28 '21 at 05:20
  • 150 isn't 4000 though, I would have expected more. Maybe it depends on the installer version. The real question is: are any packages marked auto on this system? – Tom V Dec 28 '21 at 06:50
  • @TomV No... Off course not. The number of packages depend on minimal and full install (I was telling about minimal install). If the packages were marked as auto then a simple apt autoremove will delete the entire system! – Error404 Dec 28 '21 at 07:34
  • Surely on a fresh install, there are many packages marked as auto. Much more than those marked as manual. The number 30 of manual packages is from N0rberts comment above for a fresh Ubuntu MATE 22.04 LTS daily VM. I still don't exactly understand the discussion here. In my original post, I'm quite sure there are packages which I have never installed manually and which are also not marked as manual on any fresh installation. – Albert Dec 28 '21 at 12:53
  • The last time I ran a release installer (ubuntu 21.04 mate) every package flagged essential or important and every package with priority higher than optional was marked non-auto by the installer, even libraries that have no purpose unless you install something that depends on them. As well as this the mate gui packages were non-auto too. showmanual listed many many hundreds of packages. – Tom V Dec 28 '21 at 17:27
  • @TomV I just found this, and there is a bug report saying that this is wrong and buggy behavior, bug/424643. Also, see the answer above which indicates that this is not a problem in Ubuntu 20 and 22 anymore? – Albert Dec 30 '21 at 13:24
  • That question and bug report are over 10 years old (but I think you said your system is very old too). I've definitely seen this much more recently on Debian and Ubuntu. – Tom V Dec 30 '21 at 21:39
0

According to the man-page of apt-mark all dependencies of meta-packages are marked as manual. So you can run:

apt-mark minimize-manual

to mark all of these packages as auto. After that,

apt-mark showmanual

shows less packages.

GooDeeJAY
  • 176
hans-helmut
  • 1