What is the reason behind apt-get's W: Duplicate sources.list entry?

Question

This question is really about the why of the error message. It would seem that duplicate entries in configuration file such as sources.list would be fairly simple to correct automatically and can be scripted fairly easily, IMHO.

According to Anwar's answer on the canonical question about this error message:

If your sources.list have a line like this

deb http://archive.ubuntu.com/ubuntu precise universe 

Then it can't have another line like the below (which your files have)

deb http://archive.ubuntu.com/ubuntu precise main universe

This seems like a fairly easy thing to resolve - split line into tokens and merge the two.

And yet, apt-get explicitly tells the user You may want to run apt-get update to correct these problems, even though in multiple posts on Ask Ubuntu the error actually appears when people do try to run apt-get update.

So is there any particular reason why apt-get doesn't resolve these automatically ? Is there some form of design concept/reasoning where it actually is desirable for this to be fixed manually ?

Answer 1

And yet, apt-get explicitly tells the user You may want to run apt-get update to correct these problems, even though in multiple posts on Ask Ubuntu the error actually appears when people do try to run apt-get update.

But that's not the only place this error is seen. It so happens that most people do tend to run apt-get update after making changes to the sources, which is perfectly fine, but this can also happen with other apt operations:

$ sudo vim /etc/apt/sources.list
[sudo] password for muru: 
$ sudo apt install vim-gnome    
Reading package lists... Done
Building dependency tree       
Reading state information... Done
vim-gnome is already the newest version.
0 upgraded, 0 newly installed, 0 to remove and 328 not upgraded.
W: Duplicate sources.list entry http://jp.archive.ubuntu.com/ubuntu/ trusty/main amd64 Packages (/var/lib/apt/lists/jp.archive.ubuntu.com_ubuntu_dists_trusty_main_binary-amd64_Packages)
W: Duplicate sources.list entry http://jp.archive.ubuntu.com/ubuntu/ trusty/main i386 Packages (/var/lib/apt/lists/jp.archive.ubuntu.com_ubuntu_dists_trusty_main_binary-i386_Packages)
W: You may want to run apt-get update to correct these problems

And the problem here is the two warnings are somewhat independently generated.

W: You may want to run apt-get update to correct these problems is printed when any error happens while building the apt cache. It's a catch-all generic solution: the code that prints this doesn't know (or care) which error happened, but I'd say that it's reasonable to assume for most errors that could happen when building the cache, fetching a fresh package list might well be the solution. (Think of all the apt problems we have seen because someone ran apt-get install or apt-get upgrade without running apt-get update.)

But, the developers know that it's not a great suggestion. See apt-pkg/cachefile.cc:

/* This sux, remove it someday */
if (_error->PendingError() == true)
   _error->Warning(_("You may want to run apt-get update to correct these problems"));

(As always, patches should be welcome.)

So is there any particular reason why apt-get doesn't resolve these automatically?

Because, IMHO, this is always the result of PEBCAK. add-apt-repository is smart enough not to add duplicate entries. So, this error always happens when someone edits the sources manually. And since humans have an infinite potential for creating interesting errors, it's not safe to have it automatically corrected.