4

This was put on hold as unclear. Maybe I put in too much background explanation. Mea culpa. So:

THIS IS THE QUESTION, REWORDED YET ANOTHER WAY:


Can apt-check, i.e.,

/usr/lib/update-notifier/apt-check

be made to treat potential kernel upgrades, the same way it does non-kernel upgrades, detecting them and distinguishing between those that incorporate security fixes, and those that don't, so that those that do will be reflected in the second field of apt-check's output, without having the meta-package linux-generic installed?

And if apt-check can't do this, is there some other program that can?


user535733 seems to have understood and answered it explicitly in a comment:

". . . If you uninstall the metapackage, there is NO other automated method to check for kernel upgrades...unless you write it yourself."

If nobody disagrees, and user535733 doesn't say I've misunderstood him, and puts that in an answer, I'll give him the Brownie points.

user535733 and waltinator both suggest that I could write a program to do this myself. Waltinator may be up to rewriting the source code of apt-check but I'm not. I might could script a work-around, but I'd need to find some way to distinguish between kernel upgrades that have new security fixes and those that don't.

Thanks, gentlesapients.

= = = added material ends - original post follows = = =

apt-check (from update-notifier package), which is normally used to write the MOTD, but can be called directly, returns output of the form:

x;y

where x is the number of possible upgrades
and y is the number of possible SECURITY upgrades

When linux-generic is not installed, a possible kernel upgrade does NOT count toward the first number. If a kernel upgrade is needed to correct security shortcomings in older kernels, does it count toward the second number? In other words, can apt-check be depended on to tell you that you NEED to upgrade the kernel in the absence of the metapackage that depends on the latest kernel?

If this isn't clear, here is a concrete example:
Every time I upgrade to 4.4.0-77 it borks my Xenial systems, of which I have 2, both on the same machine. The only solution I've found that actually works is to restore an fsarchiver backup of the borked system, mount all my systems, run lilo (like update-grub for a different bootloader) to find the new/old kernel, reboot into the restored system and uninstall linux-generic so it won't automatically install 4.4.0-77. Right now I check what kernel linux-generic WOULD install by running:

apt-get install --simulate linux-generic.

Maybe when we get to 78 or 80 or even 4.4.1 I'll try linux-generic again.

So will apt-check tell me when one of the new kernel upgrades isn't just for shiny new gee-whiz features but is actually correcting a security flaw? Or does it depend on linux-generic to do this? And if the latter, is there some alternative to apt-check for this purpose?

  • I do not understand the question, but, since Linux is FOSS, you could start with apt-src install update-notifier and modify the source. – waltinator May 07 '17 at 23:39
  • 1
    Kernel upgrades are determined by apt ONLY by using the metapackage version number. If you uninstall the metapackage, there is NO other automated method to check for kernel upgrades...unless you write it yourself. – user535733 May 08 '17 at 02:03
  • @ 535733: So, you are saying essentially "No, apt-check can't see a potential kernel upgrade to classify it as having or not having security implications if linux-generic isn't installed. – Lew Rockwell Fan May 08 '17 at 15:34
  • As for rolling my own mechanism to do this, I'm half way there by using "apt-get install --simulate linux-generic" and greping the version number in a script, but I don't know any scriptable mechanism to do what apt-check does for non kernel packages - check if a potential upgrade has security advantages as opposed to merely new features. Release notes for kernel versions would tell, but I doubt they'll be easy to parse in a script and I'm not sure where to find them. Apt-check presumably uses some sort of flag. I don't know if kernels are so flagged or where to find the flag if they are. – Lew Rockwell Fan May 08 '17 at 15:52
  • @ David F: That's interesting. I may be all wet, but it isn't clear to me though, how "apt" is involved. Despite a similar name, unlike "apt-get", "apt-cache", apt-cdrom", "apt-config", and "apt-key" - "apt-check" is not part of the package "apt" but of the package "update-notifier-common" which doesn't depend on apt AFAIK. It's a poor name choice IMO. – Lew Rockwell Fan May 08 '17 at 22:54
  • 1
    @LewRockwellFan 1) Yes, you understood my prev comment correctly. 2) Security upgrades come from the -security repo. Other upgrades come from other repos. Apt-check does not look at the package, it looks merely at the source. – user535733 May 08 '17 at 23:35
  • 1
    I voted to leave your question open because it seemed clear enough to me. Now I'm voting to reopen it, but I'm no gent, so if you want to thank me you'll have to make another edit :P – Zanna May 09 '17 at 04:39
  • @user535733 Repeating a comment I made to David F's answer here, because I'm not sure an "@" will draw your attention to a post you haven't commented on: "The question was simply: Can apt-check be forced to extend its action to kernels, and if not, is there some program that will do the same thing with respect to kernels? user535733 promptly understood and clearly answered the question in the second comment. Paraphrasing: "No, and no. Maybe you can write one." If he'll make that an answer, I'll give him the kudos." – Lew Rockwell Fan May 09 '17 at 17:04
  • @Zanna Repeating this because I've just figured out the at-name trick (or I think I have): Thanks. I see "gent" as comprising "gentlemen" and "gentlewomen" - as opposed to something like "dung-booted peasant". Kinda like the PC use of "waiter" or "actor" as gender-neutral. "Lady" I use as the opposite of "lord". But come to think of it, even if you allow my pedantic usage, it really isn't sufficiently inclusive anyway. I'll fix it. Thanks. – Lew Rockwell Fan May 11 '17 at 16:57
  • 1
    I saw your message and when I @ David to tell him the Q was open I added a thank you for the edit, but later deleted my comment in pursuit of tidiness. I must admit though I identify as a dung-booted peasant... – Zanna May 11 '17 at 17:01
  • @Zanna Cool. Me, I'm from the deep South. We keep the dung on our bare feet. – Lew Rockwell Fan May 11 '17 at 22:14
  • Can't you just boot an older kernel in lilo? At lease when using grub you can choose on startup which kernel to boot, so it lets you boot even if there is a problem in the latest installed kernel update. – jarno May 12 '17 at 03:57
  • @jarno I can. But the failed installation of 77 breaks apt, whatever kernel I boot from. There are 2 ways I can recover from it to get back to where I was before I tried to upgrade the kernel, I haven't tracked down the why of it yet. But it works that way on both of my Xenial systems while my fairly similar Trusty systems function normally. Anyway 'tis another topic. – Lew Rockwell Fan May 12 '17 at 16:10
  • Oh, you meant that installation of 77 failed by "borking my Xenial system". I thought installation was fine, but you could not boot the kernel. It could be helpful, if you showed the output of the failed upgrade command. – jarno May 13 '17 at 08:05

3 Answers3

3

Your question is a bit convoluted but I think it boils down to the single question in the title, so that's what I'm going to address.

How Apt selects package to upgrade

No, the Advanced Package Tool, the way it is intended, cannot install new packages (e. g. linux-image-*-generic) automatically unless another package depends on it. It only upgrades already installed packages automatically and replaces their previous version in the process.

This is one reason why we use meta-packages like linux-image-generic: to make Apt aware of new packages to install without the need to replace older versions. We do this for kernel packages because it would be difficult to replace the currently running kernel and because people want to revert to an earlier, known-to-work kernel version more easily in case anything goes wrong.

Furthermore, Apt doesn't know or care about the semantics of version numbers. All it cares about is the order of version number strings and a list of available versions to select the most suitable one for installation based on a configurable rating system. The package (repository) managers are responsible for the incorporation and publication of upstream changes incl. security fixes in replacement packages with a suitable version string.

What that means for apt-check

Now that we covered Apt in general, I can address how that affects your question regarding update-notifier, the package that provides apt-check: like Apt it cannot be aware of new packages to install if those don't depend on already installed or scheduled to be installed packages. If you don't have linux-image-generic installed then Apt won't see new kernel packages and neither will update-notifier when it queries Apt for upgradeable packages.

What if I really want this feature?

Of course, as with most things in Linux, you're welcome to write a tool that searches for patterns among all available packages to install them (semi-)automatically. I at least don't know any on-board tools that can do this though this task seems generic enough that I wouldn't be surprised to see a script for this that some admin hacked together.

David Foerster
  • 36,264
  • 56
  • 94
  • 147
  • Thanks. That's a well written explanation of the logic of meta-packages and their roll in apt, and pretty much the way I've always understood it, although I would be curious to see you expand on "based on a configurable rating system". But, and maybe I'm missing something, I don't see any bearing on the question.

    "Your question is a bit convoluted . . ." My expository skills must be weaker than I thought. And I a disciple of The Good Doctor. /me sighs.

    – Lew Rockwell Fan May 09 '17 at 16:50
  • Thanks. That's a substantial edit. You've gone to some effort here. Now my comments above will look insane to somebody who finds this later via a search site, since they were about the version that's gone and now I can't edit them. That's OK as long as they don't commit me. – Lew Rockwell Fan May 11 '17 at 05:25
  • See if I understand you correctly:

    You are concurring that the answer is essentially:

    "No, and no. Maybe you can write one."

    or more fully:

    "No, apt-check can't classify potential kernel upgrades if linux-generic isn't installed, and no, I don't know of a program that can, but it sounds like it might be do-able."

    But you are amplifying this by saying that the REASON for the first "no" is that apt-check gets it's information from apt, and that apt won't know anything about potential kernel upgrades unless linux-generic is installed.

    Is that a sound summary?

    – Lew Rockwell Fan May 11 '17 at 05:30
  • @LewRockwellFan: Yes, this sounds like a reasonable summary of my outline above. You're welcome and in this case even encourages to delete your own obsolete comments to remove clutter. – David Foerster May 11 '17 at 17:48
  • Ah! Thanks. My FFx Stylish style "Global Black for AMOLED Red" hid that button. I blame transparency in icons. I'll do that. I upvoted your answer, but I didn't hit the checkmark for 2 reasons: 1- I feel obligated to user535733 to give him a chance to put in a formal answer since he essentially answered this promptly and clearly in the second comment. and 2-In the meantime I've been researching this elsewhere and I believe there is a factual error there. I've made notes and I'll post them as an "under construction" answer", just to have sufficient space, if I figure out how to do that. – Lew Rockwell Fan May 11 '17 at 20:12
2

In Xenial, unattended-upgrades installs only security fixes by default. And in Software Updater you may select only security updates to be installed. But as for your problem, the upcoming security updates will likely contain the changes of earlier non-security updates, that wreck your system, too, so you had better report a bug against linux to get the issue fixed in upcoming updates.

Anyway, in command line you could run

apt-cache policy linux-generic

to see, if the latest available update is a security update.

In Xenial, you could install the latest security update of linux-generic manually by

apt-get install linux-generic/xenial-security
jarno
  • 5,600
  • VERY Interesting. I didn't know this trick. It fails for me, but it may still be a step toward figuring out why "regular" linux-generic trying to install 77 breaks my system. Once I get that fixed I'll try this again. It definitely ignores 77 (which does NOT have security fixes) and tries to install 75 (which DOES) so at least on the first round, it is trying, albeit failing, to do exactly what I want a kernel upgrade to do, and it failed gracefully. If I get it to work, will it leave me with a persistent linux-generic that ignores kernel upgrades published outside the securtity pocket? – Lew Rockwell Fan May 12 '17 at 16:37
  • In root terminal, with context: https://pastebin.com/d0Naz6pE Note that I fully updated and apt-check said "0;0" with no problems before I ran this. Note also that dpkg disagrees with the last error message. The only held package I have is and older conky which isn't broken as far as I can see. This is exciting. Hope I can get it to work. – Lew Rockwell Fan May 12 '17 at 16:53
  • Gist of the above paste:

    apt-get install linux-generic/xenial-security

    . . . The following packages have unmet dependencies: linux-generic : Depends: linux-headers-generic (= 4.4.0.75.81) but 4.4.0.77.83 is to be installed E: Unable to correct problems, you have held broken packages.

    dpkg -l | grep ^..r

    – Lew Rockwell Fan May 12 '17 at 17:00
  • BTW, Synaptic agrees with dpkg - it sees no held broken packages. I suspect if I achieve understanding of WHY this is failing, it will point the way to fixing whatever is causing regular linux-generic to break apt. I further suspect that if I can get this to work, and it works the way I understand it to, it is a total solution, a one-line solution at that, to the original problem as stated. You've made my day, sir. – Lew Rockwell Fan May 12 '17 at 17:16
  • Meanwhile, I'll look into forcing the version on linux-headers-generic. Think that might get this to work? – Lew Rockwell Fan May 12 '17 at 17:32
  • I ran "apt-get install linux-generic/xenial-security linux-headers-generic=4.4.0.75.81". I THINK it ran with no problems. I failed to pipe it to tee, and I have the verbosity setting in lilo.conf too high, so after that command ran up to the Y/n with no error messages (that much I'm sure of) the rest of it went by a little too fast to be certain, and then lilo ran automatically & filled my terminal history with output, so I couldn't go back and be certain, but I THINK this ran without any errors. The portion of the output of lilo still in the terminal history looked good. Looks very promising. – Lew Rockwell Fan May 12 '17 at 19:32
  • That got 75 up and running fine. But afterwards dist-upgrade wanted to bring in 77, so I entered n to stop it. Tried uninstalling linux-generic to prevent that happening again but that uninstalled 75. So I repeated "apt-get install linux-generic/xenial-security linux-headers-generic=4.4.0.75.81" and everything was shiny again. Ran dist-upgrade again, this time entering "y". Confirmed my fears. 77 was back and broke apt again. I'll have to fix this system before I can go on. Maybe the answer is to never do dist-upgrade or to do it first with --simulate. Anyway, this is big progress. – Lew Rockwell Fan May 12 '17 at 20:06
  • After I fix this system, I'll check back here before I do anything, but my plan is too study the distinction between upgrade, dist-upgrade, and I think we now have a 3rd option (?), full-upgrade. And look to see how to uninstall linux-generic without removing the dependencies it pulled in. – Lew Rockwell Fan May 12 '17 at 20:11
  • Your grep command makes no sense. – jarno May 12 '17 at 20:53
  • "Your grep command makes no sense. – jarno" Do you mean "dpkg -l | grep ^..r"? I don't have a broken package to test it at the moment, as I just finished fixing the system linux-generic broke, but it is the accepted and twice upvoted answer here https://askubuntu.com/questions/772653/how-to-list-broken-packages-in-console Quoting from there: "You can list broken packages :

    dpkg -l | grep ^..r

    r state (on the third field) means: reinst-required (package broken, reinstallation required)" I'd have piped it through cut instead if I were writing it myself, but I assumed that was valid.

    – Lew Rockwell Fan May 12 '17 at 22:06
  • And it agreed with Synaptic, which I also saw as supporting evidence of validity. Also the poster has a 1647 rep. What is wrong with it? Need the -E option maybe? I wondered about that myself. – Lew Rockwell Fan May 12 '17 at 22:10
  • "the upcoming security updates will likely contain the changes of earlier non-security updates, that wreck your system, too, so you had better report a bug against linux to get the issue fixed in upcoming updates." Actually, this is already a known bug filed against 4.4.0-77. Doesn't seem to affect most people. – Lew Rockwell Fan May 13 '17 at 05:52
  • Oh, can you tell the bug number? – jarno May 13 '17 at 07:16
  • As for the question you mentioned, I gave my own answer (even though I have lower reputation score). – jarno May 13 '17 at 07:52
  • @LewRockwellFan (I don't know, if you get notification without this.) – jarno May 13 '17 at 18:13
  • "As for the question you mentioned, I gave my own answer" And I clicked the up arrow to rep you for it. That was indeed a helpful post. – Lew Rockwell Fan May 13 '17 at 19:41
  • @nameless admin who put in the notice about extended chat. Sorry., you are correct. Indeed, while all the responses to this question are interesting, the bulk of it is off topic with respect to the actual question. I didn't wish to seem churlish by ignoring a response or repeatedly pointing out the topic drift more than I have. As I mentioned several times, user535733 answered the question succinctly, promptly, and correctly in the 2nd comment. The why of it, my problems with 77, & alternative approaches to inferred goals, are properly different topics. I desist. Thank you all. – Lew Rockwell Fan May 13 '17 at 19:58
0

Thanks to you all and thanks to Andy at Canonical who gave some good pointers in another forum. I've poked around at this a bit, not just here, and I've learned a few things and come to suspect some others. Too much to put in a comment.

Strictly speaking, the question was answered correctly and promptly by user535733 in the second comment to my initial post and in fairness, if he'll put that in an answer, I'll be glad to accept it.

@user535733 - Note the above, please.

But the subject was expanded by waltinator in the first comment to include the possibility of creating a tool to do the job and by David into HOW apt-check works and WHY it won't do this without linux-generic installed.

The following is nowhere near complete, some of it is speculation (clearly labeled I hope), there might even be a few outright mistakes (imagine that) and I may come back and edit some more stuff in. Maybe somebody stumbling on this page will find some of this of interest. You may have to run the commands on your own system to see the output clearly in the correct format.


re apt-check & linux-generic:

Linux-generic seems to be a red herring. Mea culpa, I assumed it's lack was the problem. I now don't think apt-check can't do this AT ALL, even if linux-generic IS installed. It is a tricky question to test without installing 4.4.0-77. I'll test in the future, when I don't have to worry about a kernel that consistently breaks my xenial systems.

The problem is NOT that apt-check doesn't have access to the necessary data. You definitely do NOT need linux-generic installed to get metadata about potential kernel upgrades sufficient to show what's available and if it has security fixes.

I assume "apt-get update", a part of apt, gets the info (more on that below), but WHATEVER gets it:

  • it does NOT require linux-generic be installed to see potential kernel upgrades.

  • it does NOT require linux-generic be installed to determine which kernel version linux-generic currently depends on.

  • it does NOT require linux-generic to determine if a potential kernel upgrade has security fixes in it.

Furthermore, whatever gets this information to begin with, the information persists locally (presumably until you clear it). So even off line and without linux-generic installed, apt-check has access to the necessary information - it just doesn't use it, probably BY DESIGN. But it can be used in a script - I do not have linux-generic installed and here are 2 examples of a test that works on my system even offline:

$ apt-cache showpkg linux-image-4.4.0-75-generic | grep "$(lsb_release --codename --short)-security"

and the output:

4.4.0-75.96 (/var/lib/apt/lists/us.archive.ubuntu.com_ubuntu_dists_xenial-security_main_binary-amd64_Packages) (/var/lib/apt/lists/us.archive.ubuntu.com_ubuntu_dists_xenial-updates_main_binary-amd64_Packages)

File: /var/lib/apt/lists/us.archive.ubuntu.com_ubuntu_dists_xenial-security_main_binary-amd64_Packages

File: /var/lib/apt/lists/us.archive.ubuntu.com_ubuntu_dists_xenial-security_main_i18n_Translation-en

$

So, 4.4.0-75 HAS security fixes.

In contrast:

$ apt-cache showpkg linux-image-4.4.0-77-generic | grep "$(lsb_release --codename --short)-security"

and the output (just a return of the prompt):

$ 

4.4.0-77 does NOT.


re apt-check & apt:

Apt-check, part of update-notifier-common (not update-notifier as stated above), may depend on apt for its info about packages, as David indicates, probably specifically on "apt-get update", even if apt does not depend on linux-generic for info on potential kernel upgrades. It is what I would expect and I don't see any alternative, but strangely, unless the devs have marked the dependencies wrong, apt is NOT a dependency for apt-check:

$ apt-rdepends update-notifier-common | grep apt

and the output:

Reading package lists... Done
Building dependency tree       
Reading state information... Done
  Depends: python3-apt
python3-apt
  Depends: libapt-inst2.0 (>= 1.1~exp9)
  Depends: libapt-pkg5.0 (>= 1.1~exp9)
  Depends: python-apt-common
libapt-inst2.0
  Depends: libapt-pkg5.0 (>= 1.1~exp9)
libapt-pkg5.0
python-apt-common
$

So, in theory, you don't need apt to run apt-check - you could use it on a sytem managed with dpkg and dselect, although I fail to see how that could work. I certainly wouldn't give odds that it would until I actually tried it (and I may out of curiosity later) but that is the way the dependencies are marked. You'd think the devs would mark apt as a dependency for update-notifier-common if one of its commands REQUIRED apt to work, but it may be an oversight.

So, with respect to the original question, here is what I now suspect is true:

"Can apt-check . . . be made to treat potential kernel upgrades, the same way it does non-kernel upgrades, detecting them and distinguishing between those that incorporate security fixes, and those that don't . . . without having the meta-package linux-generic installed?"

No, and it probably can't WITH linux-generic either.

"And if apt-check can't do this, is there some other program that can?"

No, not out of the box, so to speak.

But, re creating something that can

As waltinator suggested apt-check could probably be re-written to do so if you have the knowledge and inclination. But, as David suggested, this function can probably be scripted with other tools and I can probably do that a lot easier than learning how to re-write apt-check.

As mentioned earlier you can get the name of the package containing the newest, shiniest kernel blessed by Canonical by applying some grep and cut magic to the output of:

apt-get install --simulate linux-generic

and you can classify it as having/not_having security fixes with something like

apt-cache showpkg linux-image-4.4.0-75-generic | grep "$(lsb_release --codename --short)-security"

as illustrated above.

Ideally, to script this, we need a complete version history of what linux-generic has depended on to avoid a false negative in a situation this:

You have version 100 installed.

Version 101 had security fixes but your computer wasn't turned on that week.

Version 102 does NOT have security fixes (relative to 101, but it does relative to 100 since it still has the ones that were made in 101).

It would be nice if I could curl a complete version history of linux-generic, and somehow find out which "pocket" of the repo the dependent kernels were published in. So far, I've only discovered how to do this for old versions that are still in the repo.

This command:

apt-cache madison linux-generic

is headed in the right direction but I think it only gets what IS in the repo, not what WAS in the repo, so I'm not sure the info it yields will be sufficient to prevent the false negative scenario detailed above. If in the example above, 101 was already gone from the repo, I don't think the info from "apt-cache madison linux-generic" would be sufficient.

And, oh yeah

One more thing I should point lest somebody get interested in this and spend a lot of time on it without knowing it:

I'm still poking at this because I got started and I'm stubborn. But while, by pure happenstance as far as I can tell, the kernel that has given me problems does NOT have security fixes, and therefore if I had a working script for this, it never would have broke my xenial systems, Andy tells me that 4.4.0-77 is a rare case and that the great majority of kernel upgrades that linux-generic depends on DO have security fixes. I suspect he was being figurative/hyperbolic when he said "99%", but even allowing for that, I suspect an apt-check like mechanism that applies to potential kernel upgrades isn't going to prevent an un-needed (security-wise) upgrade very often, and so will prevent un-needed kernel upgrades breaking apt or other crucial system components, even more rarely. It appears to be just random happenstance that it would have done so for me. BTW, 4.4.0-78 works fine for me. Andy was right - whatever it was about the 77 is fixed.