203

I recently upgraded to the Kubuntu Natty Beta 1 and I've been having a lot of issues with the process kworker. At moments it uses almost half my CPU. Also, strangely enough it seems to affect my USB ports; whenever I plug in an USB drive, the process kworker goes into hyperdrive, leaving me unable to work.

I have thought about filing a bug but since I haven't even found any reasonable explanation on what kworker is I figured I should find out first.

user2428118
  • 383
davorao
  • 7,057
  • Strange, kworker is running and contributing to the wakeups by 10% but I have no Kubuntu programs installed. Alos Nepomuk isnt installed. – dago Jun 09 '11 at 21:16
  • 1
    from the response of afrazier i think now that it has something to do with the kernel (so the k in kworker is for kernel). so that is why you would also have kworker running on your Ubuntu machine. – davorao Jun 11 '11 at 07:12
  • 1
    This answer may also be helpful for finding out what a kworker is doing: http://unix.stackexchange.com/questions/22851/why-is-kworker-consuming-so-many-resources-on-linux-3-0-0-12-server/65270#65270 – anarcat Feb 19 '13 at 04:08
  • 1
    You shouldn't have taken that last OS update. If you're lucky, the next one will fix it. – Brent Bradburn Feb 20 '17 at 06:44
  • I have a Dell T40 and the following solution worked for me: https://askubuntu.com/questions/1044872/ubuntu-16-04-kworker-using-high-cpu-constantly – Alexis Wilke Dec 07 '20 at 01:35
  • Does this answer your question? Why does kworker cpu usage get so high? – vanadium Dec 17 '21 at 12:06
  • This question is older than the suggested duplicated question. That means this question is the original and the listed duplicated question is actually the duplicate. I vote to leave open. – C.S.Cameron Dec 17 '21 at 16:46

6 Answers6

154

"kworker" is a placeholder process for kernel worker threads, which perform most of the actual processing for the kernel, especially in cases where there are interrupts, timers, I/O, etc. These typically correspond to the vast majority of any allocated "system" time to running processes. It is not something that can be safely removed from the system in any way, and is completely unrelated to nepomuk or KDE (except in that these programs may make system calls, which may require the kernel to do something).

There were some reports of excessive kworker activity for relatively idle systems starting during 2.6.36 development (example discussion), and wide reports of confusion and problems with 2.6.38 (although many of these reports include the word "Natty", so I presume these people not to have used any kernel between 2.6.35 (distributed in Ubuntu 10.10) and 2.6.38 (distributed in Ubuntu 11.04).

I've found many reports of something that "fixed" this for one or another user. Most "fixes" seem to be related to updates of the kernel of various sorts. Where the update can be tracked to a specific issue, it seems to often be some driver or kernel service that has been patched to not misbehave: I have the impression that there are a very large number of things in the kernel that can cause a behaviour which is observed as excessive kworker usage.

If you find the system unusable due to excessive kworker activity, I would recommend trying to do fewer things. If you think you're not doing anything, try shutting down long-running services or timers (RSS readers, mail readers, file indexers, activity trackers, etc.). If this doesn't work, try restarting. If your system allows you to enable or disable hardware in a pre-boot environment, try turning off hardware you aren't using. If it happens on every restart before you do anything, you could try uninstalling things, but at this point you'll want to be running syscall profiling tools to track down specific applications that seem to be causing this overload.

It is to be hoped that your specific system will stop expressing this behaviour with a future kernel upgrade (and many of the most common causes of this have been solved).

tshepang
  • 1,967
Emmet Hikory
  • 4,075
  • 1
    I had a slow login problem that I think is related to this; I manage to workaround it by delaying all that is started this way: http://askubuntu.com/a/484856/46437 – Aquarius Power Apr 02 '15 at 20:24
  • 2
    Perhaps this is superfluous, but kudos on a very clearly articulated response. That's some finely expressed English, sir. – Jon Carter May 24 '16 at 23:49
  • 1
    "syscall profiling tools to track down specific applications that seem to be causing this overload" Like which ones? I can only find tools that tell me which part of the system was called, which is usually pretty cryptic, but I would like to know which program caused the system calls (and preferably even why). – Fabian Röling Jan 13 '20 at 13:50
  • Do kworkers only do work put into workqueues, or do they have other responsibilities as well? – Severyn Kozak Apr 18 '20 at 21:07
114

What is kworker? kworker means a Linux kernel process doing "work" (processing system calls). You can have several of them in your process list: kworker/0:1 is the one on your first CPU core, kworker/1:1 the one on your second etc..

Why does kworker hog your CPU? To find out why a kworker is wasting your CPU, you can create CPU backtraces: watch your processor load (with top or something) and in moments of high load through kworker, execute echo l > /proc/sysrq-trigger to create a backtrace. (On Ubuntu, this needs you to login with sudo -s). Do this several times, then watch the backtraces at the end of dmesg output. See what happens frequently in the CPU backtraces, it hopefully points you to the source of your problem.

Example: e1000e. In my case, I found a backtrace like this nearly every time:

Call Trace:
 delay_tsc+0x4a/0x80
 __const_udelay+0x2c/0x30
 e1000_acquire_swflag_ich8lan+0xa2/0x240 [e1000e]
 e1000e_read_phy_reg_igp+0x29/0x80 [e1000e]
 e1000e_phy_has_link_generic+0x85/0x120 [e1000e]
 e1000_check_for_copper_link_ich8lan+0x48/0x930 [e1000e]
 e1000e_has_link+0x55/0xd0 [e1000e]
 e1000_watchdog_task+0x5e/0x960 [e1000e]

It hinted me to a problem in the e1000e Ethernet card module, and indeed a sudo rmmod e1000e made the high CPU load go away immediately [e1000e bug #26].

tanius
  • 6,303
  • 1
  • 39
  • 49
  • 5
    echo l > /proc/sysrq-trigger seems to not work on proxmox saying sysrq: SysRq : This sysrq operation is disabled. sadly. – hak8or Jan 29 '18 at 00:51
  • 4
    sysrq need to be reenabled with sysctl -w kernel.sysrq=1 see https://askubuntu.com/questions/911522/how-can-i-enable-the-magic-sysrq-key-on-ubuntu-desktop – Sebastien Apr 06 '18 at 06:36
  • How did you understand e1000 was causing the problem ? Just because it was being kept repeated ? – Our Jul 15 '19 at 10:11
  • 1
    @onurcanbektas Yes – the most frequently repeated result is the most probable cause. Because the backtraces are spot checks what the CPU is busy with while it's hogged. – tanius Jul 16 '19 at 11:18
98

Why does kworker hog your CPU (cont.)? As an alternative to my other answer here, Perf is a more professional way to analyse what kernel tasks are hogging your CPU:

  1. Install perf:

    sudo apt-get install linux-tools-common linux-tools-3.11.0-15-generic

    (The second package must match your kernel version. You can first install just linux-tools-common and call perf to let it tell you which package it needs.)

  2. Record some 10 seconds of backtraces on all your CPUs:

    sudo perf record -g -a sleep 10

  3. Analyse your recording:

    sudo perf report

    (Navigate the call graph with ←, →, ↑, ↓ and Enter.)

Community
  • 1
tanius
  • 6,303
  • 1
  • 39
  • 49
  • 5
    On my system, running inside a VMware virtual machine, using perf I traced the issue down to the sd_mod kernel module. Disabling SCSI in the vmx file prevented the module from loading and got the system back to normal speed: scsi0.present = "FALSE" – feklee Dec 28 '15 at 00:17
  • E: Unable to locate package linux-tools-3.11.0-15-generic E: Couldn't find any package by glob 'linux-tools-3.11.0-15-generic' E: Couldn't find any package by regex 'linux-tools-3.11.0-15-generic' – Vadim Feb 27 '19 at 12:09
  • @Paddy: Please check the answer again :-) "The [linux-tools-*-generic] package must match your kernel version. You can first install just linux-tools-common and call perf to let it tell you which package it needs." – tanius Feb 27 '19 at 18:22
  • For RHEL and fedora the package name is perf. i.e. yum install perf. I used it like perf top thich shows which calls eat most cycles. Very nice. Would be nice to include perf top in your answer. Upvoted both your nice answers. – akostadinov Apr 11 '20 at 17:51
  • 2
    You can just install linux-tools-generic to get the up-to-date version. In my case, the reason was a faulty PCI to USB controller. – StSav012 Aug 27 '20 at 02:56
  • 1
    On Raspbian (Raspberry Pi OS) the package is called linux-perf-5.10. – Defozo Feb 25 '21 at 23:56
  • perf looks really clever, but looking at the report I still have no idea what I'm looking for, or what to do about it! – artfulrobot Jan 27 '23 at 08:48
12

Just to let everyone know. I ran into this issue, installed perf (which is a great tool), it pointed to spin locking and XFS. That pointed to NFS. Then I realized one of my mounts was out of space. Freeing up space caused kworker CPU to drop to 0.

So apparently this can be a symptom of running out of drive space on a busy NFS server!

Erik Aronesty
  • 359
  • 3
  • 6
  • 1
    I noticed I had a kworker meltdown when a specific virtualbox was running. Turns out that one was running an NFS with my host for sharing files. However it was nowhere near full. Unmounting the NFS "resolved" the issue. I guess I'll share with an sshfs mount. – Programster Apr 22 '14 at 17:57
4

I recently installed Ubuntu Natty on an external drive usb wd passport. When I start on my desktop which is about two years old, everything works like a charm. When I start on my new laptop (MSI gt680r system), it slows down after I wake up the computer from sleep, or if I plug another usb disk.

Kworker processes take more and more cpu, and the mouse freezes from time to time.

I have read several solutions on various forums that did not work.

I went into the bios of my laptop, where there was:

Hand XCHI OFF: Enabled
EHCI Hand OFF: disabled

I changed for:

Hand XCHI OFF: disabled
EHCI Hand OFF: disabled

and since that, it doesn't freeze anymore on natty on my laptop.

I would enable hand back if and when the problem is corrected.

Nikana Reklawyks
  • 115
CedCannes
  • 41
0

I think disabling Nepomuk can help you:

http://www.freetechie.com/blog/disable-nepomuk-desktop-search-on-kde-4-4-2-kubuntu-lucid-10-04/

Extender
  • 2,358