0

I often do some scientific calculations with Python code. These are very CPU and at times RAM hungry. Usually my Xubuntu 16.04 GUI freezes for periods of time during such a task, which means I can't really use the machine during those calculations which is of course annoying. Mouse movement is stopped or extremely slowed down, as well as any other action. Is there a way to avoid this?

David Foerster
  • 36,264
  • 56
  • 94
  • 147
smcs
  • 1,032
  • 1
    Do you have disk activity during those times? (ie. being ram hungry as you describe, it's page faulting regularly) You may find altering the swappiness value may improve things a little (ie. the level of which it uses swap; normally 60(%); I'd suggest reducing to 10-30 but adjust for your hardware & requirements) If you're not familiar with swappiness; refer https://askubuntu.com/questions/103915/how-do-i-configure-swappiness – guiverc Jul 06 '18 at 12:48
  • That could be it. I ran a calculation and watched htop. All cores close to 100%, Mem close to 100% and Swp showed up to 20G/31.9G. So this means physical memory is 'overflowing' onto the hard drive, which could slow down the GUI? I have a conventional HD and an SSD--could I make swapping use the SSD to speed things up? – smcs Jul 06 '18 at 13:32
  • 1
    Of course your gui slows when the system is overloaded; esp. if the page it needs isn't there (page-fault) and it has to wait for it to be reloaded from swap. Yes you can have swap on ssd; pro's & con's (mostly pros in your case). I'd suggest though you should offload/move your python-processing to another [remote] machine you control from there, or if you don't have another box, try and limit yourself to terminal apps that use less resources whilst its running. The python job is your problem, not Xubuntu. – guiverc Jul 06 '18 at 22:43
  • 1
    @guiverc: It looks like your suspicion was confirmed which makes a base for an answer. Do you want the honours? – David Foerster Jul 07 '18 at 17:48
  • @guiverc So the way I see it, the only 'real' solution would be to get more physical RAM. I can expect an overall speedup of calculation with that as well, right? Probided I don't have to use any swapping then – smcs Jul 09 '18 at 09:34
  • 1
    My guess is yes (more ram would help). Eventually you hit a point of diminishing returns (the page lookups to see if page is in memory or swapped-out, taking as long than grabbing straight from disk is reached but that seldom is). Without knowing your python app I can't know, or using faster swap (ssd you mentioned; but using ssd that way usually shortens ssd life, but that may still be cheaper than ram [cost/benefit]), as may be using another language (cost of re-write versus time saved), or faster box - the best for you only you can decide. Swap use isn't bad, too much though is :) – guiverc Jul 09 '18 at 10:10
  • Thanks, I'll look at all options. I thought that maybe there's some way to force Ubuntu to keep some physical memory free for the GUI processes, but that hope seems to be based on a wrong understanding of how it works. What is the 'page' object you keep mentioning? – smcs Jul 09 '18 at 11:11

1 Answers1

2

Do you have disk activity during those times? (ie. being ram hungry as you describe, it's page faulting regularly and needing to go to swap)

You may find altering the swappiness value may improve things a little (ie. the level of which it uses swap; normally 60(%). I'd suggest reducing to 10-30 but adjust for your hardware & requirements).

If you're not familiar with swappiness; refer

How do I configure swappiness?

The gui slows when the system is overloaded; esp. if the page needed by a gui process isn't there (page-fault) and it has to wait for it to be reloaded from swap.

Yes you can have swap on ssd; pro's & con's (mostly pros in your case). I'd suggest a better choice maybe offloading/moving your python-processing to another [remote] machine you control from your device, or if you don't have another box, try and limit yourself to terminal apps that use less resources whilst your large python job(s) are running. The python job is your problem, not Xubuntu

guiverc
  • 30,396
  • 1
    You were right. I resolved the issue by using less CPU cores during computation, thus avoiding swapping. It sped up the whole process, even though the swap file was already located on my machine's SSD, as lsblk showed. – smcs Jul 18 '18 at 08:15