3

I have bulky Fortran source code (WRF) to be compiled.

Every time I compile the code, the system gets stuck. To diagnose the problem, I saw memory performance in the system monitor of Ubuntu, and compilation eats the whole RAM. Then it also starts eating swap memory. My laptop has 4 GB RAM. I extended it to 8 GB RAM. I created a swap partition of 12 GB. But once my swap partition utilization reaches 4 GB. system hangs. Why is my whole swap partition is not being used (see below)?

Peter Mortensen
  • 884
Anand Kakarla
  • 33
  • What is the longest time you have let it run? SWAP is a file or partition on your hard drive, and it is used to move data out of RAM, and store it till needed again. So it is very much slower than RAM. I am not a Fortran programmer, and have no clue about its compiler, but from what you are saying it sounds like you may be being impatient with it. I know in the old days these kinds of things could take days to compile depending on how big the code was. Just give it an over-night and see if it finishes . – Christopher Angulo-Bertram Jun 01 '16 at 15:41
  • Maybe is the old PAE - nonPAE problem. PAE (Physical Adress Extension) lets you add more RAM on old unsupported systems, but with some limitations. Maybe the additional 4 gb of RAM is not identically to the old one and it's creating a physical write-delay because of different latency/clock speeds. You need to check that in your BIOS and set the same latency/clock speeds (is possible) for both RAMs. Sorry for the limited answer, i'm not very technical. – ipse lute Jun 01 '16 at 16:03
  • Can you log the compiler message to a file that you can hopefully inspect and upload here later? Which compiler and compiler version are your using and on which architecture does the compiler run? It may simply be unequipped to handle 1) large memory objects (≥ 2^32 bytes) or 2) large or weird programs. Are you reasonably sure that the source is correct and doesn't trip a compiler bug? Can other compilers or systems compile it? – David Foerster Jun 01 '16 at 16:24
  • thank you people . i dont know what happened. I tweeked some of the flags. and now everything is working fine. may be some flags were set wrong and the compiler was searching for some directory it was not able to get. I was stuck on this problem for a week and everything is set. Now even during compilation only 50 % of ram is being used. – Anand Kakarla Jun 01 '16 at 17:26

2 Answers2

4

Why would you want swap usage to be higher?

Reading/writing to the swap is 1000 times slower than reading/writing to the RAM.

The system may not be hanging, it's just that it's so busy swapping stuff back and forth, it appears as though it hung.

@phihag said in the comments:

Most likely, when swap reaches 4GB this happens to be around the time that some part of the user interface gets swapped out. For instance, that could be the code to handle Ctrl+C, the keyboard layout handler, the code that displays output in the terminal, some data buffer of the terminal, some data buffer of the X server which paints pictures, etc. . Chances are if you wait a couple of hours either your swap is filled up or all of the data and code necessary for the Ctrl+C to be handled at the various layers has finally resulted in killing your Fortran process

I would suggest getting some more RAM. It will drastically speed up your compile. It's quite inexpensive nowadays, ~$60 for 16GB.

You'reAGitForNotUsingGit
  • 14,809
  • I an quite new to Linux too, but I thought that when whole RAM is used the system begins to use SWAP. And if it needs more memory it will finally use whole SWAP. Or does it have some limit on SWAP usage for running processes? – NonStandardModel Jun 01 '16 at 15:39
  • 1
    @NonStandardModel - AFAIK, there is no limit. But as I said in my answer, I don't think it's actually hanging. – You'reAGitForNotUsingGit Jun 01 '16 at 15:51
  • 1
    Running vmstat 1 and observing the si (swaps in) and so (swaps out) columns will show the swapping activity – Colin Ian King Jun 01 '16 at 16:29
  • @Android Dev I will take your suggestion seriously. But now even though i have my problem solved, i have one query .i was sure my system was hung . Ctrl-C was not working. terminal or any other process was not responding. only hard rebooting the system could make my laptop work. i sent you the snapshot by setting the capture delay of 15 seconds because i was not really able to control anything once it reaches close to 4 gb of swap partition. – Anand Kakarla Jun 01 '16 at 17:35
  • @AnandKakarla Most likely, when swap reaches 4GB this happens to be around the time that some part of the user interface gets swapped out. For instance, that could be the code to handle Ctrl+C, the keyboard layout handler, the code that displays output in the terminal, some data buffer of the terminal, some data buffer of the X server which paints pictures, etc. . Chances are if you wait a couple of hours either your swap is filled up or all of the data and code necessary for the Ctrl+C to be handled at the various layers has finally resulted in killing your Fortran process. – phihag Jun 01 '16 at 21:45
  • @phihag - Would you mind if I quoted part of that in my answer? – You'reAGitForNotUsingGit Jun 01 '16 at 22:07
  • Of course not, feel free to add that to the answer itself. – phihag Jun 02 '16 at 05:36
2

Using too much swap is bad for your system. More swap means more lag. Android Dev has already pointed out why you don't really want to use more swap. However there is a way to configure how often your system will use the swap partition. It is controlled by a setting called swappiness. You will find a detailed guide about modifying swappiness in this link.

I do not recommend you to increase swappiness. You can't even utilize a 12GB swap partition. The general rule is that swap should be half of your main memory. For you that is 2GB.

Community
  • 1
Amir
  • 73
  • 6