make (use of -j option)

Question

I'm trying to undersand the true meaning of -j option in make command. According to manual, with this option the user can "specifie the number of jobs to run simultaneously". What I actually want to understand is the link between the numeric value specified in this option and the number of core/threads of the pc. So, if, for example I use:

make -j 8

it means that my pc has at least 8 cores (or threads)?

And, if this link exists, what happens if the value specified in -j option is greater than the number of cores/thread of the pc? Will that option be simply ignored?

Thanks to anyone so kind to answer to my (stupid) doubts.

Pilot6 · Answer 1 · 2020-11-27T21:14:43.817

7

This option specifies how many simultaneous threads will be used.

If you use too many of them, the option won't be ignored, but the compilation may run slower.

If you set too few threads, not all CPU cores will be loaded.

There are some recommendations to use + 1, or even *2. I am using twice # of CPU cores.

You can test which way gives a better result.

edited Nov 27 '20 at 21:14

answered Nov 27 '20 at 16:29

Pilot6

90,100
91
213
324

heynnema · Answer 2 · 2024-03-03T14:36:40.543

6

The -j option tells make how many jobs (commands) to run in parallel. This is limited by how many physical CPUs and RAM your system has.

Many make jobs use as many CPUs as it finds in the system. On my computer, it has four CPU's, which is the equivalent to setting "-j 4". This will allow the make to complete in the fastest time.

However, if you use all, or too many CPUs, you lose some system control, as there are no free CPUs. So I've always been told to use the -j option with one LESS CPU than you really have, so if you have to control-c the make (for example), the remaining CPU can do it. In my case, I'd be using "-j 3".

Try using -j CPUs, or CPUs+1, or CPUs+2, and see what works best for you. To time each run, prefix your command with the time command, and it'll give you specific compile/run times.

edited Mar 03 '24 at 14:36

answered Nov 27 '20 at 18:55

heynnema

70,711

1

it seems you are not compiling a lot ;-) – Pilot6 Nov 27 '20 at 19:02
@Pilot6 Do me a favour... how many CPUs do you have? Try running time make -jX, replacing X with the number of physical CPUs you have, and then make clean the code and run time make -jXX replacing XX with your number. Let's see what the time differences are :-) – heynnema Nov 27 '20 at 19:48
1

I tried that already ;-) The best result for my 4 real cores to build a kernel is -j 8 ;-) There is no multithreading on my build machine. It is AMD Phenom. – Pilot6 Nov 27 '20 at 20:00
@Pilot6 Pick a smaller code base, and show me the results of -j4 and -j8 :-) – heynnema Nov 27 '20 at 20:03
I compiled rtw88 driver. 0m10,998s with -j4 and 0m10,512s with -j8 ;-) – Pilot6 Nov 27 '20 at 20:17
@Pilot6 That's a whopping .486 second faster (If I read your nomenclature correctly) :-) – heynnema Nov 27 '20 at 20:20
Let us continue this discussion in chat. – Pilot6 Nov 27 '20 at 20:25
I suggest adding about the time command for testing. This is the way to find the best value. – Pilot6 Nov 27 '20 at 21:15
@Pilot6 Added. Thanks! – heynnema Nov 27 '20 at 21:17

make (use of -j option)

2 Answers2