“Invalid MIT-MAGIC-COOKIE-1 key” message - MPI on Ubuntu 20.04

Question

Running a program that uses MPI gives me an "Invalid MIT-MAGIC-COOKIE-1 key" warning on Ubuntu 20.04. I only recently upgraded; I was having no such issues when I was using 19.10.

I want to know what's going wrong.

Here are some system results:

$ xauth list
Yantra5/unix:  MIT-MAGIC-COOKIE-1  d27adab3ff430390b17c59fb0f6e7e28
#ffff#59616e74726135#:  MIT-MAGIC-COOKIE-1  d27adab3ff430390b17c59fb0f6e7e28
$ hostnamectl status
   Static hostname: Yantra5
         Icon name: computer-laptop
           Chassis: laptop
        Machine ID: 42b9dd9e9200409c81515ce51e5f9d52
           Boot ID: 42be2b4557bf4d29934673457c4560b2
  Operating System: Ubuntu 20.04.1 LTS
            Kernel: Linux 5.4.0-42-generic
      Architecture: x86-64

I'm running openmpi on Ubuntu 20.04. All programs with MPI_Init() is giving me the same warning.

This may be a related question.

I have had the same problem. I have left a message on the launchpad forums, but it hasn't been responded to for weeks. — Adrian Barbuio, Oct 07 '20 at 02:57

score 1 · Answer 1 · answered Sep 28 '20 at 22:23

Did the problem persist? I had the same as you or the one you mention and fixed it ("it fixed itself" would be more accurate).

I had installed a NVIDIA driver to use OpenCL and CUDA before installing MPI on my local computer and had the same warning. I switched back to X.Org driver (my way of doing it: Software & Updates -> Settings -> Additional Drivers -> Using X.Org), rebooted and warning messages had disappeared.

I tried to switch again to my NVIDIA driver, rebooted and the warning didn't reappear, so eventually just changing driver may solve the problem. It's not elegant and doesn't explain much, but it is worth trying. Also, as I fail to reproduce the problem, it is possible that simply "turn it off and on again" was what did the magic, which is why I asked if your problem persisted.

This was done out of total despair after an hour or more of looking for an answer to similar problems. xhost +local:, xhost +, unset DISPLAY or export DISPLAY=whatever:X.X didn't change anything, nor did it work to specify DISPLAY in the mpirun options (as stated in the official documentation). I had also rebooted (before changing the driver), and grep xauth ~/.bash* didn't give any result (so my .Xauthority file had not been modified from the terminal).

Another way I found to avoid the problem while I had it was to disable the graphic interface and go with only the command line (ctrl+alt+F3), but I believe the warning is a minor inconvenience compared to no graphic interface.

Clues for people who are far more knowledgeable than little me about mpi and Xserver and somehow want to tackle this:

The question here may be related, and apparently compiling with mpich solves the problem, while when I compiled a small c script with mpicc I had the warning.

Clues for people who would have this, and for whom changing driver + rebooting didn't work and are as clueless as I was about xauth and .Xauthority:

The path to your .Xauthority file referenced in so many similar problems (like here and there.) can be found with echo $XAUTHORITY.
As stated in this problem, you interact with it through xauth. I was about to try to manually change MIT-MAGIC-COOKIE-1 as a last resort, although I believe at this point it would probably be easier to install older versions of mpi and switch between them with update-alternatives

This worked for me too, after I even compiled openmpi myself to see if that would fix the issue (it didn't). — Miguel, Aug 05 '21 at 10:58
From 2023 here, changing to X.Org instead of Nivida driver resolves the issue. Using the Nividia driver always fires this warning. — ahmathelte, Jan 16 '23 at 15:23

score 1 · Answer 2 · answered Mar 22 '23 at 11:14

The issue is caused by the libhwloc library, which is used by OpenRTE (the OpenMPI component) to explore the hardware topology of the node during the launch of parallel processes. When hwloc_topology_load() is called, libhwloc iterates over all of its available components, which are implemented as plug-ins. One of these components, gl, is responsible for gathering information about the system's graphics devices and uses X11 calls like XOpenDisplay(), which can generate the error message "Invalid MIT-MAGIC-COOKIE-1 key" if it fails to connect to the X11 server.

In many cases, this gl component is not needed on HPC clusters and can be safely disabled by setting the HWLOC_COMPONENTS environment variable to exclude it:

export HWLOC_COMPONENTS="-gl"

for me not. check here: https://unix.stackexchange.com/questions/659362/how-to-generate-correctly-a-new-xauthority-file-for-a-display-missing-it-witho. Newest installations of debian make as default easy as hostname for whatever reasons. this created a lot problems for me. check sysctl.conf, set kernel.hostname=NAME, and populate changes in bashrc adding sysctl -p --system. Also, cat .Xauthority to check what is the hostname given there. — Estatistics, Dec 31 '23 at 14:40

“Invalid MIT-MAGIC-COOKIE-1 key” message - MPI on Ubuntu 20.04

2 Answers2