Prevent GRUB from adding EFI entries to other disks

Question

Some context to understand my situation:
I had this idea of a portable drive (in my case a SSD) containing a couple of operating systems that I could carry around and simply plug into any computer and start working from there. So I bought a SSD and installed kali linux without GRUB and ubuntu with GRUB. During the installation of ubuntu, for booting I selected the EFI partition on the SSD that I created before and proceded with the installation.

My problem:
Since I want use the ssd on other computers as well, I just simply unpluged it after I booted into ubuntu to check whether everything was ok. I was about to restart the computer with the intend to boot into Windows located on the computers drive. Instead, I was presented the grub command line. I managed to boot into Windows with a few commands. But after checking in with diskpart on the state of the EFI located on the computers drive, there was an entry called ubuntu in the EFI directory. I deleted all enrties that didnt belong to Windows; after that everything worked fine.

My question:
(I think that) Everytime I do update-grub, it again installs its files on the wrong drive. How to prevent that? I know that there are some config-command labeled GRUB_EFI_BOOT_PARTITION or GRUB_INSTALL_DEVICE. Are these the things I am looking for?

// EDIT
I marked the link above as the solution for this question, however, I myself came up with a more radical (and in my opinion a more simpler) one: I just unscrewed all drives within my pc and then reinstalled ubuntu and grub. Now it works like a charm!