Use your Ubuntu install disk/usb and open Terminal
then the efibootmgr
to delete the entries not needed.
https://www.linuxbabe.com/command-line/how-to-use-linux-efibootmgr-examples
You can install it with in a live session on usb if it is not there, no clue on optical media never have tried to install a program on them, never use them anymore.
For the grub mount the drive in the live session and in the EFI
directory there delete the ubuntu you see in it.
sudo mount /dev/sda1 /mnt
sudo rm -r /mnt/EFI/ubuntu
You may need to repair the windows boot loader too.
https://www.howtogeek.com/howto/33433/restore-the-windows-boot-loader-after-an-ubuntu-update/
Be careful when removing the ubuntu directory that you have the correct /dev/sd??
.