I'm sure you know this but sometimes hearing it from someone else helps you realise a mistake — modern BIOSes usually allow you to select a device when you boot. Try to access this feature. If this fails, open your BIOS configuration and make sure the boot order will boot USB before the hard disk. It may help to enable "legacy" boot (which may be named differently in different BIOSes...)
If you have done all these things it is possible that your USB drive was incorrectly imaged, or perhaps it is one of the cheap ones that don't support non-standard partition schemes.
See if another computer will boot off the USB drive, if not then re-image or use a different USB drive.
You can also access the GRUB command line and search for the commands to enter to instruct GRUB to boot from the USB drive. Thankfully GRUB2 supports tab-completion.