A less radical fix
Most computers enable you to enter the firmware setup utility, or at least get a boot manager menu, by hitting a function key (or sometimes Esc, Del, or something else) in the first few seconds after you power on the machine. Unfortunately, there's no standardization in what key does what, so if you don't see a prompt, you may have to just try all the function keys, Esc, and Del until something works. The timing can be tricky, too. I generally start hitting the desired key 2-3 times a second as soon as I power the computer on.
If you get a boot menu, it should have options for launching your USB drive. Pick the option that includes "UEFI" in the description, if there are two entries.
If you get into the firmware setup utility, look for options to disable "fast start," enable USB ports, or something similar. ("Fast start" in the firmware causes it to do little or no initialization of USB devices, among other things, which speeds the boot process but prevents booting from external media.) You may also be able to adjust the boot order to boot from a USB device first, but that device may need to be plugged in for the option to be available.
A more radical fix
If you can't get a boot menu or the setup utility, you could try using a partitioning tool to delete all the partitions on the disk. With no partitions present, the firmware will either enter its own setup utility, which should enable you to adjust the boot order, or boot to whatever external medium has a valid boot loader.
Be sure to get all the partitions, though. The Windows boot loader is stored on the EFI System Partition (ESP), so if you leave it intact, the firmware will think everything's just fine and boot to the Windows boot loader, which will then fail, leaving you in a worse state than you're in now. A few partitioning tools, like OS X's Disk Utility, intentionally hide the ESP from view, so there is a risk of missing it with some tools. I'm pretty sure that the standard Windows tool does show the ESP. If you want to be 100% sure of getting it, you could use my GPT fdisk (gdisk
), which is available in Windows form.
You'll be unable to restore the system to a factory default settings if you do this. A compromise might be to delete the ESP but leave everything else intact. This should still enable you to boot an external disk and use it to back up the remaining partitions. (In fact, you could even back up the ESP by recovering it with TestDisk or the like, once you've booted your external medium.)
Of course, the big risk here is that if you try this approach and it doesn't work, you'll have no way to use the computer. If it's new, though, I'd return it to the store at that point because if the firmware doesn't enable you to boot from an external medium when there's no OS on the built-in disk, it's badly broken by design and deserves to be returned.