I think you had made the USB bootable using Rufus or LiveUSB Creator. If so, your USB is bootable.
Your USB is not booting when you are restarting the computer because the USB drive will be below your hard drive in the boot order. So your, hard drive actually gets booted first and you are not able to boot from USB.
So when your computer is getting on, actually on the first boot screen, press (F8, F10, F12,...). One of these keys will land you in the boot menu page and select your USB drive using arrow keys and then press enter.
You could change the boot order permanently by pressing the key required for landing you to the BIOS options. It might be F2.
For getting the key regarding boot menu, you could search your hardware (Motherboard) page on the web.