You can start with two USB flashes.
First USB-flash should be prepared with Unetbootin or something similar and contain installation media of Ubuntu (let's name it install-flash).
Second should be fast, empty, with >= 16Gb capacity. You can install Ubuntu on it (let's name it target-flash).
Then boot from install-flash, select target-flash as destination for /
(and /home
, but do not create swap area here), install GRUB on it.
After such installation you will have Ubuntu installed on this flash.
You can insert this target-flash into PC or laptop and temporary change its boot-order in BIOS or UEFI (this operation is different for various vendors - may be one of the Esc, F8, F10, F12 and so on).
Note: I have such flash and external SSD - they work as expected on various hardware.