Installing Ubuntu in a pen drive is a bad choice. However, you can always install Ubuntu on an external hard disk if you don't want Ubuntu in the internal hard disk.
Installing Ubuntu on external hard disk is much simple! Just plug in the hard disk. Run Ubuntu setup and choose the external disk while installing. Follow instructions during installation and you're good to go.
You need to change the boot order through BIOS, so that whenever you want to boot Ubuntu, you just insert the hard disk and the computer will automatically boot Ubuntu.
You can use this hard disk to run Ubuntu on any other computer as well, if it meets the minimum requirements needed to run Ubuntu.
Note :
- The only problem arrives in this method is the proprietary drivers that are needed for some hardware like Broadcom, Synaptics, ATI, Nvidia are different for different machines. You can install these drivers the first time you get the problem on your machine.
I hope this solves your problem.