Persistence is the space on your USB key that is used to store information, so that changes you make to the system are still there after a reboot.
Live system without persistence:
- Portability but no changes are saved from one boot to the next
Live system with persistence:
- You have the portability of live media with a home folder and possibly other temp storage
- The USB drive may wear out faster due to more disk writes
Full installation on USB device:
- Just like a normal Ubuntu installation but more portable
- The lifetime of the USB drive will be shorter due to more disk writes