Direct answer:
removing packages using terminal/cli/tty:
# apt-get remove SoftwareName
common example:
$ sudo apt-get remove firefox
The above would result in something like:
The following packages will be REMOVED:
firefox
0 upgraded, 0 newly installed, 1 to remove and 9 not upgraded.
After this operation, 110 MB disk space will be freed.
Do you want to continue? [Y/n]
Entering 'Y' would then remove firefox.
You can also use purge,
$ sudo apt-get remove --purge somePackage
$ sudo apt-get purge somePackage
the 2 commands above are equivalent. (see man apt-get).
'purge' removes the selected package and its configuration file(s)
Less certain/more general:
From your description of using the Software Center, it sounds like you are using Ubuntu directly from the live image; meaning it seems like Ubuntu is not installed to a drive. I don't have personal experience with using the Software Center, so I could be wrong about this point.
Re
I like to have only what I need on my computer and hence I always follow the practice of removing the programs I know I'm not going to use.
That's fine, but be careful about assumptions based on other operating systems. If you find you enjoy Linux you may consider reversing that, and installing only what you need -building your system from the ground up.