Since you are new here: Could you do us favor before your do? See if you are running the recommended videocard driver.There have been issues with running the latest one.
Ubuntu 11.04 should not be alot slower than 10.10. Booting is a bit slower, ok but not alot to make a difference. The desktop itself should not be slower.
If you do want to go back to 10.10: reinstall is indeed the best option. It is the best option to make sure nothing from 11.04 stays on your system what could possible disturn your 10.04 setup.
Besides the link Takkat posted also have a look at the other questions on the right hand side under 'related' with almost the same titles since Duplicate questions do get closed on askubuntu.