Here's a method that in theory should work.
If you visit the "Accessibility" tab in System>Preferences>Keyboard then you can enable the "Slow Keys" "Only accept long keypresses" option. Turn the delay all the way down and you basically null the effects of this option. Then go to "Audio Feedback..." and enable "Beep when a key is pressed" under the "Slow Keys" section.

This method SHOULD work. I couldn't get it to work, honestly. I do have the sound-theme-freedesktop sound theme installed and enabled. Perhaps it's a bug with gnome 2.32.0.
These configuration options simply play the system bell. On Ubuntu the system bell does not work when desktop effects are enabled. Turn off desktop effects, or on Ubuntu 11.04 login as "Ubuntu Classic (No effects)", and your system bell should work again.