Open Terminal: CTRL+ALT+T
Install these Guest Additions Packages:
sudo apt-get install virtualbox-guest-dkms virtualbox-guest-utils virtualbox-guest-x11
Make sure you put your sudo user password and packages are installed correctly.
After that installation is Done. Restart your VM
If the problem still stays...
Try:
Go to System settings -> Software and Updates -> Additional drivers
Use x86 virtualization solutions...
After that reboot your virtual box
Source: Screen Resolution Problem with Ubuntu 14.04 and VirtualBox
After doing this if you still have the same resolution:
Open Terminal CTRL+ALT+T (Hotkey) type in terminal:
xrandr -s 1024x768
It will set your screen resolution to 1024x768