Does Ubuntu do this out of the box?
Ubuntu will automatically turn on Physical Address Extension (PAE) if it is available; otherwise it will not even install.
Wikipedia, important parts (see the link for more):
In computing, Physical Address Extension (PAE) is a memory management feature for the IA-32 architecture, first introduced in the Pentium Pro. It defines a page table hierarchy of three levels, with table entries of 64 bits each instead of 32, allowing these CPUs to access a physical address space larger than 4 gigabytes (232 bytes).
Linux
The Linux kernel includes full PAE mode support starting with version 2.3.23,enabling access of up to 64 GB of memory on 32-bit machines. A PAE-enabled Linux kernel requires that the CPU also support PAE. The Linux kernel supports PAE as a build option and major distributions provide a PAE kernel either as the default or as an option.
Linux distributions now commonly use a PAE-enabled kernel as the default, a trend that began in 2009. As of 2012 many, including Red Hat Enterprise Linux / CentOS, Ubuntu have stopped distributing non-PAE kernels, thus making PAE hardware mandatory.