If you update a kernel, then all you are going to do is placing a couple of files in
/boot/{config, initrd, System.map, vmlinuz}-{version}
/usr/src/linux-headers-{version}
The kernel itself is only an executable. That gets started by the bootloader (configuration is in /boot/grub/grub.cfg ).
This in turn allows a user to have multiple kernel versions installed simultaneously. Not knowing how you installed the 3.4 kernel version, I don't think that installing these updates will do any harm to your computer (especially if you downloaded the .deb from kernel.org and installed it manually).
If you can find the files for a 3.2 kernel in /boot/ and the corresponding /usr/src/linux-headers you should be safe. Since you should be able to boot a 3.2 kernel in case your 3.4 kernel somehow doesn't work anymore.
I would however recommend using a PPA for unstable kernels, since it will take care of all the dependencies.
The suggestions for updating 3.2 kernel files might come from the fact that you weren't able to install the 3.4 kernel correctly and you're still running a 3.2 kernel. In this case I recommend using the PPA to install the 3.4 kernel.
This comment provides a good explanation on how to upgrade to unstable kernels
Nothing actually changed, no problems.
– st3fUn Jun 23 '12 at 20:28