Sorry new to linux/ubuntu. I have read through a bunch of htop posts here, but I just wanted to clarify what the values mean. I run a 32-bit program, and my understanding is that roughly 4.2 is the max amount of memory this application can use. If I go over that, I will get a calloc error.
When looking at htop, am I looking at the virt or the res amount to see when it's getting close to that 4gb limit?
As stated in the htop man page:
M_RESIDENT (RES)
The resident set size (text + data + stack) of the process (i.e. the size of the process's used physical memory).
M_SIZE (VIRT)
The size of the virtual memory of the process.
However, RES just shows that - the physical memory, it doesn't mean in any way that the 32-bit binary process will be limited to 4 GB in total, and in fact if you have Physical Address Extension enabled the process could use more than that. Physically, yes, 232 addresses means 4GB of physical RAM. As per Gilles's answer (with my emphasis):
A 32-bit Linux kernel can only execute 32-bit processes. Depending on the kernel compilation options, each process can only allocate 1GB, 2GB or 3GB of memory (the rest is reserved for the kernel when it's processing system calls). This is an amount of virtual memory, unrelated to any breakdown between RAM, swap, and mmapped files.
Also according to an answer by Ramesh on Unix & Linux site, a process can have large quantities of memory allocated:
The most that the process can address is 4GB. You are potentially confusing memory with address space. A process can have more memory than address space. That is perfectly legal and quite common in video processing and other memory intensive applications. A process can be allocated dozens of GB of memory and swap it into and out of the address space at will. Only 2 GB can go into the user address space at a time.
If you have a four-car garage at your house, you can still own fifty cars. You just can't keep them all in your garage. You have to have auxiliary storage somewhere else to store at least 46 of them; which cars you keep in your garage and which ones you keep in the parking lot down the street is up to you.
Corroborated with Breakthrough's answer:
In terms of an address being "valid", each process has it's own unique address space (thus implementing a virtual memory scheme), so any address is technically valid. Remember, a process can allocate more memory than is physically available.
...
A virtual address for a given process is mapped to some physical storage hardware (RAM, disk, etc...), but the mapping is done at run-time by the operating system & MMU.
Of course, address space being 4GB implies pointers still being 32-bit.
Additionally Linux kernel has Out Of Memory Killer/Manager, which will clean up and free up memory. In other words, don't stress out about the 4GB limit, if you're worried about the process memory overall.
If by contrast we're talking about allocating 4 GB in your code, then according to a related post malloc() syscall will be in fact limited to 4 GB of allocations. But you can do more than one malloc() requesting more than 4GB in total (though I'm not entirely sure how that works out with the physical limit).
Other interesting readings: