How to know whether enough memory is free to deploy a new application on a Linux machine?
(It's probably a bit late for the OP, but this is asked quite often, so I'll give it a shot)
free normally shows something like this:
total used free shared buffers cached
Mem: 8195284 8137708 57576 0 1232328 2651156
-/+ buffers/cache: 4254224 3941060
Swap: 18892216 759852 18132364
People tend to look at the Mem: line when trying to find out how much free memory they have. Unfortunately that line is quite misleading, because the Linux kernel tries to make optimal use of the available memory in (at least) these ways:
It will cache data from the I/O subsystem (e.g. the disk), so that it will be readily available if needed.
It will actively evict processes that have been inactive for some time to the swap space, in favour of caching data for active processes. This tends to favour throughput over responsiveness, so some people tune their kernel to change this behaviour.
The first point is the source of confusion regarding free, because the Mem: line includes the memory used for caching in the used memory amount. The kernel, though, will cache as much as possible for performance reasons. In fact, on any Linux system that has been up for some time, the free memory tends to be close to zero - unused memory is wasted memory.
The cache memory, though, can be freed by the kernel if needed by another process. While it will impact I/O performance to a degree, other processes can have more memory without using the swap space. Therefore, for most intents and purposes, that memory is free.
That's why free includes a second line, where the cache memory is considered free:
-/+ buffers/cache: 4254224 3941060
This second line is what people should be looking at when they want to know if they have enough free memory for a certain purpose.
In the example above, according to the Mem: line there are ~57 MB of free memory. If one reads the second line, though, there are in fact about 3.9 GB that can be used without forcing active processes to swap. As a sidenote, there are also about 760 MB of rarely-used data that have been swapped out, to make more space in the main memory for processes and caching.
At roughly the same time, the contents of /proc/meminfo:
MemTotal: 8195284 kB
MemFree: 57660 kB
Buffers: 1232352 kB
Cached: 2651156 kB
SwapCached: 119936 kB
.
.
.
MemTotal: the available physical memory detected by the kernel.
MemFree: the unused physical memory - the free memory shown in the Mem: line of free.
Buffers: relatively temporary storage of raw disk blocks.
Cached: in-memory cache for files read from the disk. It does not include SwapCached memory.
SwapCached: memory that was once swapped out, then swapped back in but is still in the swap space. If needed, its contents can be just discarded (very fast!), without having to swap them out (slower).
So, to have a semi-accurate estimate of the memory that is actually available
MemFree + Buffers + Cached + SwapCached
is a good starting point - and the one free shows in that second line.
Naturally, memory management and the related statistics and measurements are more complicated than this. The numbers shown by free are mere estimates at best, since there are a lot of other variables to take into account if you want to go deeper. For people who regularly perform memory usage optimization, this is almost a form of art.
EDIT:
A somewhat humorous link about this "issue":
http://www.linuxatemyram.com/
EDIT 2:
To confirm the comment about memory use analysis almost being a form of art:
Even free misses a major chunk of cached data on modern Linux systems. From /proc/meminfo on my system:
SReclaimable: 2253576 kB
That's about 2GB of memory that is used by the system slab allocator for caching directory entries and such and it is reclaimable (i.e. it can be cleared and used by processes if necessary). Yet free does not consider it cache memory and does not enter it in any of its calculations and therefore it shows up as used memory.
The slabtop utility, if available, allows the system administrator to find out what the slab cache is used for.
A way (for the root user only) to have free show the actual memory use of the system is the following:
# swapoff -a
# sync
# echo 3 > /proc/sys/vm/drop_caches
# free
total used free shared buffers cached
Mem: 8195284 3181468 5013816 0 8656 228832
-/+ buffers/cache: 2943980 5251304
Swap: 0 0 0
# swapon -a
The first command disables the swap space. It should not be issued if the available memory may not be enough to hold the data that have been swapped out - in that case one has to take into account the Swap: line of free in their memory usage calculations.
The second command pushes all buffered data to the disk. It allows more cache memory to be freed in the next step.
The third command is the most important of the set - it forces the kernel to discard as much cached data as possible (page cache, directory entries, inodes etc).
Then free finally shows what the running processes actually use in its -/+ buffers/cache: line. It is quite noticeable that even after dropping all cached data the kernel quickly starts caching again - in this case it has already reached almost 250MB of cached data within a few seconds.
The final command enables the swap space again - it is only necessary if the first command was used too.
It should be noted that these commands should be executed by the root user in order to have the necessary privileges.