Linux How To Free Wasted Memory RAM in Buffers
We all know Linux is known for good memory management but is it really? It seems all on its own with hardly anything running that you can come back in days or weeks and find that almost all of your RAM is used!
And many will say "no don't worry it's buffers for optimization" but it doesn't seem to help because what is in buffers is not available to use for new programs running or ones that allocate more RAM as far as I can tell.
The reason I know this is because you can see that SWAP will get used so if there was enough RAM not eaten by buffers it should be freed for new or growing memory allocations but it is not.
By freeing your RAM in Linux you will often realize performance benefits and lower loads in many server based scenarios. A lot of admins will think they need more RAM when they don't. In fact the cache/buffers in Linux will eat up all of your RAM no matter how much you have. The kernel documentation warns you could lower your performance but this to me only makes sense for a basic user who doesn't generate a lot of IO, but for servers and heavy reads you would be better to never have the cache.
The key is Linux's vm/drop_caches which I totally disagree with (but I am not aware of anyway to actually disable the feature). Basically it starts caching and eating up your RAM. A lot of people say "it is fine don't worry it is normal". But it is not, after some days, or weeks depending on your usage you could be swapping whether you have 32GB, 64GB or even hundreds of GB of RAM because of the page caching/buffers resulting from a lot of IO.
To free pagecache:
echo 1 > /proc/sys/vm/drop_caches
To free reclaimable slab objects (includes dentries and inodes):
echo 2 > /proc/sys/vm/drop_caches
To free slab objects and pagecache:
echo 3 > /proc/sys/vm/drop_caches
So how do we fix this or clear out Linux's memory buffers to free RAM?
I would recommend running the command as a cronjob every minute or at least every hour depending on your usage.
[root@realtechtalk.com ~]# free -m
total used free shared buffers cached
Mem: 3851 3738 113 0 3269 222
-/+ buffers/cache: 246 3605
Swap: 2999 0 2999
The least invasive is to free page cache by echoing 1:
I find that on a Desktop machine that it will actually kill processes you may be using if you use 3.
echo 1 > /proc/sys/vm/drop_caches The most invasive is to free page cache and slab:
In plain English I have found this to kill programs I have been using.
echo 3 > /proc/sys/vm/drop_caches
[root@realtechtalk.com ~]#
[root@realtechtalk.com ~]#
[root@realtechtalk.com ~]# free -m
total used free shared buffers cached
Mem: 3851 146 3705 0 0 9
-/+ buffers/cache: 137 3714
Swap: 2999 0 2999
Now we can see that we have our RAM back!
From the kernel documentation:
drop_caches
Writing to this will cause the kernel to drop clean caches, as well as
reclaimable slab objects like dentries and inodes. Once dropped, their
memory becomes free.
To free pagecache:
echo 1 > /proc/sys/vm/drop_caches
To free reclaimable slab objects (includes dentries and inodes):
echo 2 > /proc/sys/vm/drop_caches
To free slab objects and pagecache:
echo 3 > /proc/sys/vm/drop_caches
This is a non-destructive operation and will not free any dirty objects.
To increase the number of objects freed by this operation, the user may run
`sync' prior to writing to /proc/sys/vm/drop_caches. This will minimize the
number of dirty objects on the system and create more candidates to be
dropped.
This file is not a means to control the growth of the various kernel caches
(inodes, dentries, pagecache, etc...) These objects are automatically
reclaimed by the kernel when memory is needed elsewhere on the system.
Use of this file can cause performance problems. Since it discards cached
objects, it may cost a significant amount of I/O and CPU to recreate the
dropped objects, especially if they were under heavy use. Because of this,
use outside of a testing or debugging environment is not recommended.
You may see informational messages in your kernel log when this file is
used:
cat (1234): drop_caches: 3
These are informational only. They do not mean that anything is wrong
with your system. To disable them, echo 4 (bit 2) into drop_caches.
An illustration of what can go wrong without clearing your RAM!
The example below was a fresh system, install and boot but a number of VMs were created from 20GB templates. The result is that a server with 128GB RAM was using swap space! But the drop_caches solution cleared it up in no time.
[root@abc ~]# free -m
total used free shared buffers cached
Mem: 128785 128469 315 0 194 114645
-/+ buffers/cache: 13629 115155
Swap: 28614 11 28603
Tasks: 322 total, 3 running, 319 sleeping, 0 stopped, 0 zombie
Cpu(s): 13.7%us, 12.9%sy, 0.0%ni, 49.4%id, 23.5%wa, 0.1%hi, 0.3%si, 0.0%st
Mem: 131875952k total, 38573652k used, 93302300k free, 992k buffers
Swap: 29301244k total, 11056k used, 29290188k free, 26495228k cached
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
10990 root 20 0 6908m 2.1g 5616 S 13.3 1.6 38:54.92 qemu-kvm
11010 root 20 0 6907m 2.1g 5608 S 15.3 1.6 38:05.71 qemu-kvm
12843 root 20 0 6781m 2.0g 5616 S 73.3 1.6 30:31.32 qemu-kvm
11985 root 20 0 6280m 2.0g 5612 S 65.3 1.6 31:23.33 qemu-kvm
16231 root 20 0 4875m 1.1g 5612 R 23.2 0.9 11:48.82 qemu-kvm
7241 root 20 0 4059m 919m 5608 S 7.6 0.7 21:26.74 qemu-kvm
7902 root 20 0 6134m 346m 5612 S 0.0 0.3 19:59.35 qemu-kvm
13707 root 20 0 6070m 343m 5612 S 7.3 0.3 2:47.46 qemu-kvm
11503 root 20 0 99.9m 4188 3120 S 0.0 0.0 0:00.23 sshd
10301 root 20 0 99.9m 4184 3120 S 0.0 0.0 0:00.48 sshd
7267 root 20 0 99.7m 4144 3120 S 0.0 0.0 0:01.22 sshd
1746 haldaemo 20 0 38268 3104 2408 S 0.0 0.0 0:00.32 hald
Tags:
linux, ram, bufferswe, quot, buffers, optimization, doesn, programs, allocate, swap, freed, allocations, realtechtalk, cached, mem, cache, echo, proc, sys, vm, drop_caches,