Linux How To Free Wasted Memory RAM in Buffers

We all know Linux is known for good memory management but is it really?  It seems all on its own with hardly anything running that you can come back in days or weeks and find that almost all of your RAM is used!

And many will say "no don't worry it's buffers for optimization" but it doesn't seem to help because what is in buffers is not available to use for new programs running or ones that allocate more RAM as far as I can tell.

The reason I know this is because you can see that SWAP will get used so if there was enough RAM not eaten by buffers it should be freed for new or growing memory allocations but it is not.

By freeing your RAM in Linux you will often realize performance benefits and lower loads in many server based scenarios.  A lot of admins will think they need more RAM when they don't.  In fact the cache/buffers in Linux will eat up all of your RAM no matter how much you have.   The kernel documentation warns you could lower your performance but this to me only makes sense for a basic user who doesn't generate a lot of IO, but for servers and heavy reads you would be better to never have the cache.

The key is Linux's vm/drop_caches which I totally disagree with (but I am not aware of anyway to actually disable the feature).  Basically it starts caching and eating up your RAM.  A lot of people say "it is fine don't worry it is normal".  But it is not, after some days, or weeks depending on your usage you could be swapping whether you have 32GB, 64GB or even hundreds of GB of RAM because of the page caching/buffers resulting from a lot of IO.


To free pagecache:
    echo 1 > /proc/sys/vm/drop_caches
To free reclaimable slab objects (includes dentries and inodes):
    echo 2 > /proc/sys/vm/drop_caches
To free slab objects and pagecache:
    echo 3 > /proc/sys/vm/drop_caches

So how do we fix this or clear out Linux's memory buffers to free RAM?

I would recommend running the command as a cronjob every minute or at least every hour depending on your usage.

[root@realtechtalk.com ~]# free -m
             total       used       free     shared    buffers     cached
Mem:          3851       3738        113          0       3269        222
-/+ buffers/cache:        246       3605
Swap:         2999          0       2999

The least invasive is to free page cache by echoing 1:

I find that on a Desktop machine that it will actually kill processes you may be using if you use 3.
echo 1 > /proc/sys/vm/drop_caches

The most invasive is to free page cache and slab:
In plain English I have found this to kill programs I have been using.

echo 3 > /proc/sys/vm/drop_caches

[root@realtechtalk.com ~]#
[root@realtechtalk.com ~]#
[root@realtechtalk.com ~]# free -m
             total       used       free     shared    buffers     cached
Mem:          3851        146       3705          0          0          9
-/+ buffers/cache:        137       3714
Swap:         2999          0       2999

Now we can see that we have our RAM back!

 

From the kernel documentation:

 

drop_caches

Writing to this will cause the kernel to drop clean caches, as well as
reclaimable slab objects like dentries and inodes.  Once dropped, their
memory becomes free.

To free pagecache:
    echo 1 > /proc/sys/vm/drop_caches
To free reclaimable slab objects (includes dentries and inodes):
    echo 2 > /proc/sys/vm/drop_caches
To free slab objects and pagecache:
    echo 3 > /proc/sys/vm/drop_caches

This is a non-destructive operation and will not free any dirty objects.
To increase the number of objects freed by this operation, the user may run
`sync' prior to writing to /proc/sys/vm/drop_caches.  This will minimize the
number of dirty objects on the system and create more candidates to be
dropped.

This file is not a means to control the growth of the various kernel caches
(inodes, dentries, pagecache, etc...)  These objects are automatically
reclaimed by the kernel when memory is needed elsewhere on the system.

Use of this file can cause performance problems.  Since it discards cached
objects, it may cost a significant amount of I/O and CPU to recreate the
dropped objects, especially if they were under heavy use.  Because of this,
use outside of a testing or debugging environment is not recommended.

You may see informational messages in your kernel log when this file is
used:

    cat (1234): drop_caches: 3

These are informational only.  They do not mean that anything is wrong
with your system. To disable them, echo 4 (bit 2) into drop_caches.

 

An illustration of what can go wrong without clearing your RAM!

The example below was a fresh system, install and boot but a number of VMs were created from 20GB templates.  The result is that a server with 128GB RAM was using swap space!  But the drop_caches solution cleared it up in no time.

[root@abc ~]# free -m
             total       used       free     shared    buffers     cached
Mem:        128785     128469        315          0        194     114645
-/+ buffers/cache:      13629     115155
Swap:        28614         11      28603


Tasks: 322 total,   3 running, 319 sleeping,   0 stopped,   0 zombie
Cpu(s): 13.7%us, 12.9%sy,  0.0%ni, 49.4%id, 23.5%wa,  0.1%hi,  0.3%si,  0.0%st
Mem:  131875952k total, 38573652k used, 93302300k free,      992k buffers
Swap: 29301244k total,    11056k used, 29290188k free, 26495228k cached

    PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND                                                                                                                                                                       
  10990 root      20   0 6908m 2.1g 5616 S 13.3  1.6  38:54.92 qemu-kvm                                                                                                                                                                      
  11010 root      20   0 6907m 2.1g 5608 S 15.3  1.6  38:05.71 qemu-kvm                                                                                                                                                                      
  12843 root      20   0 6781m 2.0g 5616 S 73.3  1.6  30:31.32 qemu-kvm                                                                                                                                                                      
  11985 root      20   0 6280m 2.0g 5612 S 65.3  1.6  31:23.33 qemu-kvm                                                                                                                                                                      
  16231 root      20   0 4875m 1.1g 5612 R 23.2  0.9  11:48.82 qemu-kvm                                                                                                                                                                      
   7241 root      20   0 4059m 919m 5608 S  7.6  0.7  21:26.74 qemu-kvm                                                                                                                                                                      
   7902 root      20   0 6134m 346m 5612 S  0.0  0.3  19:59.35 qemu-kvm                                                                                                                                                                      
  13707 root      20   0 6070m 343m 5612 S  7.3  0.3   2:47.46 qemu-kvm                                                                                                                                                                      
  11503 root      20   0 99.9m 4188 3120 S  0.0  0.0   0:00.23 sshd                                                                                                                                                                          
  10301 root      20   0 99.9m 4184 3120 S  0.0  0.0   0:00.48 sshd                                                                                                                                                                          
   7267 root      20   0 99.7m 4144 3120 S  0.0  0.0   0:01.22 sshd                                                                                                                                                                          
   1746 haldaemo  20   0 38268 3104 2408 S  0.0  0.0   0:00.32 hald                           

Tags:

linux, ram, bufferswe, quot, buffers, optimization, doesn, programs, allocate, swap, freed, allocations, realtechtalk, cached, mem, cache, echo, proc, sys, vm, drop_caches,

Latest Articles

  • qemu: could not load PC BIOS 'bios-256k.bin' solution
  • Proxmox How To Custom Partition During Install
  • Hyper-V Linux VM Boots to Black Screen, Storage, NIC Not Found Issues
  • Ubuntu Mint How to Fix Missing/Broken /dev and /dev/pts which causes terminal to immediately close exit and not work
  • How high can a Xeon CPU get?
  • bash fix PATH environment variable "command not found" solution
  • Ubuntu Linux Mint Debian Redhat Youtube Cannot Play HD or 4K videos, dropped frames or high CPU usage with Nvidia or AMD Driver
  • hostapd example configuration for high speed AC on 5GHz using WPA2
  • hostapd how to enable and use WPS to connect wireless devices like printers
  • Dell Server Workstation iDRAC Dead after Firmware Update Solution R720, R320, R730
  • Cloned VM/Server/Computer in Linux won't boot and goes to initramfs busybox Solution
  • How To Add Windows 7 8 10 11 to GRUB Boot List Dual Booting
  • How to configure OpenDKIM on Linux with Postfix and setup bind zonefile
  • Debian Ubuntu 10/11/12 Linux how to get tftpd-hpa server setup tutorial
  • efibootmgr: option requires an argument -- 'd' efibootmgr version 15 grub-install.real: error: efibootmgr failed to register the boot entry: Operation not permitted.
  • Apache Error Won't start SSL Cert Issue Solution Unable to configure verify locations for client authentication SSL Library Error: 151441510 error:0906D066:PEM routines:PEM_read_bio:bad end line SSL Library Error: 185090057 error:0B084009:x509 certif
  • Linux Debian Mint Ubuntu Bridge br0 gets random IP
  • redis requirements
  • How to kill a docker swarm
  • docker swarm silly issues