cat /proc/user_beancounters produces the following:
kmemsize 1861537 5139870 12752512 12752512 26965041
Notice the failcnt "26965041", that is for kmemsize and at first it confused me. The system had enough guaranteed and enough burst RAM available. kmemsize is a variable indepedent of that, but who cars about the explanation right, let's just make things work. I was installing CPanel and wondered why the install process was taking forever until I checked my beancounters:
The solution is to of course increase kmemsize, in many cases you'll want to double, triple or even quadruple it. Also note it's bad that my barrier and limits are the same, so I've arbitrarily changed the limit in the example below. I can't remember the consequences but I read certain processes will get killed if the server runs out of memory.
vzctl set vpsid --kmemsize $((1275251*4)):$((1875251*4)) --save
The above increased my kmemsize allowances by 4x and should keep everything good. In my opinion tihs is a silly feature and should be more tuned/controlled by the guaranteed memory allocation in OpenVZ.