For whatever reason the current OpenVZ yum repo file enables the RHEL6 version of OpenVZ, why is this bad? Because if you're running Centos 5 it still defaults to using the kernel from RHEL6 which won't work on RHEL5/Centos 5.
I only realized this after wondering why I couldn't boot into OpenVZ that it was using one meant for RHEL6.
To fix the problem you have to edit /etc/yum.repos.d/openvz.repo and disable the "[openvz-kernel-rhel6]" section by changing "enabled=1" to "enabled=0", see the example of the two sections below. Make sure that you don't install "vzkernel" of any sort if you're on Centos 5, it should be called "ovzkernel" and that's another way to know if things are working as you like.
ovzkernel=Centos 5
vzkernel=Centos 6
vi /etc/yum.repos.d/openvz.repo
# Stable branches
[openvz-kernel-rhel6]
name=OpenVZ RHEL6-based kernel
#baseurl=http://download.openvz.org/kernel/branches/rhel6-2.6.32/current/
mirrorlist=http://download.openvz.org/kernel/mirrors-rhel6-2.6.32
enabled=0
gpgcheck=1
gpgkey=http://download.openvz.org/RPM-GPG-Key-OpenVZ
exclude=vzkernel-firmware
[openvz-kernel-rhel5]
name=OpenVZ RHEL5-based kernel
#baseurl=http://download.openvz.org/kernel/branches/rhel5-2.6.18/current/
mirrorlist=http://download.openvz.org/kernel/mirrors-rhel5-2.6.18
enabled=1
gpgcheck=1
gpgkey=http://download.openvz.org/RPM-GPG-Key-OpenVZ
centos, openvz, kernel, installfor, yum, repo, enables, rhel, defaults, couldn, edit, etc, repos, disable, quot, enabled, sections, install, vzkernel, ovzkernel, vi, branches, baseurl, http, download, org, mirrorlist, mirrors, gpgcheck, gpgkey, rpm, gpg, exclude, firmware,