Here is an old VM example which is strange, it looks like Debian has removed this from all the repos. The easiest solution is just to upgrade to a kernel with a working/supported headers, but we don't like doing things the hard way.
In this case it's a 1 year old Debian 11 install, but this can and does happen on random Linux distros sometimes, especially Debian based.
apt-cache search `uname -r`
linux-image-amd64 - Linux for 64-bit PCs (meta-package)
linux-image-5.10.0-34-amd64 - Linux 5.10 for 64-bit PCs (signed)
Normally you should find a headers but you don't in this case, even with an apt update, it just doesn't exist. We even enabled the backports with no luck:
deb http://archive.debian.org/debian bullseye-backports main contrib non-free
deb-src http://archive.debian.org/debian bullseye-backports main contrib non-free
In this example we are using 5.10.0-34 but there's no headers for it, but if we increment and search for -35 it does exist:
apt-cache search 5.10.0-35-amd64
linux-headers-5.10.0-35-amd64 - Header files for Linux 5.10.0-35-amd64
linux-image-5.10.0-35-amd64-dbg - Debug symbols for linux-image-5.10.0-35-amd64
linux-image-5.10.0-35-amd64-unsigned - Linux 5.10 for 64-bit PCs
linux-image-5.10.0-35-amd64 - Linux 5.10 for 64-bit PCs (signed)
So no you're not crazy, 5.10.0-34-amd64 does not exist in the repos, at least not any we have.
Let's try searching directly on Debian's packages site:
https://packages.debian.org/search?keywords=linux-image-5.10.0-34-amd64