Here is an old VM example which is strange, it looks like Debian has removed this from all the repos. The easiest solution is just to upgrade to a kernel with a working/supported headers, but we don't like doing things the hard way.
In this case it's a 1 year old Debian 11 install, but this can and does happen on random Linux distros sometimes, especially Debian based.
apt-cache search `uname -r`
linux-image-amd64 - Linux for 64-bit PCs (meta-package)
linux-image-5.10.0-34-amd64 - Linux 5.10 for 64-bit PCs (signed)
Normally you should find a headers but you don't in this case, even with an apt update, it just doesn't exist. We even enabled the backports with no luck:
deb http://archive.debian.org/debian bullseye-backports main contrib non-free
deb-src http://archive.debian.org/debian bullseye-backports main contrib non-free
In this example we are using 5.10.0-34 but there's no headers for it, but if we increment and search for -35 it does exist:
apt-cache search 5.10.0-35-amd64
linux-headers-5.10.0-35-amd64 - Header files for Linux 5.10.0-35-amd64
linux-image-5.10.0-35-amd64-dbg - Debug symbols for linux-image-5.10.0-35-amd64
linux-image-5.10.0-35-amd64-unsigned - Linux 5.10 for 64-bit PCs
linux-image-5.10.0-35-amd64 - Linux 5.10 for 64-bit PCs (signed)
So no you're not crazy, 5.10.0-34-amd64 does not exist in the repos, at least not any we have.
Let's try searching directly on Debian's packages site:
https://packages.debian.org/search?keywords=linux-image-5.10.0-34-amd64
linux, debian, upgrade, kernel, headers, solutionhere, vm, repos, easiest, supported, install, distros, apt, cache, uname, amd, pcs, meta, update, doesn, enabled, backports, deb, http, archive, org, bullseye, contrib, src, increment, header, dbg, debug, symbols, unsigned, packages, https, keywords,