proxmox vm networking breaks when you restart your network on the hostnode
I did a systemctl restart networking and it broke Proxmox VM connectivity!
#proxmox is the problem after restarting the network the tap devices go to disabled state
[2230884.919905] vmbr0: port 7(tap118i0) entered disabled state
[2230884.948864] vmbr0: port 8(tap122i0) entered disabled state
[2230884.972748] vmbr0: port 6(tap119i0) entered disabled state
[2230885.004745] vmbr0: port 5(tap117i0) entered disabled state
[2230885.036736] vmbr0: port 4(tap112i0) entered disabled state
[2230885.060693] vmbr0: port 3(tap111i0) entered disabled state
[2230885.100729] vmbr0: port 2(tap110i0) entered disabled state
[2230885.227515] vmbr0: port 1(enp0s8) entered blocking state
[2230885.227518] vmbr0: port 1(enp0s8) entered disabled state
[2230885.228071] device enp0s8 entered promiscuous mode
As we can see the tap devices went disabled which makes sense BUT they didn't come back so all VMs now have no network connection (the equivalent of basically unplugging the cable from the server):
#use brctl to fix
brctl addif vmbr0 tap118i0
Feb 25 06:30:42 proxmoxdal01 kernel: [2231303.128510] vmbr0: port 2(tap118i0) entered blocking state
Feb 25 06:30:42 proxmoxdal01 kernel: [2231303.128513] vmbr0: port 2(tap118i0) entered disabled state
Feb 25 06:30:42 proxmoxdal01 kernel: [2231303.128754] vmbr0: port 2(tap118i0) entered blocking state
Feb 25 06:30:42 proxmoxdal01 kernel: [2231303.128756] vmbr0: port 2(tap118i0) entered forwarding state
Here is a handy bash script that will automatically readd all of your tap devices to vmbr0
for tapdev in `ip a|grep tap| awk '{print $2}'|sed s/://`; do
brctl addif vmbr0 $tapdev
done
And they all came back:
Feb 25 06:31:52 proxmoxdal01 kernel: [2231373.163869] vmbr0: port 3(tap110i0) entered blocking state
Feb 25 06:31:52 proxmoxdal01 kernel: [2231373.163873] vmbr0: port 3(tap110i0) entered disabled state
Feb 25 06:31:52 proxmoxdal01 kernel: [2231373.164059] vmbr0: port 3(tap110i0) entered blocking state
Feb 25 06:31:52 proxmoxdal01 kernel: [2231373.164062] vmbr0: port 3(tap110i0) entered forwarding state
Feb 25 06:31:52 proxmoxdal01 kernel: [2231373.165739] vmbr0: port 4(tap111i0) entered blocking state
Feb 25 06:31:52 proxmoxdal01 kernel: [2231373.165742] vmbr0: port 4(tap111i0) entered disabled state
Feb 25 06:31:52 proxmoxdal01 kernel: [2231373.165926] vmbr0: port 4(tap111i0) entered blocking state
Feb 25 06:31:52 proxmoxdal01 kernel: [2231373.165928] vmbr0: port 4(tap111i0) entered forwarding state
Feb 25 06:31:52 proxmoxdal01 kernel: [2231373.167515] vmbr0: port 5(tap112i0) entered blocking state
Feb 25 06:31:52 proxmoxdal01 kernel: [2231373.167520] vmbr0: port 5(tap112i0) entered disabled state
Feb 25 06:31:52 proxmoxdal01 kernel: [2231373.167717] vmbr0: port 5(tap112i0) entered blocking state
Feb 25 06:31:52 proxmoxdal01 kernel: [2231373.167721] vmbr0: port 5(tap112i0) entered forwarding state
Feb 25 06:31:52 proxmoxdal01 kernel: [2231373.169253] vmbr0: port 6(tap117i0) entered blocking state
Feb 25 06:31:52 proxmoxdal01 kernel: [2231373.169260] vmbr0: port 6(tap117i0) entered disabled state
Feb 25 06:31:52 proxmoxdal01 kernel: [2231373.169396] vmbr0: port 6(tap117i0) entered blocking state
Feb 25 06:31:52 proxmoxdal01 kernel: [2231373.169398] vmbr0: port 6(tap117i0) entered forwarding state
Feb 25 06:31:52 proxmoxdal01 kernel: [2231373.170867] vmbr0: port 7(tap119i0) entered blocking state
Feb 25 06:31:52 proxmoxdal01 kernel: [2231373.170869] vmbr0: port 7(tap119i0) entered disabled state
Feb 25 06:31:52 proxmoxdal01 kernel: [2231373.171023] vmbr0: port 7(tap119i0) entered blocking state
Feb 25 06:31:52 proxmoxdal01 kernel: [2231373.171026] vmbr0: port 7(tap119i0) entered forwarding state
Feb 25 06:31:52 proxmoxdal01 kernel: [2231373.172605] vmbr0: port 8(tap122i0) entered blocking state
Feb 25 06:31:52 proxmoxdal01 kernel: [2231373.172608] vmbr0: port 8(tap122i0) entered disabled state
Feb 25 06:31:52 proxmoxdal01 kernel: [2231373.172768] vmbr0: port 8(tap122i0) entered blocking state
Feb 25 06:31:52 proxmoxdal01 kernel: [2231373.172770] vmbr0: port 8(tap122i0) entered forwarding state
Tags:
proxmox, vm, networking, restart, hostnodei, systemctl, connectivity, restarting, devices, disabled, vmbr, enp, blocking, promiscuous, mode, didn, vms, equivalent, unplugging, server, brctl, addif, feb, proxmoxdal, kernel, forwarding, bash, automatically, readd, tapdev, ip, grep, awk, sed,