The following assumes the computer is local/physical to you and/or it always has a LAN IP so it can be accessed on site without having a default gateway.
The key to this is not to set a default gateway for your computer or you can set a script on boot or other time to delete the gateway (where eth0 is the NIC you are using):
route del default eth0
126.96.36.199 is the VPN server you connect to
192.168.1.1 is your LAN default gateway/router
ip route add 188.8.131.52 via 192.168.1.1
The routing command above makes it so the only external traffic not passing through the VPN is the connection to the IP of your VPN (necessary of course to be able to establish the connection).
With the above even if you lose the connection to your VPN no traffic can reach the outside without being connected to the VPN.