The following assumes the computer is local/physical to you and/or it always has a LANIP so it can be accessed on site without having a default gateway.
The key to this is not to set a default gateway for your computer or you can set a script on boot or other time to delete the gateway (where eth0 is the NIC you are using):
route del default eth0
50.80.20.2 is the VPN server you connect to
192.168.1.1 is your........