If you want to make sure only a certain IP can access your server for any service or protocol here is a way to do it (just be sure you have access to the IP(s) mentioned or you will be locked out).
iptables -A INPUT -i lo -j ACCEPT
iptables -A INPUT -m state --state ESTABLISHED,RELATED -j ACCEPT
iptables -A INPUT -p tcp -s IP.IP.IP.IP -j ACCEPT
iptables -A INPUT -j DROP
service iptables save