If you need quick access from your linux desktop to the VPS there is an easy solution that won't require a password to login. First, create a private and public ssh key pair on your desktop:
ssh-keygen -t rsa
Copy the public key to the vps (replace vps by the hostname or IP address):