Make this entry in your ~/.bash_logout file:

kill $SSH_AGENT_PID

Don’t do this if you’re using keychain, because the whole point of using keychain is to be able to log in and out without having to re-enter your passphrases.

If you don’t have a ~/.bash_logout file, create one. There should be a system-wide default logout file at /etc/skel/.bash_logout to use as a model.