The Security page on the Preferences dialog allows you to configure how passwords are secured and what certificate authorities are trusted.
Refer to documentation of page sections:
Uncheck Use master passwords to remove master password. You will be prompted for current master password, even if you already entered it in current session.
Use Change master password to change master password. Master password setting dialog will appear.
Uncheck the Remember password for duration of the session checkbox to avoid your password (or private key passphrase) being stored in the memory for a duration of the session. You will be prompted for your password/passphrase again, when reconnecting a lost connection and for background transfers.