Set permissions: let's make it hard, anyways.
~/.pgp ~/.premail
~/.pgp/* ~/.premail/*
Remember, premail is designed for being used on secure machines. This means that all someone has to do to compromise every aspect of your premail setup is read a file that they shouldn't be able to (your secrets file). This is what the premail passphrase is for: it makes your secrets file be encrypted when you're not actually using it, but this still isn't good enough, because when you are using it it's in the clear. You could premail -login/logout every time you needed to use premail, but then it wouldn't be saving you much time.