Since gnupg is shipped non-suid, it always warns with "Warning: using insecure memory!" every time it is invoked, which is a bit annoying. Consider either suid gpg or patching so that the warning isn't given by default.
Use the --no-secmem-warning option. I'm personally more worried about possible buffer overflows in GnuPG (how ironic) than about people physically stealing my disks and examining my swap space. It's a trade off, but I think that it's quite right that we ship GnuPG without the setuid bit.