[gentoo-user] {OT} GPG: pub & sec keys required to decrypt?

[Grant]

I've been encrypting and decrypting email on the same remote server.
I was under the impression that this was a security risk because it
meant having the public and private keys on the same machine.  I tried
importing the public key to my local system and decrypting via
enigmail but I got "Error - secret key needed to decrypt message".  I
imported the private key locally and now it decrypts fine, but I have
both keys on the same system again.

My understanding of GPG is weak.  Can someone point out my misconception(s)?

- Grant