I get an error with removeprivkey on encrypted wallets. Works fine on non-encrypted.
foo@bar:~/work/bitcoin/src$ ./bitcoind getnewaddress
mgVu7WXEdbCHnKiQW8q1g7ND5e6jR97NGX
foo@bar:~/work/bitcoin/src$ ./bitcoind dumpprivkey mgVu7WXEdbCHnKiQW8q1g7ND5e6jR97NGX
93AywmB22cb5H59m6vttmPUFCPf98sB7NEwj48q9RvbaMGQHzbd
foo@bar:~/work/bitcoin/src$ ./bitcoind removeprivkey 93AywmB22cb5H59m6vttmPUFCPf98sB7NEwj48q9RvbaMGQHzbd
foo@bar:~/work/bitcoin/src$ ./bitcoind importprivkey 93AywmB22cb5H59m6vttmPUFCPf98sB7NEwj48q9RvbaMGQHzbd
foo@bar:~/work/bitcoin/src$ ./bitcoind encryptwallet password
foo@bar:~/work/bitcoin/src$ ./bitcoind walletpassphrase password 1000
foo@bar:~/work/bitcoin/src$ ./bitcoind getnewaddress
msKK1iq6LYLxz8iUvovWMqh37T3U9fha4M
foo@bar:~/work/bitcoin/src$ ./bitcoind dumpprivkey msKK1iq6LYLxz8iUvovWMqh37T3U9fha4M
92isYRmvbjH7TcYnqKU3YZUmQvUatRCmYSPYsAoEku6Xv8s7qhy
foo@bar:~/work/bitcoin/src$ ./bitcoind removeprivkey 92isYRmvbjH7TcYnqKU3YZUmQvUatRCmYSPYsAoEku6Xv8s7qhy
error: {"code":-1,"message":"GetAllReserveKeyHashes() : unknown key in key pool"}