Ahh, but you are indeed taking advantage of the unfortunate.  Fans of demurrage allege that it is unfair that you have a future time preference while someone else has a now time preference.  Because you are capable of planning for the future, you must be punished with a currency that loses value over time, so that you can be encouraged to be as wasteful as everyone else.

I'm paraphrasing slightly, of course.  But that is the essence of Impaler's argument.