There was a rumor or allegation that Josh referred to his customers, poor people, regular people, basically anyone that was not a millionaire as a "muggle"
 
Someone uploaded a moderator chat here: https://bitcointalk.org/index.php?topic=857670.msg11107922#msg11107922

In which it was confirmed to be true and posted here: https://bitcointalk.org/index.php?topic=857670.msg11108104;topicseen#msg11108104

So the term "muggle" has been thrown around the thread a lot lately and many people are flipping this around and calling Josh a muggle.   Based on his use of the term and the information which has been leaked and posted Josh fits his own description of a muggle so it seems fitting.