It is hypocritical and dishonest at best, and dangerous at worst. If no one is allowed to build on your code or use your code for anything, then you are going to have far fewer people looking at it, examining it, testing it, using it. As you say, few people can actually interrogate the code themselves, and most users rely on independent developers or power users examining the code of open source projects on their behalf. If you aren't actually allowed to do anything with the code, then there is far less incentive to spend your time going through it.