Obviously you can, that's why called "peer to peer" network. That means you are able to send fund to any other user directly through internet without using third party involvement. But is it not bit risky? If I want to buy any token and you want to sell same token so we could exchange it each other. But question is how will you trust me since you don't know my real identity. That's why people just simply use an exchange for buy or sell anything. I don't see any problem with it.