Yes you are downloading a binary. Don't worry the software make sure the download is actually compiled by the ppa account holder (or the person responsible for a package in the repositories), But you would have to actually trust that person not to inject something bad in the code.

But again if you are not checking the source code and compiling it without reviewing the code. IMO it is the same thing. You would have to trust someone.