That depends on how you define "another layer of trust".

Colored coins are inherently used for tracking things outside the blockchain - that by definition means representing obligations a.k.a counterparty risk. Any technique that tracks obligations outside the blockchain will be tracking counterparty risk.

But none of that has nothing to do with the underlying technology used to create the token. Colored coins as tokens are no different from other bitcoins. A colored coin token doesn't all of a sudden become less trustworthy than a non-colored Bitcoin.