CPUs were mining for approximately 2 years before GPU mining started. The first GPU miner was known to keep the mining software to himself and mined a lot of coin at the time before public GPU mining software became available.
GPU mining appeared roughly 1.5 years after the Bitcoin genesis block, in mid 2010. ArtForz was the first to use it (see ArtForz story here
http://www.ofnumbers.com/2014/04/20/how-artforz-changed-the-history-of-bitcoin-mining/) and indeed he did not go public but GPU public mining code appeared very shortly, see, eg.
https://bitcointalk.org/index.php?topic=1334.0. In late 2010/ early 2011 GPU mining was already dominating but CPU mining was still possible. First mining pools appeared in late 2010.
CPU mining stopped being profitable (although malware bots continued to mine it longer) roughly in Q2 2011.