In a non-parallelizable PoW, we will have, say, 1.000.000 processing units all competing individually for a block. Some are faster, some are slower, but they do not differ so widely in performance. Every processing unit corresponds to a single processor (no parallelization advantage for a GPU or a multi core; however a single person might own several competing processing units, which might sit on a single die or a single GPU or several racks).
You lost me here: as others have said, you can parallelize by buying more computers. If you want to use a lot of memory and branching to remove the advantage of GPUs, I can still parallelize if I have more money than anybody else.
Oracle quotes me just over $44,000 USD for a fully loaded
SPARC T4-1 Server (PDF) with 256GB of RAM, supporting 64 simultaneous compute threads. It can consolidate 64 virtual machines (with 4GB of RAM each) in 2U, while drawing under 800 Watts. Granted, the 4MB L3 cache becomes 64kB afer being split 64 ways (128KB L2 becomes 16kB split 8 ways (8 cores)), but each "machine" is only costing you about $687.50 USD.
If you had money to burn you could probably put 20 in a rack for 1280 virtual machines in a rack. How is anybody doing this not taking advantage of parallelism? Edit: assuming your "racks" of 4 GPUs are 4U each, your example for the "parallel" case has only 8 times the density (256 threads per U vs 32).