Technically not correct since the miner hashes the fixed length block header (80 bytes) and it won't change whether the block has 1 transaction or thousands.
The miner, indeed, hashes a 80 bytes block header, but to reach the certain merkle root, he has to hash transactions. Hashing transactions may take some time; it may not seem slow in practice, but in theory, doesn't it take more time to hash multiple transactions over and over again, instead of just one?