Correct. But with increasingly powerful hardware becoming increasingly cheaper, it is becoming increasing easier for attackers to mimic existing addresses.

You'll often see people saying something along the lines of check 3 characters at the start of the address and 3 characters at the end. This is completely insufficient. An attacker can very easily brute force an address which matches this criteria, or even 4 or 5 characters. The only way to be safe is to double check the entire address. It takes <10 seconds to do this, so there really is no excuse for ever falling to such a scam.