That just tells YOUR router to assume the IP will stay the same. If your ISP thinks you're using the new number and you router thinks it's supposed to use the old number, things will break.

If you have a cable modem (or any other almost-always-on connection), you could wait months before your current IP actually gets recycled and assigned to someone else. Which means everything will work perfectly for months, until you've completely forgotten that you did this, THEN break.