Someone could definitely steal your list if recovery words if they really wanted, but really you shouldn't be writing them down on a piece of paper with the header 'my bitcoin wallet recovery words'. For example - What if you were to number specific words in your dictionary? Sure, someone who knew what they were looking for could still steal it, but the vast majority of people wouldn't realize what it was even when looking at it. Security isn't just physical - it can also be obtained through obscurity and storing data in an unexpected way.