I'd be nervous with that kind of money whether it's bitcoin, dollars, or something else.  If you use some common sense and have proper real life security you should be fine.

Like others said, the ability to crack that key is so far fetched right now.