It won't destroy your wallet. It will delete all of your transaction and do a rescan (your confirmed transactions will show up again after rescan). Unless you get rid of that transaction bitcoin core will keep rebroadcasting it until it gets confirmed.
2016-12-16 11:20:05 ResendWalletTransactions: rebroadcast 1 unconfirmed transactions
2016-12-16 12:06:48 ResendWalletTransactions: rebroadcast 1 unconfirmed transactions
2016-12-16 12:27:15 ResendWalletTransactions: rebroadcast 1 unconfirmed transactions
2016-12-16 12:56:27 ResendWalletTransactions: rebroadcast 1 unconfirmed transactions
2016-12-16 13:16:29 ResendWalletTransactions: rebroadcast 1 unconfirmed transactions
2016-12-16 15:21:54 ResendWalletTransactions: rebroadcast 1 unconfirmed transactions
Always rebroadcast, so maybe I should only wait?