Moral of the story: DON'T collect dust.
Or collect dust into an old-fashioned single-address wallet or perhaps into a paper wallet. Would that work better? If so, then the moral could be amended to, "Don't collect dust into a HD wallet."
Makes no difference fee-wise. When you collect on a single address, you just have worse privacy until you spend.