Not afraid of Bitcoin XT, however its a shame Mike and Gavin are even trying to label it with a different name. Bitcoin layman will assume the state of things are really bad, and that the people invested in Bitcoin are going to be scammed out of wealth etc. etc. I would be surprised if we did not see a number of large sell offs soon and its going to be attributed too peoples misunderstanding of us working towards consensus on a Blocksize.
Price will plummet once the media has a chance to 'learn' about Bitcoin XT and report on it. In the past the media has put forth zero effort towards researching articles on Bitcoin, typically I have seen them as the largest sources of FUD and impact on the relative price.
Your concerns are valid, however my beliefs are that it is placed in the wrong boat, you should be concerned about the media backlash by not understanding whats going on that will occur sooner than later. The technology/coins are not going to just vanish magically, when in doubt WWSD. What would Satoshi Do: in this case Nothing, his coins are still valid and yours will be valid regardless if agreement on Blocksize is: 1mb, 4mb, 20mb, or 32GB per block.
My fear is if we become so centralized, that we get to a situation where we have someone or something choose which transactions to allow in blocks. My fear is someday someone can take away our rights to use Bitcoin the way it works today...we cant allow for this to occur.