They are of the believe that there is one correct viewpoint and it is their own. So your observation is correct. Their worldview is absolutely not compatible with moral relativism.
This is an unfair generalisation of libertarians. Some libertarians think that their view is the only correct one but not all of us do. I'm sure the same could be said for some authoritarians and some people in between.
I am a libertarian and I wouldn't go as far as to say I am a moral relativist. I believe I am right on the whole about moral issues but I accept that I may be wrong and keep an open mind to other viewpoints. I also acknowledge that there are many difficult grey areas in morality.
I often find this to be the case. Libertarians are generally quite stubborn in their personal views but far less prone to calling for the muzzling of opinions that they personally see as harmful. They have a personal constitution that is strong yet are less prone to propose one-size-fits-all social policies as most believe in self-determination over moral absolutism.
The trend is most definitely not universal, though. I have met a number of modern objectivists that are quite absolutist and unwilling to accept any level of dissenting opinion on religious or economic issues.