If authority is the willingness and ability to command obedience to one’s will, authoritarianism would be a belief that someone is superior in some manner (ethically, politically, ect.) because he or she has such authority.
Since government, as popularly constructed, is given the sole discretion of interpreting and enforcing the law, even in conflicts between the government and an individual, a government of limited constitutional powers is still authoritarian by nature.
In many cases, authority is based in aggression. But that is not necessarily always true. Widespread racism and sexism, for example, enforced through rightful private property claims could also constitute instances of authoritarianism. Some forms of indoctrination could manifest non-aggressive means of gaining authority over others as well.
In a society of widespread authoritarianism, liberty would have very little substance. Even if we achieved a libertarian society, it would be short-lived if the prevailing notion about the non-aggressive authority that can be held over others were left unchallenged.