For context, I have written before why libertarians, and particularly libertarians committed to small government, should support open immigration as a matter of principle. Further, I have given a consequentialist argument for open immigration and what that entails.
For me, the least impactful line of argument I would think is the constitutional argument. That so many constitutionalists nevertheless support uniform immigration restrictions demonstrates how meaningless the constitution is if its most ardent defenders conveniently pervert it so far from the original meaning.
As an Implied Power
A common line is that the Naturalization Clause, which gives the legislature the power to make a uniform process of becoming a citizen, implies the power to regulate immigration in context with the Necessary and Proper Clause.
That is an interesting idea, and it would have been worth mentioning by the Federalists since immigration had been under the domain of the states during the existing constitution. Yet, the framers who supported the constitution never so much as hinted at that idea during ratification. In fact, “Agrippa,” the Anti-Federalist who is supposed to be John Winthrop, lamented that congress would have no such power under the then-proposed constitution.
It was not until 1875, after congress had passed four separate naturalization bills, did the Supreme Court discover the new-found power to control immigration.
As a Protection from Invasion
Further in Article 1, Section 8, congress is also given the power to summon the militia to “repel Invasions.” This line of argument has been given by Ron Paul and other less distasteful politicians as a reason to resist open immigration.
For this to be true, we would need to look at the meaning of the word “invasion” at the time of ratification. The widely circulated Johnson’s Dictionary defined an invasion as “a hostile entrance, an attack.”
I have defended extending open immigration, at a minimum, to peaceful, honest people. Obviously, that would exclude violent criminals who have not offered restitution for their crimes. With that said, peaceful, honest people entering the country to better their lives should not fall within the scope of “a hostile entrance” by any means.
As a Limit on Slavery
I do not encounter this argument often, but the constitution does provide for the prohibition of “Persons as any of the States now existing shall think proper to admit” after to 1808 in the 13 original states. Ironically, this was meant as a check on congressional power to control the importation of slaves.
In all other cases, immigration control should be reserved for the states, according to the constitution. However, as a practical matter, any federal immigration controls like that would break down under political pressure within a generation, so soon enough all the states would be setting their own policy. After all, it is unlikely that the other 37 states would be willing to pay for the immigration enforcement of others states.
Lessons from History
Mary Ruwart once wrote, “We reap as we sow. In trying to control others, we find ourselves controlled. We point fingers at the dictators, the Communists, the politicians, and the international cartels. We are blithely unaware that our desire to control selfish others creates and sustains them.”
The decentralization of power is a good thing. For one, it would slow plans for this New World Order that so many constitutionalists tell me about. The expansion of immigration controls follow closely with the expansion of government power in general.
For the most part, peaceful, honest foreigners are trying to escape exploitation so they might live somewhere they do not have to get permission to create wealth. It is a false choice to have to choose between our own happiness and abundance and that of others. All interests are served by practicing non-aggression. By refusing to aggress against others, the special interest groups and politicians in government have no authority over of us.
Image credit: Jaume d’Urgell, with a Creative Commons license
Who Plans Whom? 
Now that I think about it, a better title would have been "Constitutional Arguments Against Immigration Restrictions."