What does "Just" mean when a Libertarian says it?
Clearly Libertarian society would not work if every individual could opt out of their duty to enforce the "perfect" rights of others while they are so fortunate that their own rights were not presently being violated.
The question of interventionism versus isolationism is really a question of morality versus amorality. A moral person believes that action is required in the presence of injustice, even if that injustice occurs to others. A person who does not believe that he must intervene in a situation where he believes rights are being violated in others is effectively amoral because to him justice is a narrow pursuit of self interest i.e. self defense is their only exercise of "morality".
The reflexivity of amoral libertarianism is no defense. A defense of individualistic libertarianism based on the fact that there are no problems in theory if everyone were to accept this philosophy could be equally applied to Christian or Socialist altruism, and thus is equally invalid.
If Libertarians were to adopt an amoral or anti-violent attitude, a pair of fascists could exterminate an entire country of amoral Libertarians by restricting their genocidal persecutions to minorities consisting of one individual at a time.
Thus, it is imperative that a Libertarian idea of legitimate violence must be extended to include not only self defense, but also defense against a disproportionate distribution of the costs associated with the presence of injustice. In the example above, each moral person could not justly self defend from the prodding of others to assist the person being persecuted by the two fascists, thus assisting in bringing the collective interest more into line with each individual's interests.
Since a sense of justice necessitates acting in the defense of others, and not exclusively for oneself, how could any separation of distance negate that obligation?