After Numenor fell, Elendil lead the rest of the Numenorean men back to middle earth, to the northwestern coasts where many other humans blood already lived, although very few of them remembered the elvish tongues. This way, the Numenoreans were always scarce compared to the other human races, over whom they ruled because they were men of long lives and important wisdom and power.
Given this, Numenoreans always ruled over men using the common tongue, but it was extended and enriched with some words extracted from the elvish tongues. This common tongue would extend very far, even among enemy tribes, and even the Duneidan used it, this meant that by the time of the war of the ring at the end of the third age the elvish tongue was only known by a select group of Gondorean men, and they rarely used it, even though most places and cities had elvish names and elvish meanings.
