"The total mass of the asteroid belt is approximately 4% that of the Moon" from Wikipedia. I think you're massively underestimating the sheer quantity of stuff in the Earth. That said, of course physical resources are not literally unlimited - they're just so abundant that the primary limiting factor (by a long way) is our resourcefulness in manipulating them.