I once interned at Cato, where I read John Allison's book on the financial crisis. He had a similar insight; housing is not an investment since it doesn't produce value per say (like a factory).

I also read somewhere that it's not really worth it unless you plan on living at the same place for some time. I can attest to that; moving is quite a hassle, the paperwork is confusing...