I find the latter scenario to be more terrifying. It's like being the last man on Earth only the ratio of one human to Earth is much, much smaller than the ratio of one Earth to the universe. Statistically speaking it's basically a lock that we are not alone. It's also incredibly unlikely we will encounter what we would consider "intelligent life" for many, many years (if ever). There's just too much space out there in space.