Since 1970's fiat hasn't been backed by gold either but that hasn't affected it's price tho, it keeps losing value at a relatively steady speed. The reason to back something up by physical objects is to keep the value even if we loose all electricity or if the internet goes down or something like that. Imagine how that would affect a regular bank and it's easy to see why virtual centralized currency is unsafe.

However decentralization opens up for more possibilities, at least in theory. We could run a full node deep down in a bunker safe from nuclear bombs to keep cryptocurrencies working even after a nuclear war.

We could also send a full node to the moon and sync via satellite links and it would survive even if the entire earth is destroyed (which would also remove all the gold).

With above in mind I'd say cryptocurrencies are safer than physical objects as it would take so much to destroy it that we'd most likely destroy ourselves first.