There are many, many instances of defensive gun use that either happened so fast that there wouldn't have been enough time to chamber a round and still survive the fight, or their other hand was unavailable to manipulate their firearm.

Defensive gun uses often happen in close quarters, so there is a good possibility that your other hand will be occupied keeping the attacker away from your gun while you try to get it into the fight.

I prefer to carry my firearm in such a way that I can draw and use it with one hand. That means having it on me (concealed carry), having a round in the chamber, and having the safety off. It also means I have to be particularly aware of safety, and train often so that those safe gun handling practices become second nature.

I understand the risk that you're trying to mitigate by not keeping a round in the chamber, but doing so exposes you to other risks.