Note that locking in BIP 91 is not the same thing as locking in Segwit. BIP 91 is an activation mechanism for Segwit, that functions like the UASF does. Locking in BIP 91 amounts to miners saying, "336 blocks from now, we all promise to ignore non-Segwit-signalling blocks, and build a chain of Segwit-supporting blocks."

Assuming that all the miners now supporting BIP 91 stick to the terms of that agreement, after a short, 336 block grace period, they will begin mining, and mining on top of, Segwit-signalling blocks only. This means that if there are still miners creating non-Segwit blocks, they will have limited hashpower, and consequently be building a much smaller chain than the Segwit-mining majority.

Because the network accepts the longest blockchain as the valid one, these non-Segwit miners will be building an invalid chain, giving them a significant financial incentive to switch to the Segwit chain. Because all blocks in this chain will be signalling for Segwit, the next 2016-block signalling period for Segwit itself should reach the needed 95% signalling threshold required for activation. Then, after another grace period, Segwit itself should activate, and will be usable on the network.

This all assumes that these miners do what they've comitted to do. If they break their word at some point, the situation sort of explodes in complexity, and a lot of what I've written above becomes invalid.

To speak more directly to your question, the miners who do NOT support Segwit, as noted above, will be mining a minority chain, and subject to having their blocks rejected by the majority of the hashpower.

If miners and users prefer a chain without Segwit (and presumably bigger blocks) they will likely hard fork themselves away from the majority consensus rules, creating an altcoin. This prevents a lot of potential harmful interference between the two chains. Currently, this hard fork effort is known as BitcoinCash, and uses client software BitcoinABC.

Note that the "success" or "failure" of a lot of these initiatives, whether they be activation mechanisms, technologies, concensus rules, or altcoins, are subject to the support of both miners and users. I mention this because we don't have a clear idea of where miner and user support really lands on a lot of these proposals. We know what they're saying, yes, but we don't know if they are being genuine in signalling their support, or if they'll change their minds.

It's completely possible, for example, for users to support Segwit2x long enough to activate Segwit, and then not follow a 2 MB hard fork. It's also possible for a fork of the Bitcoin software to start out with overwhelming miner support, and for that support to migrate away from that chain if there is insufficient use of that coin to maintain the value of that coin (and thus, profitable mining).

There have been a couple of attempts (1 & 2) at Sybil-resistant straw polls. Thus far, both polls reflect the biases of the faction that created them (among other biasing issues), making them minimally effective as real gauges of opinion. I'm not aware of any good indicators, but there are ideologues on either side who exhibit (I'll say this diplomatically) misleading optimism and confidence about how things "will" work out. I advise you to interpret such claims with caution, even when they are represented as fact, and to do your own research, using a variety of sources and opinions.