That's true. Sex isn’t everything. The most important component is that your partner wouldn't irritate you with his/her actions and that your habits would be somewhat similar to “sleep with open window vs closed window.” Also, it’s important that you have the same attitude in money matters, helping people matters, and raising children matters.
You also have to have the same understanding of what closed and open relationship means. If you feel like dipping your beak once in a while, then you shouldn’t call your partner a whore is s/he does it as well.