I tend to agree with you on this matter. On a 10x100m test in the pool, I average at 1'12'' without wetsuit. With wetsuit, this tends to drop between 1'07'' and 1'05'', indeed depending on the pool temperature.
During an open water swim however, the difference is not that pronounced. It is still faster, but not nearly by 10''/100m.
My general rule of thumb is: Wetsuit legal? Wetsuit swim! (Eventhough compared to the competition, my advantage in a non-wetsuit legal swim is bigger)