Never trust a survey where you don’t know the question

A few years ago, in front of the federal elections here in Germany, the biggest (and ruling) party was frantically searching for something that would give them some votes and cost their voter base nothing.

They found it. It was a very stupid idea, but the targets were pedophiles, so it was not a problem, since nobody likes them, right?
The proposition: Put stop signs on websites with pedophile material.

Unfortunately for the ruling party it was not an auto-victory. Many people – including the victims of pedophiles – spoke out loud against it, for several reasons (you can try finding out how many counter-arguments you can find in one minute).

(source: Stupidedia CC-by-sa 2.0)

In the end this stupid idea was good for one thing: The official petition website for the Bundestag got new servers, since 130’000 people (record by far) signed the petition against the filters and overloaded the old servers.

But during all this a survey was made by the ruling party after the first backfire from the “people from the internet”.

It was a phone question and the survey was made by “the” institute for such things in Germany.

The result: 91% of asked people were for the stop sign filters.

Which made MOGIS (club for victims of child abuse) order a survey of their own. Same company, same method, just a different question (and more objective imho).

Result? 92% against the stop signs. Just two weeks later, in a phase where it was not widely discussed after the first head shakes, so it was not because everyone saw the truth and converted in those two weeks.

The difference was just the question asked. 91% for or 92% against, depending on the question asked.

NEVER EVER trust a survey where you don’t know the exact question. Without the question, the result is useless.