By outward appearances Vladimir Putin seems to be very popular in Russia.
This is interesting.
Putin started a war against Ukraine that has resulted in thousands of Russian casualties,
has seen few major strategic objectives be accomplished,
and has made NATO more united than any time since the Cold War, with two countries,
Sweden and Finland, likely to join soon precisely because of the war.
Yet Putin is still drawing large crowds at pro-government rallies.
And the poll numbers show similar levels of support.
Before the war began, Putin’s approval rating in Russia was 67.2%.
After the war began, in a classic rally-round-the-flag effect, it jumped to 81.6%.
Things seem pretty great for the Kremlin.
But there is a deeper question of whether we can trust any of this information as being
accurate.
It is common practice in autocratic regimes to force state employees to attend rallies
under threat of losing their jobs.
Meanwhile polling is subject to a problem known as preference falsification.
Imagine you lived in Russia and received a phone call asking whether you support Vladimir
Putin.
It is possible that the caller comes from a totally legitimate news agency.
But it is also possible that the caller is with a state entity and would not take kindly
to hearing about your negative opinion.
Thus, even if you hated Putin, you might still say that you support him just to avoid a trip
to the Gulag.
A poll showing 67.2% or 81.6% approval
may therefore only reflect a much smaller percentage of true support.
This is a problem for outsiders.
Putin’s ability to continue the war depends on the war’s popularity.
If all we see are inflated poll numbers,
then it will be hard to gauge the effectiveness of policies like sanctions, which have come
from almost all major economic powers.
Fortunately researchers have found a way to deal with this kind of problem.
It’s called a list experiment.
Instead of directly asking whether a respondent supports Putin, the pollster calculates the
support in an end-around way.
The poll begins by asking, say, 800 people a question that has nothing to do with Putin.
“Think about Joseph Stalin, Leonid Brezhnev, and Boris Yeltsin.
How many of these people’s activities do you generally support?”
The answers the survey respondents can give are numbers: 0, 1, 2, or 3.
Thus, the respondents are not being asked which of these leaders they support,
just how many.
That question by itself appears to give no insight at all about Putin.
But the pollster can solve that problem by asking a separate set of 800 people a similar
question.
They still ask the same question about
Stalin, Brezhnev, and Yeltsin.
But now they add Putin to the list.
And instead of asking for a number 0 through 3,
the respondents now answer 0 through 4.
Again, these people are not being asked whether they support Putin or any of the other leaders.
They are just being asked a number.
And yet the pollster can combine these two surveys and obtain the implied level of support
for Putin.
Imagine that the average number of the survey group without Putin was 1.82.
Meanwhile, the average number for the survey group with Putin was 2.35.
The difference between those two numbers is .53.
Because the only difference between the surveys is the addition of Putin,
this implies that an estimated 53% of people support him.
Back in 2015, a team of political scientists used this type of list experiment to test
whether Putin’s popularity was real or hopelessly inflated.
When directly asking respondents about Putin’s popularity, they found that 86% of Russians
supported him.
In the list experiment, the average number of politicians supported without Putin was
1.18.
With Putin on the list, the average number of politicians supported was 1.98.
The difference between those two numbers was .79.
Thus, the estimated portion of people who supported Putin was 79%.
Comparing the direct versus indirect measurements, the difference is 7%.
This suggests that respondents only engaged in a small amount of preference falsification.
In fact, the confidence interval for the preference falsification estimate ranged from -3% to
17%, meaning that it is plausible that there wasn’t any falsification at all.
In short, Russians appeared to genuinely support Putin.
But these findings come with several important caveats.
To start, this experiment was conducted back in 2015, which is not to 2022.
Whether the 81.6% today is real depends on whether anything has changed over time.
And something might be changing.
The incentive to falsify preferences depends on how much security forces are working.
The more there are, the less you’d want to admit anti-government sentiments.
With a war going on, it is reasonable to think that domestic surveillance has increased.
Thus, the lack of preference falsification in 2015 might not reflect what is happening
in polls from 2022.
Another problem is that the list can implicitly reveal some individuals’ preferences.
Imagine you do not support any of the four politicians’ policies.
If you answered honestly, then the pollster would know for sure that you do not support
Putin.
Likewise, if you supported all four politicians’ policies
And you answered honestly when asked, the pollster would infer that you did support
Putin.
Thus, if preference falsification were a concern and respondents thought about the problem
strategically, we would see fewer 0s and more 4s than we should.
Fortunately, there are diagnostics that researchers can run to check for that issue, and it appeared
not to be happening in this survey.
Another issue is that the list itself changes the framing.
When a respondent is asked whether they support Putin, they are only thinking about Putin.
But imagine that the respondent is asked about support for Joseph Biden, Boris Johnson, and
Angela Merkel, with Putin still along for the ride.
This frames the question as an east-versus-west problem.
Even someone who does not particularly like Putin may still respond with a 1 if they hate
western countries.
It’s not obvious how the framing might taint things when the comparisons are Stalin, Brezhnev,
and Yeltsin.
However, the list experiment was also repeated with three contemporary Russian politicians,
and the results were basically identical.
The final issue is that a list experiment requires the respondent to pay attention for
longer.
It’s not a simple yes-or-no question.
If respondents experience some level of fatigue with the longer question, it could artificially
change the support in the experiment.
To test for this, the researchers did a separate survey.
First, they directly asked Russians about their support for Fidel Castro.
They then ran a list experiment, adding Alexander Lukashenko, Angela Merkel, and Nelson Mandela
to the list.
In the direct survey, 60% supported Castro.
In the list experiment, 51% did.
That’s a 9% difference in a situation where there shouldn’t be any reason to misrepresent
one’s true preferences.
Getting back to the original experiment, it’s possible that the 7% difference between direct
and indirect surveys for Putin are not the result of people lying in the former case.
It’s just mental fatigue.
If so, that would indicate that Putin is just as popular as the standard surveys indicate.
This ended up being a very important diagnostic.
In 2020, the researchers reran the experiment multiple times.
They found similar results then as well.
For example, this time around, one of the direct surveys showed a 63% approval rating
for Putin.
In contrast, the list experiment only showed 40%.
This would suggest that 23% of individuals falsified their preferences.
Following one of those earlier concerns, it might appear that circumstances changed on
the ground, causing more respondents to lie.
But in the Castro diagnostic, the direct survey showed 56% approval.
The list experiment estimated a 34% approval.
The 22 percentage point difference is essentially identical to Putin’s 23.
This suggests that the respondents have just become lazier, not that they are feigning
support for Putin.
The key takeaway here is that Putin is still probably very popular among Russians, despite
the apparent issues with pretending to be satisfied with Putin’s performance.
What do you think Putin’s true approval rating is?
Let me know in the comments.
And if you enjoyed this video, please like, share, and subscribe, and I will see you next
time.
Take care.