Patrick Murray, Director of the Monmouth University Polling Institute, joined the New Jersey Society of Certified Public Accountants (NJCPA)’s IssuesWatch Podcast to discuss the sampling process behind political polling.
Polling raises credulity, Murray explained. “How could you interview 500-800 thousand people and get something that is pretty close to what millions of people think? The first step is sampling.”
There’s a mathematical property that says if you take a truly random sample of a small population, of a very large population, you can get results within a certain margin, Murray said.
The first step is to ensure a representative sample that’s pulled from people who live in a particular state, all the voters in the country or whatever happens to be the studied population.
“There are all different types of populations and as long as you approach it with the sampling prospect of ‘I’m going to get a truly random sample and do the best job that I can to make that representative,’ that’s your first step,” Murray said.
The second step of polling is how people are contacted and interacted with. For many years, the go-to method was by telephone, Murray said. “It still continues to be kind of the gold standard method, but we’re moving into other methods.”
As caller ID and cell phones became more popular, participation numbers began to drop.
“People started getting more and more annoyance calls,” Murray said. “They stopped picking up the phone unless they recognized the number, so it just takes a lot more work to get somebody on the phone.”
A drop in phone-polling has not affected accuracy, Murray explained. A representative sample can still be obtained, but extra work must be done to reach people over the telephone who are willing to speak. New samples include cell phones in addition to landlines.
“Most of the calls we make nowadays are to cell phones,” Murray said. “There’s a few holdouts, particularly here in New Jersey, in terms of cutting the cord.”
Other rising methods are online. “We’ve done a lot of online work where you take a sample and try to match it to an email. We take a list of registered voters, and if it matches email addresses, you reach out to them there.”
The difference between online polling and telephone polling is the different ways you can ask and present questions, Murray said. “In online polling, you can actually show images and so forth but you don’t always get the nuances or immediacy. [Over telephone] people are kind of being forced into making a choice, so there’s pros and cons on both sides.”
It has become so easy to conduct online polls that many are low-quality and low-effort, Murray explained. “There are so many bad ones out there that don’t do all the rigorous stuff. A lot of that data you have to worry about, and unfortunately there’s more and more of it out there so we don’t know what to trust.”
The final part of doing a good poll is how you ask the questions, Murray said. You do not ask questions the way pundits or academics think about them, you ask questions the way that people on the street think about them.
“While I was a Political Science graduate student, my interest was really in how people think,” Murray said. “I take that approach when I look at how people ask questions.”
“One of the things I used to do a lot when I first started out was go to diners,” Murray said. “That’s where people talk about politics, and just kind of eavesdrop on what the conversations are, what are they thinking of and how are people talking about these things.”
It is not always as simple as approval or disapproval of an issue, as sometimes there may be a third option somewhere that academics or the media does not consider. “They always want something in black and white, and sometimes there’s a nuance there that you only catch if you actually listen to how people talk about these things.”
