AIR USE NOTICE. This document contains copyrighted material whose use has not been
specifically authorized by the copyright owner. The CHANCE project is making this
material available as part of our mission to promote critical thinking about statistical
issues. We believe that this constitutes a 'fair use' of the copyrighted material
as provided for in section 107 of the US Copyright Law. If you wish to use this
copyrighted material for purposes of your own that go beyond 'fair use', you must
obtain permission from the copyright owner.
Copyright 1996 The Christian Science Publishing Society
The Christian Science Monitor
February 20, 1996, Tuesday
SECTION: MEDIA; Pg. 15
LENGTH: 1044 words
HEADLINE: Why Polls Are So 'Scientific' and So Confusing
BYLINE: By Alexandra Marks, Staff writer of The Christian Science Monitor
DATELINE: NEW YORK
HIGHLIGHT: Problems: what you ask, how you ask it, voters who haven't made up their
minds, and margins of error
BODY: THE conflicting poll results coming out of New Hampshire all week were
enough to boggle any voter's mind. In one poll, Republican presidential candidate
Bob Dole was up by 10 points. In another, he was in a dead heat with Pat Buchanan.
In yet a third, the Senate majority leader had slipped to second place. And all within
''What we have ended up with,'' says pollster Peter Hart, head of Peter Hart
Research, ''is a multitude of tracking polls that tell us irregular heartbeats with
Polls are designed to take a snapshot in time, to relay the pulse of the country
on key issues or a candidate's standing. They have become indispensable tools for
political strategists and an integral part of political reporting, driving the headlines and pointing the spotlight where the media's attention will go next. Critics charge
that polls have pushed substance off the front pages and turned the nation's leaders
Polls' proliferation and their sometimes contradictory results have also led
to a growing public distrust of them.
''People say, 'It depends on the question; you can get anything you want out
of a poll,' so they tend to discount it,'' says Tom Kiley, a pollster with Martilla
and Kiley, a Boston-based media-consulting firm. ''It's part of the larger antipolitical mood.''
Polling has, in fact, become much more exact since it burst onto America's political
landscape in 1936. That year, George Gallup bragged that, using mathematical probability,
he could predict the outcome of the national election by interviewing only 1,000 people. He forecast a landslide by incumbent Franklin Roosevelt. The pundits
thought he was crazy, in part because an unscientific Literary Digest straw poll
several months before had predicted a huge win by Republican Alf Landon.
Gallup proved them wrong.
''It was the object lesson of all object lessons about probability sampling,''
Mr. Kiley says. ''And Gallup has gone on to do polling ever since and typically is
within a point of the election.''
Pollsters constantly work to find the most exact methods to gauge the public's
pulse. They use computers to analyze census and other data to ensure their samples
reflect a community's demographics. They use ''random digit dial'' to ensure they
include people with unlisted phone numbers and those who recently moved. They try to be sure
the wording of their questions is as neutral and precise as possible.
''Small differences in wording can make a difference,'' says Andrew Kohut, director
of the Washington-based Pew Research Center for the People and the Press.
A striking example was this fall, immediately after President Clinton announced
he was committing American troops to Bosnia. The ABC/Washington Post and CBS/New
York Times polls both found the public dubious, but a Gallup poll found a much higher
approval rating. The difference: The network polls' question included the fact that 20,000
American troops were going to Bosnia; Gallup's did not.
''In areas where public opinion is not crystallized, wording can mean a great
deal,'' Mr. Kohut says. ''But in some areas where the public opinion is strong,
no matter how you ask a question, you get roughly the same answer.''
This fall, House Speaker Newt Gingrich (R) accused the CBS/New York Times poll
of asking an unfair question about how people feel about cutting Medicare in order
to help balance the federal budget. The poll found it was an unpopular idea.
Mr. Gingrich charged that if the pollsters had asked about cutting the ''rate
of increase'' in Medicare, the outcome would have been very different.
''In point of fact, our polls used exactly the wording that Gingrich had suggested,''
Kohut says, ''and we found exactly what The New York Times found.'' Unethical
or inexperienced pollsters can sway the results of a poll, which is why professional pollsters warn consumers to look skeptically at the source of any poll and the kinds
of questions asked.
That's particularly true in New Hampshire where, with the exception of the few
weeks leading up to the November election, there is more intense polling and attention
to their results than at any other time during the presidential campaign.
All the major candidates have pollsters, as do all the major media outlets. Candidates
usually hold their results close to their vests (unless their polls show a sudden
favorable surge). But there are still enough press polls to feed the confusion.
Gallup, which works for CNN, is one of five firms that conducted tracking polls
for the media in New Hampshire last week. (Tracking polls survey fewer people over
a several-day period and report an average.)
Gallup had Dole way ahead of his challengers immediately after the Iowa caucuses.
That was markedly different from three other tracking polls that had him in a dead
heat with Buchanan.
''The truth is, polls have margins of error, and margins of error are not a joke:
This is all mathematical,'' says Gerry Chervinsky, president of KRC Research, which
polls for The Boston Globe and WBZ-TV. His polls supported Gallup's findings.
Mr. Chervinsky says the results could be due to the wording of the questions,
the time the surveys were done, and the number of independent voters included. But
he also points out that if you take a 5 percent margin of error into account, all
the polls paint roughly the same picture.
''If you look at the [Boston Herald] numbers, where Dole is at 23 percent, that's
within the margin of error where I have Dole at 27,'' he says. ''Then you have Buchanan
at 22 - that's within the margin of error where I have him at 17.''
The conflicting results also tell a straightforward story about voter volatility
and a lack of passion for the candidates who, pollsters say, have alienated the public
with their negative advertising. ''There are no issues being talked about by these
candidates, and polling can't create that,'' says R. Kelly Myers, director of the UNH
Survey Center, which is polling for the Boston Herald and WCVB-TV.
Mr. Myers and others suggest the candidates and the media focus on a different
set of poll results, such as the ones that show voters are fed up with negative ads
and the relentless focus on the horse race.