Adventures in Campaign Message Polling, part 1

The political blogosphere recently took note of a poll in New Jersey’s 3rd Congressional district purporting to show first-term Democratic incumbent John Adler with a surprising 17 point lead over GOP challenger Jon Runyan.  The poll was conducted for the Adler campaign by their own pollster, but even campaign pollsters have to produce reliable estimates if they want to stay in business.  Regardless, this poll – or to be more accurate, the memo that described the poll results – raises some red flags.

First, only 3% of likely voters say they are undecided about their choice.  Really?  In June, when most voters probably cannot name either party’s nominee?  Second, the vote choice question was posed as a 3-way race, including Adler, Runyan and a third candidate running under the “NJ Tea Party” banner.  This candidate – whose name recognition has to be near zero – received 12% of the vote in this match-up.

The poll memo reads more like a campaign fluff piece (e.g. “[Adler’s] record of independence and accountability has put him in an excellent position to win this race.”) than an insightful polling memo.  Now, I’m not saying that the poll findings were fabricated.  For one, the pollster, Greenberg Quinlan Rosner, is a well-known Democratic firm that is unlikely to jeopardize their reputation by generating poll numbers out of whole cloth.  In fact, I’m inclined to believe the results were probably valid.  The problem is that they were reported out of context.

Unless the Adler campaign has money to burn on conducting polls solely for the purpose of leaking them to the public, these results were from a longer poll that also tested all sorts of messages and strategies for the Adler campaign.  The purpose of an early summer poll is to try out a variety of messages in order to identify the most effective ones for use in the campaign.

What we don’t know about this poll is at what point in the interview this three-way vote choice question was presented to respondents.  I have my doubts that this was the first time in the interview that survey respondents were asked to name their vote choice.  There were likely some questions about candidate characteristics that preceded this question.

So why does the order of the questions matter?  Because questions asked later in a poll allow respondents to use information they heard during the course of the interview to inform their answers.  At this point, the poll results no longer reflect the mind-set of typical voters because the poll respondents now have information – i.e. messages – that most voters don’t.

This is why message testing polls rarely get released to the public.  Indeed, most reputable pollsters prefer it that way.  Unfortunately, their concerns are occasionally overridden by a campaign manager who sees some strategic advantage in releasing the poll results.

One reason a campaign may release an internal poll is to demonstrate to potential donors that they have a viable shot at winning.  Considering the healthy state of Adler’s campaign coffers, that’s clearly not a concern.  So what advantage did the Adler campaign see in selectively releasing poll results?

First, we need to consider why the campaign even bothered to include an unknown, unfunded third party candidate in one of their vote choice questions.  Especially since there will be three independents candidates on the ballot for this race in November.  [Side note:  There were four, but the state Democrats’ executive director, Robert Asaro-Angelo, successfully challenged Robert "Weedman” Forchion’s petition.  This, of course, raises questions about Angelo’s contention that he never heard of the Tea Party candidate listed on that very same ballot.]  [UPDATE: Rob Angelo contacted me re this statement. He admitted that he misspoke, since he obviously reviewed the names of all independent candidate filings in June.]

So, why did the Adler campaign only test the Tea Party candidate?  Because it makes sense in the current political environment.  While we can make a pretty strong guess as to the “Libertarian” candidate’s likely vote total in November, the impact of running under the Tea Party banner is a big question mark.

According to the Adler poll results, a Tea Party candidate may indeed peel off votes from the Republican nominee.  But this is by no means a certainty.  Why?  To start, no one knows who the Tea Party candidate is.

And that gets us to why these numbers were released.  Remember, these poll results represent one potential outcome in a context where Adler’s pollster had complete control over information presented to voters.  In other words, the message testing effects measured in campaign polls do not always play out so neatly in the real world.

More importantly, a message will certainly not work if no one knows about it.  And that is the case with the Tea Party candidate.  So, what’s a good way to get a candidate’s name out?  Show him exceeding expectations in a poll.

The message testing poll becomes the message!  The Greenberg firm issued a memo to “Interested Parties” and sure enough, the story hits the internet, including The Hill, Chris Cillizza’s “The Fix” column in the Washington Post, the National Journal’s Hotline and PolitickerNJ.

Interestingly, the poll memo was not released to newspapers, at least not to those in Adler’s district.  Did the campaign think these media outlets wouldn’t be “interested?”  Doubtful.  The reason why the poll was released only to internet sites geared to the chattering classes was a strategic one.  The intent was to let Tea Party-inclined voters “know” they have a viable option in New Jersey’s 3rd district Congressional race and to suggest to potential GOP donors that Runyan is a shaky investment.  The internet is the best way to get that buzz spread with a less critical eye, especially with the burgeoning Tea Party community.

The chosen “interested parties” did their job and disseminated the campaign’s message, any caveats in their reports notwithstanding.  Of course, this may backfire in the long run.  Tea Party activists have been denouncing the candidate, with stories now focused on whether the Tea Party candidate is a plant.  This poll is seen by some as part of a larger Democratic plot.

However, the question remains whether this poll – or more accurately the selective results in the campaign’s memo – should have been reported in the first place.  When a campaign simply claims that their candidate is ahead by 17 points, no journalist in his right mind would report it.  However, when a campaign has their pollster slap together a memo that purports to show a “51 to 34 percent” lead, suddenly the information is valid.

A good rule of thumb, no poll should be reported – in any venue – unless the pollster is willing to provide the entire set of questions and responses.  Otherwise, it’s little more than propaganda, or worse.  It’s a little too late for this poll.  The Adler campaign achieved its intent – getting out a campaign message under the guise of hard fact.

"Sit down and shut up!"
—Gov. Chris Christie, to a Hurricane Sandy heckler