UPSETRESEARCHContact

Areas of Expertise

Research Method

How the work is actually done, start to finish, and a plain account of what it cannot tell you.

Start a scope

Start to finish

The nine steps

Nothing here is proprietary. A campaign that knows how its research was produced can judge whether a finding deserves money behind it.

  1. Scope in writing. The office, the election date, the jurisdiction, the questions that matter to the candidate, the budget, and what arrives each month.
  2. The arithmetic. Vote goal built from the last three comparable elections, turnout ranges, and the number of votes that has to be found.
  3. The file. Voter file, registration and vote history, precinct returns and census geography, assembled and checked for the obvious errors before anything is scored.
  4. The model. Likelihood of voting and, where the data supports one, likelihood of being reachable, plus a precinct ranking by margin and slack.
  5. Recruitment. Respondents are invited where the district's voters already are: text, email, the campaign's own page, social posts, a code on literature, phone follow-up where a number is good.
  6. Fieldwork in waves. Short instruments, a handful of fixed questions and at least two open ones, run repeatedly on the same wording so that change over time is visible.
  7. Reading the open answers. Every open answer is read by a person and coded by hand. Phrases are kept verbatim rather than paraphrased into categories.
  8. The memo. Findings, the limit of the study on the front page in the same type size as the findings, and a recommended action with cost and deadline for each finding.
  9. Tools and repeat. Whatever the campaign is doing by hand gets built, and the next wave tests the decisions made from the last one.

Limits

What this method cannot tell you

These limits apply to every engagement, and they are stated before the work starts.

  • Respondents choose themselves. There is no probability sample here. That means no margin of error, no weighting scheme that repairs the problem, and no sentence of the form "X percent of voters in this district think Y."
  • The recruitment channel shapes who answers. Text and email reach people whose contact information is current. Online recruitment underrepresents voters who are rarely online, who in municipal races are often among the most reliable voters in the district.
  • Counts are small in small races. A school board district may produce a few dozen responses in a wave. That can carry direction and language. It cannot carry a tenth of a percentage point.
  • The model is only as good as the file. Bad addresses, recent moves and duplicate records pass straight through into the scores.
  • Early reads go stale. A finding from February describes February, and it gets retested before anyone quotes it in October.
  • No result projection. This method does not call the race, and no number from it should be published as though it did.

Where a question truly needs a probability sample, such as a number a campaign intends to publish as a measure of district opinion, the correct answer is a poll from a firm that runs them. You will get that recommendation.

What holds across every engagement

The parts that do not change from race to race

  • The same questions asked the same way wave over wave, so a change in the answers is a change in opinion rather than a change in wording.
  • Open answers quoted rather than summarized away.
  • Respondent counts, recruitment channels and field dates on every number, dates written as MM/DD/YYYY.
  • Hand coding reviewed a second time when a result looks surprising, before it reaches a memo.
  • Raw responses and the scored file handed to the campaign at the end of the engagement.

The method is deliberately plain. It is built so that a campaign manager with no research background can audit it, and so that a finding can be traced back from a recommendation to the question wording that produced it.

Ask how this would run in your race

Send the office, the election date and what you have set aside for research. We write back with a scope, a price and the questions worth asking in the first wave.

Contact Upset Research