Ask Portland
Methodology

The method is the product.

A public survey is only worth something if the people who disagree with the result still believe the number. That belief has to be earned with method, not asserted with a logo. So the approach below is published, applies to every Ask Portland survey, and we lead with the findings that survive scrutiny.

What we promise, every time

  • Raw and weighted, side by side. You always see what we literally collected and the estimate after correcting for who showed up. Neither is ever shown as if it were the other.
  • Sample, channels, and skew named. Every release states its size, where respondents came from, and how it differs from the population.
  • Neutral instrument. Questions are worded to read the same to people who disagree, and opinion questions come last so they can't color the rest.
  • Open method. The questions, the weighting, and how results are computed are all public — anyone can check our work.
  • Email stays separate. Leaving an email for results is optional, and it's stored apart from responses — never linked to your answers.

Reaching the whole city (not just the loud part)

Most civic input comes from the small number of people who show up to meetings or already follow an issue. We go wider on purpose. We distribute each survey through many channels at once — advocacy lists and social media reach the young and online; neighborhood associations, senior centers, faith networks, libraries, and culturally-specific community organizations reach the people most surveys miss. Every response is tagged with the channel it came through, so the mix is auditable, and partner organizations receive their community's results first, free, to use as they see fit. Diversifying channels is how the sample gets more representative over time.

How we weight to the population

People who opt into a survey are never a perfect mirror of the city — they skew by age, income, geography, and whether they own or rent. We correct for that with raking (iterative proportional fitting): every response is given a weight so that the sample's mix on each benchmark dimension matches the real population. We rake all dimensions together, because they're correlated, and we trim extreme weights so one thin cell can't dominate. Alongside the result we report the effective sample size — the “real” N after weighting.

Weighting reduces bias; it doesn't erase it. Until a given survey's benchmarks are set to exact local figures, we flag its weighted shares as directional — and we always show the raw sample beside the weighted estimate so you can judge for yourself.

How we group respondents

Some surveys also sort respondents into segments — for example, by their situation or their goals — so a result isn't just one citywide average that hides real differences. Where we do this, the rules are transparent and priority-ordered, never a black box: when someone fits more than one segment we record every match so overlaps stay visible, and each survey's segment definitions are published alongside it. Segments are used to understand the population in aggregate; we don't label individual respondents back to themselves.

Keeping responses honest

Because surveys are anonymous and deliberately low-friction, we protect data quality with light, layered checks rather than logins: a hidden trap field only automated bots fill, a minimum time-to-complete, and limits on repeat submissions from one browser or floods from one network — all tuned so they never block legitimate responses from shared computers at a library or community organization. Every response also carries anonymous signals (a hashed browser id, completion time, the channel it came through) so duplicates and anomalies can be removed before results are finalized. No check is perfect on an open survey; the aim is to make corruption costly and detectable, and to clean the data transparently.

The honest limits

  • These are opt-in samples, not probability samples. Weighting reduces bias; it doesn't eliminate it.
  • Weighting corrects demographics, not attitudes — so read opinion splits as ranges, with opposition a floor and support a ceiling.
  • The most trustworthy findings are the ones that hold even in a favorable sample. We lead with those.
  • Until exact local benchmarks are wired in for a survey, treat its weighted shares as directional.