A salary survey hands you a number and a feeling of safety. The number is someone else’s average, and the safety is borrowed. When a leader prices a role off open-source salary data, the move looks like diligence. Usually it is avoidance, a way to skip forming a judgment about the person in front of you. The survey cannot see your project, your margin, your mission, or the cost of losing the person you already have. You can.

Data has its place. The mistake is treating a stranger’s blended average as the price of a specific person on a specific project. Read it as a floor that informs your thinking, never a verdict you hand your judgment to.

Eighty ways the number lies

I sat down and counted the ways open-source salary data misleads a hiring decision. I stopped at eighty, sorted into ten failure modes. No single line is the argument. The stack is. Hover any reason to see why it fails.

80 reasons salary survey data is unreliable, grouped into 10 failure modes. Each reason lists a one-line explanation.
Why “the market rate” can’t be trusted
Ten failure modes. Eighty specific ways a salary survey breaks. The point is not any one flaw. It is the stack.
80 reasons · 10 failure modes hover a reason to see why
01 Bad or unknown sample quality 10
1Self-selection 2Nonresponse bias 3Small samples 4Unclear sampling frame 5Big firms overrepresented 6Small firms underrepresented 7Industry imbalance 8Geographic imbalance 9Survivorship bias 10Incumbents only
02 Poor job matching 8
11Inconsistent titles 12Vague levels 13Different scope 14Structure shifts the role 15Project type 16Project size 17Role-specific scarcity 18Hybrid roles
03 Hard to measure cleanly 9
19Base salary only 20Target vs actual bonus 21One-time payments 22Equity hard to value 23Benefits vary 24Hours ignored 25Travel ignored 26Union / prevailing wage 27Cost-of-living oversimplified
04 Usually stale 5
28Market outpaces the survey 29Inflation drift 30Conditions shift fast 31Bands lag real offers 32Collection timing
05 Misleading summaries 13
33Skewed distributions 34Mean misleads 35Unstable percentiles 36Outliers 37Top-coding 38Aggregation hides variance 39Ranges too wide 40No confidence intervals 41No standard deviation 42Opaque weighting 43Duplicate records 44Non-independent data 45Cluster effects
06 Opaque methodology 7
46Unclear definitions 47Unknown cleaning rules 48Vendor incentives 49Weak verification 50Strategic employer data 51Noisy employee data 52Biased public sources
07 Market is more dynamic 8
53Incumbents are not candidates 54Passive premium 55Counteroffers 56Urgency pricing 57Employer brand 58Risk premium 59Career upside 60Local reputation
08 Comparisons are invalid 10
61Apples to oranges 62Internal-equity clash 63Unreal benchmark roles 64One “market rate” 65Correlation is not causation 66Simpson’s paradox 67Ecological fallacy 68Accepted-offers bias 69Rejected data missing 70Performance uncontrolled
09 Privacy weakens precision 4
71Reporting thresholds 72Roles combined 73High earners masked 74Niche roles vanish
10 Practical misuse 6
75Used as a ceiling 76Cherry-picked numbers 77False precision 78Divorced from strategy 79Old data becomes truth 80Answers the wrong question
80
The data answers the wrong question. A survey tells you what people are being paid. Hiring asks what it will take to attract, close, and keep the right person now. That answer never lived in the spreadsheet. It lives in your mission and your P&L.

Price from your own numbers instead

Once you accept that the survey cannot price your role, the question changes. What is this person worth to you, on this project, against this risk, toward your mission? That answer lives in your own P&L and your own values, not in an aggregate a stranger compiled for a company you have never seen.

You already own the numbers and the mission that should price the role. The survey never did.