Field Notes · Compensation

The market rate can’t price the person in front of you.

Leaders reach for a salary survey because it feels responsible. Underneath the number is a stack of problems: bad samples, mismatched jobs, stale data, and statistics that hide more than they show. Here are eighty of them, sorted into ten failure modes. No single line is the argument. The pile is.

80 reasons salary survey data is unreliable, grouped into 10 failure modes. Each reason lists a one-line explanation.
Why “the market rate” can’t be trusted
Ten failure modes. Eighty specific ways a salary survey breaks. The point is not any one flaw. It is the stack.
80 reasons · 10 failure modes hover a reason to see why
01 Bad or unknown sample quality 10
1Self-selection 2Nonresponse bias 3Small samples 4Unclear sampling frame 5Big firms overrepresented 6Small firms underrepresented 7Industry imbalance 8Geographic imbalance 9Survivorship bias 10Incumbents only
02 Poor job matching 8
11Inconsistent titles 12Vague levels 13Different scope 14Structure shifts the role 15Project type 16Project size 17Role-specific scarcity 18Hybrid roles
03 Hard to measure cleanly 9
19Base salary only 20Target vs actual bonus 21One-time payments 22Equity hard to value 23Benefits vary 24Hours ignored 25Travel ignored 26Union / prevailing wage 27Cost-of-living oversimplified
04 Usually stale 5
28Market outpaces the survey 29Inflation drift 30Conditions shift fast 31Bands lag real offers 32Collection timing
05 Misleading summaries 13
33Skewed distributions 34Mean misleads 35Unstable percentiles 36Outliers 37Top-coding 38Aggregation hides variance 39Ranges too wide 40No confidence intervals 41No standard deviation 42Opaque weighting 43Duplicate records 44Non-independent data 45Cluster effects
06 Opaque methodology 7
46Unclear definitions 47Unknown cleaning rules 48Vendor incentives 49Weak verification 50Strategic employer data 51Noisy employee data 52Biased public sources
07 Market is more dynamic 8
53Incumbents are not candidates 54Passive premium 55Counteroffers 56Urgency pricing 57Employer brand 58Risk premium 59Career upside 60Local reputation
08 Comparisons are invalid 10
61Apples to oranges 62Internal-equity clash 63Unreal benchmark roles 64One “market rate” 65Correlation is not causation 66Simpson’s paradox 67Ecological fallacy 68Accepted-offers bias 69Rejected data missing 70Performance uncontrolled
09 Privacy weakens precision 4
71Reporting thresholds 72Roles combined 73High earners masked 74Niche roles vanish
10 Practical misuse 6
75Used as a ceiling 76Cherry-picked numbers 77False precision 78Divorced from strategy 79Old data becomes truth 80Answers the wrong question
80
The data answers the wrong question. A survey tells you what people are being paid. Hiring asks what it will take to attract, close, and keep the right person now. That answer never lived in the spreadsheet. It lives in your mission and your P&L.

Our own tools guard against some of these. No reported figure exists below five people, the public sees one only at ten, and what employers advertise is kept apart from what people report. They cannot fix the rest: a survey still averages different scopes and company sizes, still runs behind the market, and still cannot price one specific person. Read the advertised-pay benchmark and the salary survey with that in mind.

The survey answers what people are being paid. Hiring asks what it will take to attract, close, and keep the right person now. That answer starts with your mission and your P&L, not the market.