The practical move after Mercor's new benchmark map: run a short pilot on your own documents before trusting any AI leaderboard score. 90% of US working hours happen on tasks that current AI benchmarks barely cover, so top scores on public tests say little about your real daily work.
Mercor mapped 29 agentic benchmarks onto the O*NET taxonomy of 923 occupations and 18,797 tasks. Only 208 occupations have a task with 5 or more benchmark prompts behind it. Lawyers' AI-exposed tasks are well covered; legal secretaries, court clerks, and most middle-income knowledge jobs are not. Coverage follows the biggest wage bills, not the real texture of daily work.
What this means for small business automation
If a model tops a coding or legal-reasoning benchmark, that tells you almost nothing about whether it can reliably sort your inbox leads, extract line items from supplier invoices, or chase a missing document from a client. Pick your automation targets from your own process, not from Twitter screenshots of benchmark scores. Run a 20-task pilot on your actual data - receipts, tickets, contracts - and measure accuracy and hours saved there. That is the kind of pipeline I build in a week, and it is also how you avoid paying for capability you will never use.
Why it matters: Benchmarks map the shiniest 10% of work. The automation worth building usually lives in the other 90%, and the only way to find it is to test on your own documents.
Source: Hacker News