Promptbench
A micro-consulting platform that lets philosophy and ethics grads earn money doing structured AI red-team and evaluation sessions for labs that need human judgment, not more compute.
Philosophy, ethics, and rhetoric grads who want flexible paid work evaluating AI model behavior
- Curated task feed: short paid tasks ($15-80 each) like 'find edge cases in this chatbot's medical advice' or 'rank these 20 model outputs by reasoning quality'
- Built-in training: teaches non-engineers how to write rigorous evals without needing to code
- Reputation profile that lets you graduate to higher-paying longitudinal studies at labs
- Payout via Stripe within 48 hours, no employer in the middle
Anthropic, OpenAI, and Google DeepMind have all publicly scaled human evaluation teams in 2025-26 and the HN essay's central claim is that judgment, not code, is the scarce input.
Anthropic, OpenAI and DeepMind have publicly hired philosophers into core safety/ethics teams; Business Insider reports six-figure packages, and Crossing Hurdles is posting AI Red-Teamer roles at $54–111/hr with 10–40 hr/week commitment.AI Labs Recruit Philosophy Majors for Ethics Roles ↗Crossing Hurdles hiring AI Red-Teamer | Remote in Canada | LinkedIn ↗
Space is more crowded than the idea implies: Scale AI, Surge AI, Bits AI own RLHF/eval data; Crossing Hurdles is already a direct niche competitor doing exactly philosophy/eval micro-contracting; Mercor and Turing operate adjacent talent funnels.The Hidden Giants of AI: Comparing Scale AI, Bits AI, and Surge ↗Scale AI Alternatives for Enterprise AI Teams ↗AI Red Teamer Salary 2026: Rates, Pay Bands & Hiring ↗
Willingness to pay is clearly demonstrated: contractors bill $54–111/hr, FTE bands $80k–220k, and labs are paying six-figure packages specifically for philosophy/judgment roles rather than cheaper crowd labor.Crossing Hurdles hiring AI Red-Teamer | Remote in Canada | LinkedIn ↗AI Red Teamer Salary 2026: Rates, Pay Bands & Hiring ↗
The judgment-not-compute thesis is being validated structurally: Anthropic and OpenAI ran a public cross-lab alignment evaluation exercise in 2025, and philosophy hires are being embedded in core research rather than advisory boards.Findings from a Pilot Anthropic—OpenAI Alignment Evaluation Exercise ↗OpenAI, Anthropic, and DeepMind Are Hiring Philosophers ↗
A two-sided marketplace is buildable but hard: requires vetting a credentialed philosophy pool, securing enterprise contracts against entrenched vendors (Scale, Surge, Crossing Hurdles), and building scheduling/quality infrastructure — meaningful GTM lift, not trivial.Top 19 AI Red Teaming Tools (2026) ↗