What AI evaluation work asks of academics and researchers

Academic roles ask for research judgement. You write a hard problem from your field, then grade a model’s attempt at it. The sorting below, by task and by risk, uses the postings and each platform’s own pages, read 13 October 2026.

Take the slide-deck project out before you count

Of the 78 open roles filed under researchers and academics, 40 are Ethos postings. Our Ethos page says most of them describe one project: an AI lab training a model on professional slide decks, posted once for each profession. One Ethos profile covers all of them, so they are one opportunity, not 40.

The other 38 roles come from micro1 (12), SME Careers (12), Mercor (11), AfterQuery (1), Turing (1) and Sovrano AI (1). The rest of this page counts those unless it says otherwise.

Seven tasks, from most research to least

• Solve, audit, adjudicate. micro1’s physics roles for PhDs and postdocs ask for derivations in LaTeX, checked with SymPy, Python or Jupyter. One posting assigns people as Solver, Auditor or Adjudicator by subfield.
• Write the problem and the rubric. SME Careers’ STEM and technical rubric-design role asks for questions that test reasoning, then rubrics with required concepts, accepted alternatives, critical errors and partial credit.
• Referee the model. Mercor’s science listing says graders judge whether the method suits the question and whether the statistics support the claim. They also flag tidy results that would fail peer review.
• Lead quality. SME’s Quality Assurance Lead roles review trainers’ work and the model’s, write feedback and help onboard new contributors.
• Forecast as a practitioner. Mercor’s central bank, elections and company analysis roles want people who did the job, such as a former adviser from the relevant central bank.
• Make the slides. Ethos wants a PhD with three or more years of active research and a publication record. The work is conference decks, research posters, grant figures and lab-meeting slides.

The disciplines on offer

SME Careers has the widest spread. Its Quality Assurance Lead roles cover economics, anthropology, sociology, political science, geology, astronomy, philosophy, psychology, history and cultural studies. Mercor lists a humanities pool and a life, physical and social science pool.

micro1 uses language as the credential. It lists PhD psychologists who are bilingual in Japanese, Vietnamese or Cantonese, and one who speaks Georgian. PhD appears in the title of 11 postings here.

Mercor’s science pool cares less about where you trained. It asks whether you designed the studies and interpreted the data yourself, recently enough to explain your calls.

What the postings pay

The 36 hourly roles outside Ethos advertise a median ceiling of $90 an hour, an upper quartile of $200 and a top of $250. Across the whole site the median ceiling is $80.

Another 2 roles pay per task, and this site does not convert a task fee into an hourly rate.

SME’s figures are ceilings that differ by country, so read every one as “up to”. Ethos posts a flat band of $70–100 an hour.

Hours: which roles fit around a post

micro1’s physics role for PhDs and postdocs states 10 to 15 hours a week or more, with flexible scheduling. Ethos states 5 to 20 hours a week. Mercor’s talent-network listings describe typical commitments of 15 to 30 hours a week.

This site does not list roles that demand a minimum of 40 hours a week, so none appear here.

Your university has a say

The postings are silent on the legal side. None of the 38 descriptions outside Ethos mention intellectual property, employer approval or confidentiality terms. The platforms’ contracts hold those, so read them first.

Then settle three things with the people who can answer them:

• Outside work. Your employment contract may require approval for paid work elsewhere. Your head of department or HR can say.
• Intellectual property. Whether a problem built from your own research belongs partly to your institution or funder depends on its policy. The postings do not say.
• Material held in confidence. Unpublished results, a manuscript you are refereeing and a grant you are reviewing are not yours to paste into a task. Mercor says passing confidential material outside a project is a policy violation.

Mercor’s application asks you to verify work authorisation, so a visa that limits outside work is a question for your international office. Mercor’s Nonproliferation Analyst and Sovrano’s CBRN review roles work with requests on weapons-relevant topics, so raise those with your employer first.

Country limits

Of the 38 roles, 10 are limited to the US by their location field or their own text. micro1’s physics role for PhDs and postdocs lists “Remote (Worldwide)”. SME pays a different maximum in each country, so check the country list on each posting. Working from outside the US covers how to read it.

Which to try first

• PhD in physics, mathematics or chemistry, with weeks free: micro1’s physics roles. The physicists guide goes deeper on physics.
• Social scientist or humanist: the Mercor pools and SME’s discipline roles. Both are standing pools, so expect a gap before any project. Read Mercor’s interview guide and SME’s.
• A short, flexible commitment: one Ethos profile, if your slide craft is strong.

What the postings do not say

• Whether a pool leads to work. 14 of the 38 roles outside Ethos call themselves a standing pool or say there is no immediate project. Mercor says a match can come within a week or after several months.
• How much a publication record counts. Only some postings ask for one, and none that we read says how it is weighed.
• Hours after you are matched, and why people are turned down. No platform publishes either.

This site is not affiliated with any platform named here, and passing a screening is not a promise of work.