AI Agent Evaluator

Location: U.S based

##

Opportunity Snapshot

Pay: $25-45/hour

Time per task: ~1.5 hours per run (evidence and scoring included)

Project kickoff: Monday, October 12

Onboarding: Today (Fri), Saturday, or Sunday, so you’re ready to start Monday October 10th

You’ll need: An active personal popular AI account (Free, Go, or Plus; not Pro), an active personal and a smartphone that can screen record

Your tasks: Run everyday errands with two AI agents, approve key steps, screen record, and score the results

About the role

##

AI agents are starting to handle real-life errands: paying bills, booking trips, doing the groceries. We want to know how well they actually do it, from the point of view of a real user. That’s you.

You’ll test two AI agentsside by side, using your own accounts on your phone, and tell us what worked, what didn’t, and how it felt.

##

What you’ll do

• Run real-life scenarios: bills, returns, email follow-ups, bookings, shopping, groceries, travel, family logistics, outings, and forms
• Approve or stop the agent at key steps
• Screen record every attempt
• Score each run on completion, quality, control issues, satisfaction, and time
• Share clear written feedback on the agent experience

Who we’re looking for

• US-based , with a Bachelor’s degree
• Someone who already uses AI in daily life
• 2-3 years of experience, preferably in engineering, business management, or office/operations
• Detail-oriented, great at following instructions and documenting each step
• Clear written communication
• Generalists welcome, domain experts preferred
• No coding or AI background needed

##

Why you’ll like it

• Get paid to use the AI tools you already use
• Work entirely from your phone, on your schedule
• Have a direct say in how next-generation AI agents get built and improved

##

How the selection process works

• Apply
• Complete the screening questions
• Interview : 15-30 minutes max
• Profile review: if everything looks good, we present you to the client
• Onboarding: our Talent Ops team will contact you to get you started

Where you can work

This role is 100% remote, for applicants located in United States.