About this role
Join an early-stage AI evaluation company as a Research Engineer, Benchmarks, to design and implement rigorous evaluations for frontier AI agents. You will translate real-world workflows into realistic tasks, build scalable infrastructure, and develop metrics to assess agent performance. This role is ideal for someone passionate about understanding and improving AI capabilities through robust benchmarking.
What we are looking for 6
- Design and implement AI agent benchmarks for domain-specific tasks
- Build scalable infrastructure for running and evaluating AI agents
- Develop metrics and analyses for benchmark reliability and performance
- Collaborate with subject-matter experts to define evaluation criteria
- 2+ years of experience in AI benchmarking or evaluation infrastructure
- Proficiency in Python, Docker, and Linux