About this role
Join a dedicated AI evaluation team to build and refine benchmarks for cutting-edge AI agents. You will design and implement evaluations for specific domains, helping us understand agent performance on real-world tasks. This role involves building scalable infrastructure, developing metrics, and collaborating with experts to ensure our benchmarks are rigorous and relevant.
What we are looking for 6
- Design and implement domain-specific AI agent benchmarks
- Build scalable infrastructure for running evaluations
- Develop metrics and analyze benchmark performance
- Collaborate with subject-matter experts on real-world workflows
- 2+ years experience in AI benchmarks or evaluation infrastructure
- Proficiency in Python, Docker, and Linux