Research Engineer, Benchmarks

As a Research Engineer, Benchmarks, you will design and own high-quality benchmarks to evaluate frontier AI agents on realistic, domain-specific workflows. You will work within a small, technical team to ensure evaluations are rigorous and trusted by leading AI labs and customers. This role involves building and operating infrastructure, developing metrics, and validating performance against real-world needs.

• Design, implement, and own internal benchmarks for frontier AI agents
• Partner with subject-matter experts to define realistic workflows and evaluation criteria
• Build and operate infrastructure for running models and agents at scale
• Develop metrics and statistical analyses for benchmark difficulty and reliability
• Validate benchmark performance against real-world evaluations and customer needs
• 2-4 years of experience in AI benchmarks or evaluation infrastructure