About this role

Scale AI is seeking a Senior Machine Learning Engineer to ensure the reliable performance and continuous improvement of production AI agents. You will build observability tools, design evaluation frameworks, and develop ML systems to detect drift and misalignment. This role involves end-to-end ownership from idea to production, collaborating with cross-functional teams to translate requirements into robust platform capabilities.

What we are looking for 6

Build and scale observability for agent behavior in production
Design and automate evaluation methodologies and metrics for agentic applications
Develop ML systems to detect drift, anomalies, and misalignment in production
Design and run rigorous experiments to validate model and agent performance improvements
Collaborate with product managers and engineers to productionize research
Translate enterprise and government requirements into platform capabilities

Skills mentioned 1

Machine Learning