Machine Learning Research Scientist, Evaluations

Scale AI is seeking a Machine Learning Research Scientist to join their GenAI Research Organization. This role focuses on developing benchmarks and diagnosing failure modes in large language models and multimodal systems. You will analyze model behavior, design evaluation methods, and apply post-training expertise to improve AI development.

• Develop rigorous evaluations and diagnostic methods for frontier LLMs
• Analyze model behavior to identify and diagnose failure modes
• Design and build benchmarks for text and multimodal LLM capabilities
• Apply post-training expertise (SFT, RLHF, reward modeling) to address failures
• Collaborate with researchers and engineers on evaluation best practices
• Publish research findings in top-tier AI conferences