Browse All Jobs

SEAL Research Scientist, Frontier Risk Evaluations

Research Scientist for Frontier Risk Evaluations at Scale.
Scale AI
On-Site
United States
USD 176,000 - 300,000
Job Description
Scale is seeking a Research Scientist to join their Safety, Evaluations, and Alignment Lab (SEAL). In this role, the Research Scientist will focus on Frontier Risk Evaluations, designing and creating evaluation measures, harnesses, and datasets for measuring the risks posed by frontier AI systems. Scale is a leading data and evaluation partner for frontier AI companies, playing a vital role in understanding the capabilities and safeguarding large language models (LLMs).

Responsibilities include:
  • Designing and building harnesses to test AI agents for dangerous capabilities.
  • Developing and running human-in-the-loop tests of AI capabilities to deceive, manipulate, blackmail, or otherwise engage in social engineering.
  • Collaborating with government agencies or other labs to scope and design evaluations.

Requirements:
  • Commitment to promoting safe, secure, and trustworthy AI deployments.
  • Practical experience conducting technical research collaboratively, with proficiency in frameworks like Pytorch, Jax, or Tensorflow.
  • Adept at interpreting research literature and quickly turning new ideas into prototypes.
  • A track record of published research in machine learning, particularly in generative AI.
  • At least three years of experience addressing sophisticated ML problems.
  • Strong written and verbal communication skills.

The role offers:
  • Comprehensive health, dental and vision coverage
  • Retirement benefits
  • Learning and development stipend
  • Generous PTO
  • Commuter stipend (potentially)
Apply Manually

Scale AI

Scale AI accelerates the development of AI applications across industries. The company's products power advanced language models, generative models, and computer vision models. Scale AI serves generative AI companies, government agencies, and enterprises, assisting organizations in building and deploying AI. Committed to inclusivity and equal opportunity, Scale AI fosters professional growth, offering opportunities to contribute to cutting-edge AI projects and collaborate with experts in the field.

All Jobs at Scale AI (200)