Browse All Jobs

AI Infrastructure Engineer, Model Serving Platform

AI Infrastructure Engineer for building scalable LLM serving platforms.
Scale AI
On-Site
United States
USD 175,000 - 220,000
Job Description

Scale is seeking an AI Infrastructure Engineer to join their ML Infrastructure team. This engineer will be responsible for designing and building platforms for scalable, reliable, and efficient serving of LLMs and AI agents. The platform supports both internal and external use cases across various environments, powering cutting-edge research and production systems.

The ideal candidate will possess strong ML fundamentals and deep expertise in backend system design. They will work collaboratively, bridging research and engineering to deliver seamless experiences and accelerate innovation.

Role involves:

  • Building and maintaining fault-tolerant, high-performance systems for serving LLMs and agent-based workloads at scale.
  • Collaborating with researchers and engineers to integrate and optimize models for production and research use cases.
  • Conducting architecture and design reviews to uphold best practices in system design and scalability.
  • Developing monitoring and observability solutions to ensure system health and performance.
  • Leading projects end-to-end, from requirements gathering to implementation, in a cross-functional environment.

Requirements:

  • 4+ years of experience building large-scale, high-performance backend systems.
  • Strong programming skills in one or more languages (e.g., Python, Go, Rust, C++).
  • Deep understanding of concurrency, memory management, networking, and distributed systems.
  • Experience with containers, virtualization, and orchestration tools (e.g., Docker, Kubernetes).
  • Familiarity with cloud infrastructure (AWS, GCP) and infrastructure as code (e.g., Terraform).
  • Proven ability to solve complex problems and work independently in fast-moving environments.

Role offers:

  • Comprehensive health, dental and vision coverage.
  • Retirement benefits.
  • A learning and development stipend.
  • Generous PTO.
Apply Manually

Scale AI

Scale AI accelerates the development of AI applications across industries. The company's products power advanced language models, generative models, and computer vision models. Scale AI serves generative AI companies, government agencies, and enterprises, assisting organizations in building and deploying AI. Committed to inclusivity and equal opportunity, Scale AI fosters professional growth, offering opportunities to contribute to cutting-edge AI projects and collaborate with experts in the field.

All Jobs at Scale AI (200)