Browse All Jobs

Software Engineer, Inference Ops

Software Engineer role at xAI, focusing on inference ops.
xAI
On-Site
United States
USD 180,000 - 440,000
Job Description
xAI is seeking a Software Engineer to join their Inference Ops team. This role is based in the Bay Area (San Francisco and Palo Alto). The ideal candidate will ensure the reliability of inference services and develop custom debugging tools. xAI's mission is to create AI systems that can accurately understand the universe and aid humanity in its pursuit of knowledge. The company operates with a flat organizational structure, expecting all employees to be hands-on and contribute directly to the mission.Role involves:
  • Ensuring the reliability of inference services, targeting 100% uptime and a 0% error rate.
  • Developing custom debugging tools to trace, replay, and fix errors across the stack.
  • Managing autoscaling, continuous deployment, and feature rollout of inference services.
  • Benchmarking and monitoring the performance of inference engines.
  • Building CI/CD infrastructure for endpoint deployment and image publishing.
Requirements:
  • Experience with large-scale, high-concurrent production serving.
  • Experience with testing, benchmarking, and the reliability of inference services.
  • Experience with CI/CD infrastructure.
xAI offers:
  • Opportunity to work on open-source projects like SGLang.
  • A challenging environment focused on engineering excellence.
  • A flat organizational structure with opportunities for leadership based on initiative and delivery.
Apply Manually

xAI

xAI is an artificial intelligence company focused on building AI systems that deeply understand the universe and assist humanity in its quest for knowledge. It operates with a flat organizational structure that values engineering excellence, curiosity, and strong communication. xAI fosters a collaborative environment where every team member contributes directly to the company’s objectives, with a focus on continuous improvement.

All Jobs at xAI (129)