Machine Learning Engineer - Inference

Job Description

Together AI is seeking a Machine Learning Engineer to optimize AI inference systems. The successful candidate will work with large language models to ensure efficient and effective performance at scale. This role offers the chance to collaborate with AI researchers and engineers, contributing to cutting-edge AI solutions.Together AI believes open and transparent AI systems will drive innovation for better outcomes. The company aims to significantly lower the cost of modern AI systems.Responsibilities:

• Design and build production systems for the Together AI inference engine.
• Develop and optimize runtime inference services for large-scale AI applications.
• Collaborate with researchers, engineers, product managers, and designers.
• Conduct design and code reviews.
• Create services, tools, and developer documentation.
• Implement robust and fault-tolerant systems for data ingestion and processing.

Requirements:

• 3+ years of experience writing high-performance production code.
• Proficiency with Python and PyTorch.
• Experience in building high-performance libraries and tooling.
• Understanding of low-level operating systems concepts.
• Knowledge of existing AI inference systems (preferred).
• Knowledge of AI inference techniques (preferred).
• Knowledge of CUDA/Triton programming (preferred).
• Knowledge of Rust, Cython, and compilers (nice to have).

Together AI offers:

Competitive compensation

Startup equity

Health insurance

Other competitive benefits