LLM Training Frameworks and Optimization Engineer

Job Description

Together.ai is seeking a LLM Training Frameworks and Optimization Engineer to drive innovations in the development and optimization of distributed training frameworks. The engineer will ensure that Together.ai's LLM training pipelines are robust, efficient, and capable of handling the complexities of large-scale distributed systems.

Responsibilities:

• Design, implement, and optimize distributed training frameworks tailored for large language models.
• Develop custom modules, plugins, and features to enhance framework scalability and performance.
• Optimize communication patterns (e.g., gradient synchronization, all-reduce) in distributed training.
• Implement techniques like mixed precision, tensor parallelism, pipeline parallelism, and sharded training.
• Conduct in-depth profiling and debugging of training jobs to identify and resolve bottlenecks.
• Collaborate with hardware teams to optimize performance for GPUs, TPUs, and other accelerators.
• Ensure training systems scale efficiently to thousands of nodes and petabytes of data.
• Develop resilience mechanisms for fault-tolerant and checkpointed training pipelines.
• Work closely with researchers, data engineers, and platform teams to ensure training frameworks meet model and workload requirements.
• Provide guidance and tools to improve the overall efficiency of the LLM development lifecycle.

Requirements:

• 5+ years of experience in deep learning frameworks, distributed systems, or machine learning infrastructure.
• Expertise in distributed training frameworks (e.g., PyTorch DDP, DeepSpeed, Megatron-LM, TensorFlow XLA).
• Strong understanding of parallelism techniques (e.g., data, tensor, pipeline, and ZeRO-based parallelism).
• Familiarity with GPU/TPU hardware and deep learning performance optimizations.
• Proficient in Python and C++ or CUDA for high-performance computing.
• Experience with memory optimization techniques (e.g., activation checkpointing, gradient sharding).
• Knowledge of training dynamics for large-scale LLMs, including hyperparameter tuning and optimization.
• Analytical problem-solving skills and a focus on performance improvement.
• Strong collaboration and communication skills across teams.

Together.ai offers:

• Competitive compensation
• Startup equity
• Health insurance and other competitive benefits