Senior Software Engineer, Data Infrastructure at Evolutionary Scale

EvolutionaryScale is seeking a Senior Software Engineer, Data Infrastructure to join their team in either New York or San Francisco. EvolutionaryScale's mission is to develop artificial intelligence to understand biology for the benefit of human health and society.

The Senior Software Engineer will work closely with bioinformatics and research teams to ensure data jobs are reliable, efficient, and scalable. He/She will implement best practices for handling large-scale data processing, select and integrate the right technologies, and drive continuous improvements in performance and quality of our data sets.

The role involves:

• Designing, developing, and maintaining large-scale batch processing pipelines using tools like Spark and Ray, for acquiring biology datasets.
• Managing data infrastructure components to ensure robust and fault-tolerant operations.
• Optimizing data ingestion, storage, and retrieval processes for acquiring large and growing biology datasets, and for efficient pre and post training data ingestion.
• Creating systems for easy and reproducible data evaluation and experiments.
• Integrating modern ML based data curation technologies with data processing pipelines.
• Working with researchers and other engineering teams to understand data needs, create solutions that meet modeling requirements.

Requirements:

• Proven experience with large-scale data processing systems using technologies such as Hadoop, Spark, or Ray.
• Knowledge of streaming data frameworks like Kafka Streams, Spark Streaming, or Flink.
• Understanding of data processing principles and best practices.
• Strong problem-solving skills, including the ability to research, debug, and resolve complex technical problems.
• Experience with major cloud providers (AWS, GCP, or Azure), including familiarity with data warehousing tools is a plus.
• 5+ years of experience in the above systems.

The role offers:

• Flexibility around work schedules and locations, with the expectation to work half of the days or more of most weeks from one of the offices.