Clash
of Jobs
Browse All Jobs
Job Description
Anthropic is seeking a Senior Software Engineer to join its Data Acquisition team. This team is responsible for acquiring internet data through web crawling and data partnerships. The role is critical to Anthropic's efforts in producing the best pre-trained models.
Responsibilities:
- Develop and maintain a large-scale web crawler.
- Build pipelines for data ingestion, analysis, and quality improvement.
- Build specialized crawlers for high-value data sources.
- Build tools for improving the observability and debuggability of crawler systems.
- Collaborate with team members on improving data acquisition processes.
- Participate in code reviews and debugging sessions.
Requirements:
- 5+ years of experience with building and running large distributed systems.
- Familiarity with the non-technical tradeoffs of internet-scale crawling (data privacy, robots.txt adherence, etc.).
- Quickly understanding systems design tradeoffs, keeping track of rapidly evolving software systems.
- Familiarity with Cloud-based compute and storage solutions.
- Familiarity with Python.
- Bachelor's degree in a related field or equivalent experience.
Anthropic offers:
- Competitive compensation and benefits.
- Optional equity donation matching.
- Generous vacation and parental leave.
- Flexible working hours.
