Principal Data Engineer

Job Description

Lantern is seeking a Principal Data Engineer to play a pivotal role in shaping the future of its Data Platform. This is a hands-on position where the individual will be responsible for developing and implementing the strategic vision for the platform, while also championing advanced Data Engineering, ML, and AI capabilities across the organization. The Principal Data Engineer will mentor team members and apply their expertise in Data Architecture and software engineering best practices to strengthen the platform's foundation. Success in this role requires balancing technical excellence with business value creation through effective stakeholder collaboration.

Responsibilities:

• Architect and build platform foundations including Observability, lineage, Quality Control Frameworks while ensuring scalability, performance and security.
• Modularize platform frameworks to support diverse data needs, variety of data formats (EDI, API, JSON, etc.), and deliver batch and real-time insights.
• Collaborate with Data Engineers, Architects, and Business Analysts in building pipelines, data models to improve reliability and interpretability of data for analytical and business needs.
• Build technical design documents, evaluate emerging technologies, and lead proof-of-concept initiatives to validate solutions.
• Champion software engineering best practices including code standards, effective review processes, CI/CD, and Data Infrastructure.
• Evolve platform capabilities to integrate ML and AI Models to accelerate business use cases.
• Adopt an iterative product development mindset to solve for challenges on the platform.
• Partner with business stakeholders, and technology leadership to define Data Strategy, Roadmap and hiring for future needs.
• Foster a positive team environment that embraces continuous learning and technical innovation.
• Mentor team members and build a high-performance engineering culture through collaboration, and knowledge sharing.

Requirements:

• Bachelor’s or master’s degree in computer science, Engineering, or a related field.
• 12+ years of experience in Data Engineering, with at least 5+ years as Principal Engineer.
• 5+ years of experience with programming languages (Python, Scala etc.), with a deep understanding of Spark and Databricks, and ability to optimize and performance tune the pipelines.
• 3+ years of working with EDI or HIE standards (e.g. X12, HL7, and/or FHIR).
• Proven leadership experience managing and developing data engineering teams, including team growth, skill development, and strategic project execution.
• Prior experience with designing and managing large scale data infrastructure.
• Expertise with cloud platforms, databases, and modern data stack with proficiency in Distributed Data Systems (e.g. Kafka, NoSQL).
• Experience in building data lakes to support high speed querying by the end users.
• Knowledge of AI/ML technologies to support various business use cases.
• Knowledge of OOA/D and N-tier web or mobile applications best practices
• Strong communication skills, with the ability to convey complex technical concepts to non-technical stakeholders.
• Experience or knowledge with creating RESTful API’s and data visualizations.

The role offers:

• Remote work opportunity