Senior Data Engineer

Job Description

Dcard is seeking a Senior Data Engineer to join their team and contribute to building data products that serve as the foundation for their services. Dcard is a popular social media platform among young people. The company is committed to creating a space where everyone can share their stories and is expanding into new businesses and international markets.

The Senior Data Engineer will be responsible for developing and optimizing data-related products, including OLAP system development, data pipeline design, and data infrastructure construction. They will also research and introduce the latest data engineering technologies and collaborate with other developers to build and optimize Dcard's data-related systems.

Responsibilities include:

• Participating in the development and optimization of Dcard's data-related products.
• Researching and introducing the latest Data Engineering technologies.
• Collaborating with other project developers to build and optimize Dcard's data-related systems.
• Delivering feasible architectures quickly and promoting the optimization and improvement of data products.

Requirements:

• Proficiency in Python and willingness to learn new languages (Golang, Java).
• Practical experience with at least one OLAP database system (e.g., BigQuery, ClickHouse, Amazon Redshift, or Snowflake).
• Ability to design technical solutions based on defined problems.
• Good communication and collaboration skills.
• Ability to write highly readable, maintainable, and scalable code.
• Experience deploying and maintaining projects on Kubernetes.
• Experience in developing and maintaining ETL processes.
• Familiarity with any task orchestration tool (e.g., Airflow, Luigi, Prefect, Dagster).

Nice to have:

• Familiarity with distributed system design.
• Familiarity with streaming architectures and development processes.
• Experience building and maintaining monitoring and alerting systems.
• Familiarity with dbt and data modeling.
• Familiarity with any cloud computing platform (e.g., GCP, AWS, Azure).
• Infrastructure as Code experience.
• Understanding of data governance.