Browse All Jobs

Senior AWS Data Engineer (6 Months Contract)

Senior AWS Data Engineer to enhance data pipeline in AWS Cloud.
PALO IT
On-Site
Singapore
Undisclosed salary
Job Description

PALO IT is seeking a Senior Data Engineer for a 6-month contract in Singapore. The ideal candidate will enhance an existing data pipeline, migrating it to the AWS Cloud using AWS managed components. The Senior Data Engineer will work on acquiring, storing, governing and processing huge sets of structured / unstructured data.

The role involves working closely with other PALO IT experts in Data Intelligence, Research, UX Design, Digital Technology and Agile teams to enable intelligence in the digital mesh of our clients and deliver big impact to them.

Responsibilities:

  • Enhance the existing data pipeline using industry-standard tools and mechanisms.
  • Migrate data pipeline to AWS Cloud using AWS managed components.
  • Handle data download, decryption, quality checking, cleaning, processing, and prediction including comprehensive error handling, error reporting, and self-recovery mechanisms.

Requirements:

  • Bachelor's, Master's, or Ph.D. degree in IT, Information Management, and/or Computer Science with at least 7 years of experience.
  • Proficient in Python, SQL Knowledge, Schell scripts (Linux).
  • Proficient in Airflow or similar tool to create pipelines.
  • Proficient in AWS tools like S3, Glue, AWS EMR, Lambda, Step functions, Fargate, AWS Batch.
  • Familiar with tools like AWS Batch, Athena, file formats like Parquet, NoSQL knowledge and tools like AWS OpenSearch.
  • Good knowledge of the big data technology landscape and concepts related to distributed storage/computing.
  • Experience with big data frameworks (e.g. Hadoop, Spark) and distributions (Cloudera, Hortonworks, MapR).
  • Experience with batch & ETL jobs to ingest and process data from multiple data sources.
  • Experience with NoSQL databases (e.g. Cassandra, MongoDB, Neo4J, ElasticSearch).
  • Experience with querying tools (e.g Hive, Spark SQL, Impala).
  • Experience with Power BI.

What PALO IT Offers:

  • Stimulating working environments
  • Unique career path
  • International mobility
  • Internal R&D projects
  • Knowledge sharing
  • Personalized training
Apply Manually

PALO IT

All Jobs at PALO IT (38)