Data Engineer

Job Description

• Design and implement robust, scalable data pipelines and architectures to support data ingestion, processing, and storage. Including performance optimizations for data modeling and ingestion
• Develop and optimize complex SQL queries and stored procedures for data extraction, transformation, and analysis.
• Model data to meet different use casesng applications and automate data workflows.
• Collaborate with data scientists and analysts to understand data requirements and deliver high-quality data solutions.
• Lead the integration of data from various sources into data lakes and warehouses, ensuring data quality and consistency.
• Monitor and troubleshoot data pipelines and workflows to ensure optimal performance and reliability.
• Communicate with and support data users
• Document data processes, data models, and architectural designs to ensure knowledge sharing and compliance with best practices.

Job Requirements

Prerequisites
- Experience: Minimum 3 years in data engineering fields with system integration, and at least 1 year in system integration and implementation in cloud/web-based environments.
- Proven Solutions: Demonstrated experience in providing effective, working solutions and implementations, particularly in cloud-based environments.

Technical Skills:
- Solid understanding of ETL processes, data warehousing concepts, and data modeling best practices.
- Proficiency in Databricks, Azure Data lake, PowerBI, Tableau and related data processing and visualisation software.
- Familiarity with Windows, Linux, AWS and/or Azure platforms.
- Strong programming skills in languages such as Python and R is a must
- Proficiency in other programming languages such as Java, Scala and C# will be advantages
- Experience in data processing frameworks (e.g., Apache Spark, Apache Flink)

Preferred Exposure:
- Experience with large-data management system with visualisation tools.
- Experience with Data Integration and ETL Pipelines, Data Warehousing and BI reporting projects.
- Experience with Singapore Government Project will be advantages

Personal Attributes:
- Excellent problem-solving skills
- Ability to work independently
- Collaborative in a fast-paced environment

Work Location