- Company Name
- Morson Edge (Canada)
- Job Title
- Data Engineer
- Job Description
-
**Job title:** Data Engineer
**Role Summary:**
Build, maintain, and optimize modular data pipelines and data infrastructure using the Microsoft Azure stack. Deliver secure, high‑performance data lakes, warehouses, and lakehouses that serve BI, analytics, and downstream applications. Act as a technical partner to architects, analysts, and data scientists, ensuring data quality, lineage, and accessibility.
**Expectations:**
- Design and implement scalable ELT/ETL solutions that provide a single source of truth for business intelligence.
- Collaborate closely with Data Architects, infrastructure, and security teams to enforce data governance, privacy, and compliance.
- Deliver reliable, maintainable code and documentation in an Agile/Scrum environment.
- Provide Tier‑2 production support, troubleshoot ingestion and transformation issues, and drive continuous process improvement.
**Key Responsibilities:**
- Develop and production‑grade data pipelines with Azure Data Factory, Synapse Analytics, Azure Databricks, and Azure Data Lake Storage.
- Clean, transform, and optimize datasets (XML, CSV, JSON, REST) using Python, PySpark, SparkSQL, and T‑SQL.
- Build dimensional models and aggregation layers to support Power BI and other downstream consumers.
- Implement data cataloging, metadata management, and role‑based access controls via Microsoft Purview.
- Automate pipeline orchestration, CI/CD, and test harnesses; manage source control and versioning.
- Monitor pipeline performance, conduct root‑cause analysis, and apply optimizations to meet latency targets.
- Participate in code reviews, unit/regression testing, and user acceptance testing.
- Mentor teammates and promote best practices in data engineering and DevOps.
**Required Skills:**
- Proven experience designing and operating data pipelines in Azure (ADF, ADLS, Synapse, Databricks).
- Strong proficiency in Python, PySpark, SparkSQL, and relational SQL.
- Familiarity with data lakehouse and data warehouse architectures.
- Knowledge of data governance, quality, lineage, and security principles.
- Experience with Azure DevOps, CI/CD, and automated testing frameworks.
- Ability to translate technical requirements into clear, actionable user stories.
- Excellent written and verbal communication; collaborative mindset.
**Required Education & Certifications:**
- Bachelor’s degree (or higher) in Computer Science, Software Engineering, Data Engineering, AI, or related field.
- Certifications such as Azure Data Engineer Associate (DP‑203) or equivalent are preferred.