- Company Name
- Agilisys
- Job Title
- Lead Data Engineer
- Job Description
-
**Job Title:** Lead Data Engineer
**Role Summary:**
Lead the design, implementation, and optimisation of scalable, secure data platforms on Microsoft Fabric and Azure for a major public‑sector client. Provide hands‑on technical leadership, mentor a small team, and drive best practices in data engineering, DevOps, and governance over a 6‑month contract.
**Expactations:**
- Active UK security clearance.
- 6‑month remote contract, based in the UK.
- Deliver high‑profile programme, shaping data architecture and operating standards.
- Mentor engineers, champion automation, and maintain delivery excellence.
**Key Responsibilities:**
- Design and evolve modern data platforms using Microsoft Fabric, Azure Synapse Analytics, and Azure Data Factory, integrating legacy and cloud‑native systems.
- Set strategic technical direction, architecture standards, reusable patterns, and performance & security guidelines across hybrid environments.
- Lead hands‑on development of end‑to‑end data solutions (Lakehouses, Warehouses, Notebooks, Dataflows) for batch and real‑time workloads.
- Build scalable data models, ingestion frameworks, and APIs; integrate Power BI semantic models within Fabric.
- Implement CI/CD pipelines for Fabric components, versioning with Git, automated testing, and multi‑environment deployment via Fabric Deployment Pipelines.
- Apply data pipeline optimisation (query tuning, Delta/Parquet formats, compute scaling).
- Design and implement ETL/ELT workflows, CDC, SCD, and schema evolution strategies.
- Ensure secure configuration (Fabric Environments, Azure Key Vault) and enforce data security, governance, and observability.
- Conduct peer reviews, document architectural decisions, and foster a culture of automation and continuous improvement.
**Required Skills:**
- Microsoft Fabric, Azure Synapse Analytics, Azure Data Factory, Fabric Data Pipelines
- Lakehouse, Warehouse, Notebook, Dataflow development
- Power BI integration and semantic modelling
- Python, SQL, Apache Spark programming
- Data modelling (3NF, Kimball) and optimisation (consulting, indexing, query tuning)
- ETL/ELT design, CDC, SCD, schema versioning
- CI/CD, Git, Fabric Deployment Pipelines, automated testing, unit/quality checks
- Azure Key Vault, secure configuration, environment management
- Distributed data processing, storage, compute within Azure ecosystem
- Strong communication, mentoring, and leadership abilities
**Required Education & Certifications:**
- Bachelor’s (or higher) in Computer Science, Data Engineering, Information Systems, or related field.
- Azure Data Engineer Associate (DP-203) or equivalent Microsoft certification.
- Additional certifications beneficial: Microsoft Certified: Azure Fundamentals (AZ-900), Microsoft Certified: Data Engineer Associate (DP‑203), Azure Solutions Architect Expert, or Azure DevOps Engineer Expert.
---