Skills

Python Unity SQL Data Engineering Apache Spark GitLab AWS CloudFormation Machine Learning Programming apache AWS AWS Cloud Spark Databricks PySpark

Job Specifications

Senior AWS Technical Lead

Location: Glasgow – 2/3 Days on site
Duration: 12 Months
Rate: £350 – £420 Inside IR35

Role Description:
To build and maintain the systems that collect, store, process, and analyse data, such as data pipelines, data warehouses and data lakes to ensure that all data is accurate, accessible, and secure.
Candidates working for this role will either contribute to strategic deliverables by building data engineering pipelines using the AWS technology stack, or support platform modernization efforts by migrating existing AWS components to Databricks, depending on their assessed skill sets.

Should lead the project:

Strong hands-on data engineering experience delivering production-grade solutions.
Ability to design, implement and optimise complex data pipelines independently.
Strong SQL and programming (Python/Pyspark preferred); capable of building reusable and scalable code.
Experience with modern orchestration tools (e.g. Airflow, Step Functions, etc.).
Hands-on experience on AWS cloud platform with skillsets listed below
Ability to proactively analyse existing processes, suggest improvements, and drive technical solutions end-to-end.
Comfortable engaging with stakeholders, understanding business requirements, and translating them into technical design.
Strong Experience in Databricks development/migration. (this is for Databricks engineer)
Experience with Apache Spark, Databricks Delta Lake, Unity Catalog, and MLflow.( this is for Databricks engineer)

Accountabilities

Build and maintenance of data pipelines that enable the transfer and processing of durable, complete and consistent data.
Design and implementation of data warehoused and data lakes that manage the appropriate data volumes and velocity and adhere to the required security measures.
Development of processing and analysis algorithms fit for the intended data complexity and volumes.
Collaboration with data scientist to build and deploy machine learning models.

Hands-on Skills:

Data Engineering Coding background and Hands on experience on below AWS services.
S3, Lambda, Glue, Step Function, Athena, Sagemaker, VPC, ECS, IAM, KMS etc.
CloudFormation
Python
Unit Testing
Gitlab
PySpark
AI/ML knowledge (Good to have)
**UI development expereince as must along with AWS Cloudformation.**

About the Company

Community-led, diverse in behaviour, for the digital tech brave. We are Trust in SODA, the digital tech recruitment specialists. We empower and build diverse communities that represent the digital tech world we live in. We are proud to be officially B Corp certified, joining a global network of organisations dedicated to using their platform as a force for positive change. Our consultants focus on delivering talent solutions across Software Engineering, DevOps, Data, Cloud & Infrastructure and Creative. Our SODA Squad op... Know more