- Company Name
- Eaglecliff Recruitment
- Job Title
- Python Data Engineer / Software Engineer with Leading Energy Company
- Job Description
-
Job Title: Python Data Engineer / Software Engineer
Role Summary:
Contract-based role focused on designing, developing, and maintaining scalable, cloud‑native data solutions for an energy trading and commodities environment. The position requires building and optimizing Python and PySpark data pipelines, creating microservices, and delivering infrastructure as code on Azure.
Expactations:
- Proven experience in Python development and Spark/SQL data engineering.
- Strong background in Azure cloud services, containerization (Docker/Kubernetes), and microservices architecture.
- Familiarity with Terraform IaC, Agile development practices, and unit testing (PyTest).
- Ability to implement frontend components using HTML/CSS, React, and TypeScript.
- Experience with FastAPI for building RESTful services is essential.
Key Responsibilities:
- Design, implement, and optimize data pipelines using Python, PySpark, and SQL on Azure.
- Develop and maintain microservices and API endpoints with FastAPI.
- Create and manage Docker containers, orchestrate deployments via Kubernetes.
- Write IaC scripts in Terraform to provision and manage Azure resources.
- Apply functional design patterns and best practices for clean, maintainable code.
- Integrate CI/CD pipelines, automate unit tests using PyTest, and enforce code quality standards.
- Collaborate in cross‑functional Agile teams, participate in sprint planning, reviews, and retrospectives.
- Troubleshoot performance issues, recommend scalability improvements, and ensure data quality.
- Create and maintain technical documentation for code, architecture, and deployment processes.
Required Skills:
- Python (≥ 3.6), PySpark, SQL
- Azure (Data Factory, Databricks, Functions, Blob Storage)
- Docker, Kubernetes, Microservices architecture
- Terraform (IaC)
- Unit testing (PyTest)
- Front‑end basics: HTML/CSS, React, TypeScript
- FastAPI framework
- Agile methodologies (Scrum/Kanban)
- Strong problem‑solving and analytical skills
Required Education & Certifications:
- Relevant undergraduate degree (e.g., Computer Science, Engineering) or equivalent professional experience.
- Certifications not mandatory but may be an advantage (e.g., Azure Data Engineer Associate, AWS Certified Data Analytics).