- Company Name
- OneSource Consulting
- Job Title
- Data Engineer
- Job Description
-
**Job Title:** Data Engineer
**Role Summary:**
Design, build, and maintain cloud‑based data architecture (AWS and Azure) to support enterprise data initiatives. Transform business requirements into scalable, secure, and governed data solutions, ensuring integration, quality, and continuous improvement across the data pipeline.
**Expectations:**
- Deliver end‑to‑end data solutions that meet business and technical standards.
- Lead technical design and implementation of cloud data platforms and orchestration tools.
- Collaborate cross‑functionally to document, test, and audit data workflows.
- Mentor peers and champion best practices in data engineering and DevOps.
**Key Responsibilities:**
- Gather, consolidate, and prepare data for analytics and machine‑learning projects.
- Design, develop, and maintain cloud data warehouses (Snowflake, Redshift, Synapse, etc.) and data lakes.
- Build and manage ETL pipelines using Spark, Python, Hive, Airflow, Azure Data Factory, or equivalent.
- Implement and maintain infrastructure-as-code (AWS CDK/CloudFormation/Terraform, ARM templates, Terraform).
- Create and maintain data models (star schema, Data Vault) and metadata catalogs.
- Conduct risk assessments, produce mitigation plans, and enforce security and governance controls.
- Validate and test solutions, document results, and iterate on lessons learned.
- Train and coach team members on tools, processes, and best practices.
**Required Skills:**
- Proven experience as a Data Engineer with Big Data technologies (Python, Spark, Hive).
- Proficiency in SQL and relational/NoSQL databases.
- Expertise with AWS (CDK, CloudFormation, Terraform, Step Functions, Athena, Glue) and/or Azure (Data Factory, Synapse, Azure Data Lake, ARM/Terraform).
- Knowledge of data orchestration (Airflow, Kubernetes, Azure Pipelines) and CI/CD principles.
- Familiarity with enterprise data modeling (star schema, Data Vault).
- Agile mindset; strong analytical, problem‑solving, and communication skills.
- Ability to manage multiple tasks autonomously and deliver results under tight timelines.
**Required Education & Certifications:**
- Master’s Degree in Computer Science or equivalent professional experience.
- Preferred certifications: AWS Solutions Architect – Associate, AWS Data Analytics – Specialty, Azure Data Engineer Associate (DP-203) or Azure Administrator (AZ-104).
---