- Company Name
- Onepoint
- Job Title
- Consultant Databricks Engineer - Expert (F/H)
- Job Description
-
Job Title: Databricks Engineer Consultant – Expert
Role Summary: Deliver end‑to‑end Databricks solutions for clients, designing, building, and optimizing large‑scale data pipelines and cloud Big Data architectures. Champion performance tuning, data governance, and CI/CD automation while collaborating with Data Science and cross‑functional teams.
Expactations: • Minimum 3 years of professional data engineering experience (full‑time). • Proven hands‑on expertise with Apache Spark and Databricks on Azure, AWS or GCP. • Ability to solve complex performance and architectural challenges. • Strong communication and teamwork skills for multidisciplinary collaboration. • Self‑motivated, continuous learner staying updated on Big Data innovations.
Key Responsibilities: • Design, develop and optimize Spark jobs, ETL/ELT pipelines, and lakehouse solutions using Databricks. • Build and maintain cloud‑based Big Data infrastructures (Data Lake, Data Warehouse). • Tune Spark configurations and cluster resources for peak performance. • Integrate machine‑learning workflows with MLlib and deploy models. • Ensure data security, governance and compliance (GDPR, internal standards). • Automate data workflows with CI/CD tools (Git, Jenkins, Azure DevOps, Terraform). • Create interactive dashboards and visualizations via Databricks SQL, Power BI or Tableau.
Required Skills: • Apache Spark, Databricks, PySpark, Scala, SQL (Java optional). • Cloud platforms: Azure Databricks, AWS Glue/EMR, GCP Dataproc. • Delta Lake, schema enforcement and time‑travel. • Performance optimization, query tuning, job profiling. • Data modeling, lakehouse architecture, ETL/ELT patterns. • Automation & CI/CD (Git, Jenkins, Azure DevOps, Terraform). • Basics of MLlib, optional machine‑learning experience. • Understanding of data security, access control and regulatory compliance.
Required Education & Certifications: • Bachelor’s or higher in Computer Science, Data Engineering or related field. • 3+ years as a data engineer with demonstrable Databricks experience. • Relevant certifications (e.g., Databricks Certified Professional Data Engineer, Azure Data Engineer Associate, AWS Big Data Specialty) are a plus.