Job Specifications
Job Title: Senior Databricks Architect
Location: Remote (Texas Residents only)
Job Type: Contract
Note: The person has to come visit the Austin office once a Month.
Job Summary:
We are looking for a Senior Databricks Architect to join and lead our data engineering team. This role combines hands-on development with team leadership responsibilities. The ideal candidate will have strong expertise in Databricks and cloud data platforms, as well as a proven ability to guide and mentor a team of engineers. You will play a key role in architecting and building scalable data pipelines, enabling advanced analytics and driving our data strategy forward.
Key Responsibilities:
Lead a team of data engineers, providing technical guidance, mentorship, and support for project delivery.
Design, develop, and optimize scalable ETL pipelines using Apache Spark on Databricks.
Drive architecture decisions for data processing frameworks and cloud infrastructure.
Collaborate with stakeholders, including data scientists, analysts, and business teams, to understand data requirements and translate them into technical solutions.
Implement and manage data lakes, data warehouses, and Delta Lake environments on cloud platforms.
Monitor and optimize Databricks cluster performance, ensuring cost-efficiency and scalability.
Oversee integration with various structured and unstructured data sources across cloud and on-prem environments.
Establish and enforce best practices around coding, testing, security, and DevOps for the team.
Create and manage CI/CD pipelines using tools such as Azure DevOps, GitHub Actions, or similar.
Document technical architecture, workflows, and development processes.
Required Skills and Qualifications:
Bachelor's or Master's degree in Computer Science, Engineering, or a related field.
Experience in data engineering, with at least 2 years in a technical leadership role.
Extensive hands-on experience with Databricks, Apache Spark, and Delta Lake.
Strong experience with at least one major cloud platform (Azure, AWS, or Google Cloud Platform).
Proficiency in Python, SQL, and optionally Scala.
Experience managing and mentoring data engineers or developers.
Sound understanding of data modeling, data warehousing, and distributed data processing.
Familiarity with infrastructure-as-code tools (Terraform, ARM templates, etc.).
Strong problem-solving, communication, and leadership skills.
About the Company
For more than 25 years, Daman has been helping clients maximize the value of their data with a full spectrum of data management solutions. From modern data engineering to hybrid-analytics that leverage machine learning and AI, we deliver high-performance, enterprise-scale solutions for hyper-competitive advantage. Our clients are innovating every day to solve the world's biggest challenges. From healthcare and transportation to finance and manufacturing, they hire us to help unlock maximum value from their information assets...
Know more