Skills

Python SQL Encryption GitHub GitLab CI/CD DevOps Monitoring Jenkins Machine Learning AWS Analytics CI/CD Pipelines PySpark Terraform GitHub Actions

Job Specifications

Key Responsibilities

1. Data Pipeline Development

Design and implement scalable ETL/ELT data pipelines using AWS Glue, Lambda, Step Functions, and other AWS services.
Integrate structured and unstructured data from multiple sources into data lakes and data warehouses (S3, Redshift, RDS, Athena).

2. Platform Engineering

Build and maintain cloud infrastructure for analytics platforms using Terraform, CloudFormation, or similar IaC tools.
Develop reusable frameworks and automation scripts to streamline data operations.

3. Data Analytics Enablement

Partner with data engineers, analysts, and data scientists to deliver optimized platforms for analytics and machine learning.
Optimize data models, queries, and transformations for performance and scalability.

4. Monitoring & Maintenance

Monitor data pipelines and proactively troubleshoot failures or performance issues.
Implement CI/CD pipelines for data workflows using GitLab, Jenkins, Bitbucket, or GitHub Actions.

5. Security & Governance

Ensure data security, access control, and governance compliance across all environments.
Implement encryption and role-based access controls for sensitive datasets.

Required Skills

Proficiency in Python, SQL, and PySpark.
Hands-on experience with AWS services (S3, Glue, Lambda, Redshift, RDS, Athena, Step Functions).
Experience with Airflow or MWAA for orchestration.
Familiarity with DevOps/CD pipelines, Terraform, and AWS CDK.
Strong understanding of data modeling, warehousing, and ETL/ELT design patterns.

About the Company

Galent is an AI-native digital engineering firm at the forefront of the AI revolution, dedicated to delivering unified, enterprise-ready AI solutions that transform businesses and industries. Our mission is to empower organizations to thrive in an ever-evolving digital landscape through cutting-edge AI-native services, consulting, and digital engineering. As enterprises face the complexities of integrating advanced technologies, Galent provides scalable, AI-enabled solutions that optimize processes, enhance productivity, an... Know more