Job Specifications
Title: Software Engineer 4 (Lead Cloud Engineer)
Duration: 12 Months
Type: W2 Only
Location: Charlotte, NC – Hybrid Role
Key Responsibilities
Platform & Architecture Leadership
Lead design, deployment, and management of OpenShift clusters or GCP projects supporting containerized Spark applications
Architect and enhance the platform used for executing statistical risk models
Drive platform modernization and cloud adoption initiatives
Distributed Data Processing
Architect and oversee large-scale data processing workflows using Apache Spark
Optimize Spark jobs for performance and cost efficiency using Kubernetes orchestration and auto-scaling
Oversee integration of Spark with data sources (Kafka, S3/GCS, databases, data lakes)
Application & Tooling Development
Lead development of Python-based microservices and backend tools (Django preferred) to support statisticians and data users
Ensure scalability, usability, and reliability of tools and platform features
Automation & CI/CD
Design and govern CI/CD pipelines in GitHub Actions, Sonar, Helm, Harness, etc.
Implement automation across deployments, environment management, and testing
Monitoring & Reliability
Monitor and tune cluster health, performance, and resource allocation using Prometheus, Grafana, and GCP tools
Troubleshoot distributed and cloud-native system issues
Security & Compliance
Ensure solutions follow enterprise security requirements
Implement RBAC, encryption, and secure coding patterns
Cross-Functional Leadership
Partner with data scientists, platform engineering, DevOps, and architecture teams
Provide technical guidance and mentorship to junior and senior engineers
Lead production deployments and root-cause analysis efforts
Qualifications:
Experience
5–7+ years with Apache Spark for big data processing
3+ years with Python/Django backend development
3+ years building tools or platforms for high-volume data users
3–5+ years working with Google Cloud or equivalent cloud platforms
3+ years managing OpenShift/Kubernetes containerized workloads
Experience supporting data-heavy financial or regulated environments preferred
Technical Skills:
Proficiency in Spark frameworks (PySpark, Scala, or Java)
Strong experience with Docker/Kubernetes concepts
Hands-on OpenShift administration
Experience with conda environments
Strong coding in Python, Scala, or Java
CI/CD exposure: GitHub Actions, Helm, Harness, Sonar
Strong debugging skills across distributed systems
Knowledge of GCP services (GCS, IAM, GKE, Cloud Run, etc.)
Education
Bachelor’s degree in Computer Science, Engineering, or related discipline
Role Level
Lead Engineer: 8–10+ years with demonstrated leadership of complex platforms
About the Company
TrueSkilla specialize in shaping the future of your workforce. As the trusted partner for businesses seeking exceptional talent, we take pride in connecting the right people to the right opportunities. Our mission is to empower your organization with a talented workforce that drives success and innovation. With a commitment to excellence, a deep understanding of your unique needs, and a proven track record of matching exceptional talent with top-tier companies, TrueSkilla is your gateway to a brighter and more prosperous fut...
Know more