Skills

Python SQL Data Governance Apache Spark CI/CD DevOps Version Control Azure DevOps Machine Learning Programming apache git Azure Spark r programming Databricks PySpark

Job Specifications

Sr. Data Engineer (Databricks)

Remote, Canada

Long Term Contract

Note: Security clearance is mandatory

Programming Languages

Python Certification (e.g., PCAP)
R Programming Certification (e.g., DataCamp, Coursera)

b) Technical Expertise

Proven experience converting SAS EG code to PySpark, SQL, and/or R
Familiarity with SAS macros, data steps, and PROC SQL
Experience with Databricks notebooks, Delta Lake, and Spark SQL
Ability to optimize PySpark code for performance and scalability

c) Mainframe & DB2 Integration

Experience accessing DB2 data via ODBC/JDBC or ETL tools
Familiarity with mainframe systems (JCL, COBOL, IMS)
Ability to replicate or migrate data from DB2 to cloud data lake

d) Tooling & Automation

Use of SAS-to-PySpark conversion tools (e.g., SAS2PY, SPROCKET, Scintilla, AI tools)
Automated testing and validation frameworks for data parity
Schema mapping and lineage tracking capabilities

e) Project Delivery & Governance

Experience successfully onboarding with client systems.
Clear documentation of conversion logic and testing results
Version control and CI/CD integration (e.g., Git, Azure DevOps)
Compliance with internal data governance and security policies

Databricks Platform

Databricks Certified Data Engineer Associate or Professional
Databricks Certified Machine Learning Associate or Professional

Apache Spark

Apache Spark Developer Certification

About the Company

Based in Toronto, Ontario, Epsilon Solutions Ltd. has been providing business and IT solutions throughout Canada since 2013. At Epsilon Solutions, we are cognizant that you have many choices when selecting an IT services company. Epsilon Solutions delivers expert information technology consultants who can contribute to a fast-paced and ever-changing technological world. Whether you need to accelerate your IT projects or get your career moving full speed ahead, Epsilon Solutions helps you achieve success. Our IT talent mana... Know more