Home
Jobs
Simon James I.T. Ltd
Principal Test Engineer – AI Core Services (Azure / .Net)

Principal Test Engineer – AI Core Services (Azure / .Net)

Hybrid

Cambridge, United kingdom

Senior

Freelance

16-10-2025

Share this job:

Score my CV

Skills

Leadership CI/CD DevOps Azure DevOps Test Security Testing Azure Agile .NET Data Science Microsoft Azure

Job Specifications

Principal Test Engineer - AI Core Services

Location: Cambridge, UK (Hybrid) | Contract

Simon James IT are hiring a Principal Test Engineer to lead the validation of cloud-native AI services and APIs built on Microsoft Azure and .NET. You'll design automated testing frameworks and AI-specific evaluation tools to ensure reliability, security, and ethical compliance.

Responsibilities

Lead functional, performance, and security testing of Azure-based services
Build automated test suites for APIs, service components, and AI pipelines
Develop tools to assess AI outputs for accuracy, consistency, and safety
Define quality metrics and observability with engineering and data science teams
Ensure compliance with NIST AI RMF, EU AI Act, and ethical AI standards
Conduct threat modelling and validate exposed APIs
Mentor junior testers and drive innovation

Qualifications

Proven experience testing cloud-native apps using Microsoft/.NET
Strong background in automated testing and QA leadership
Deep understanding of AI systems and regulatory compliance

Required Skills

AI evaluation: LLM scoring, hallucination detection, prompt engineering, RAG, model orchestration
AI safety: bias detection, explainability, ethical AI practices
Cloud & DevOps: Azure DevOps, CI/CD, containerized environments
Testing: API validation, performance profiling, OWASP security testing
Agile collaboration across global R&D teams
Experience with advanced AI testing methodologies

About the Company

Simon James IT is an international specialist IT consultancy based in the City of London. Coming from successful consultancy backgrounds we have taken the positive elements of large scale recruitment and built a business model to provide a personalized solution for your recruitment needs. We offer a dedicated service to ensure you get the best possible candidates within both your timescales and budget. Our business has grown organically from building solid foundations within niche technology areas and working closely wit... Know more