Apply on Employer Site

Pearson · 3 months ago

GenAI/LLM Systems Test Engineering Manager

United States

Full-time

Remote

Senior Level

$165K/yr - $170K/yr

6+ years exp

Pearson is seeking a highly technical GenAI/LLM Systems Test Engineering Manager to lead quality engineering efforts for the XL+ initiative, delivering next-generation AI-powered learning solutions. The role involves designing innovative test strategies for AI solutions and collaborating with development, product, and data science teams to ensure the quality of GenAI-powered educational products.

BooksDigital MediaEdTechEducationHigher Education

H1B Sponsor Likely

Responsibilities

Define and drive AI Test Strategy: Create comprehensive test strategies specifically for GenAI products and LLM-based workflows, including model accuracy, bias, safety, and reliability validation

Develop and maintain AI testing frameworks, including prompt testing and evaluation of model outputs

Hands-on Technical Leadership: Build and evolve test automation frameworks, pipelines, and evaluation harnesses for AI/ML models, APIs, and integrated systems

AI-Specific Quality Validation: Design tests around prompt engineering, hallucination detection, model evaluation metrics, and edge-case scenarios

System & Integration Testing: Validate end-to-end workflows, including multi-agent orchestration, MCP models, APIs, and UI interactions

Work closely with Performance & Scalability Test team to ensure AI-driven systems perform consistently at scale across different use cases and data sets

Cross-Functional Collaboration: Partner with Dev, Product, and Data Science to align quality standards with evolving GenAI architectures

Mentorship: Grow and mentor a team of QE engineers, building technical depth and expanding the team’s expertise in AI testing

Continuous Improvement: Drive defect analysis, implement lessons learned, and adopt emerging tools and methods for testing AI

Coordinate testing activities across globally distributed QE teams

Manage defect tracking, root cause analysis, and continuous improvement for all releases

Mentor and support a growing team of QA engineers

Qualification

AI/ML product testingTest automation frameworksAPI/UI testingCI/CD pipelinesPythonPerformance testingScalability testingSecurity testingAgile/Scrum environmentsCross-functional collaborationCommunicationLeadership skills

Required

6+ years of experience in Quality Engineering, with at least 3 years in a technical leadership or managerial role

Strong technical expertise in AI/ML product testing, including GenAI workflows, model evaluation, and/or LLM-based solutions

Hands-on experience with test automation frameworks, API/UI testing, and CI/CD pipelines

Familiarity with LLM testing challenges (e.g., prompt variation, non-deterministic outputs, evaluation metrics, model drift)

Knowledge of Python or similar languages for building test tools or harnesses

Experience in performance, scalability, and security testing of distributed systems

Proven ability to work in cross-functional, Agile/Scrum environments

Excellent communication and leadership skills

Bachelor's degree in Computer Science, Engineering, or related field

Preferred

Exposure to prompt engineering, bias testing, or AI ethics frameworks is a plus

Advanced degree preferred

Company

Pearson

Glassdoor3.7

Pearson operates as a media and education company that offers a wide range of services to its customers.

Founded in 1998

London, England, GBR

10001+ employees

https://www.pearson.com/

H1B Sponsorship

Pearson has a track record of offering H1B sponsorships. Please note that this does not guarantee sponsorship for this specific role. Below presents additional info for your reference. (Data Powered by US Department of Labor)

Distribution of Different Job Fields Receiving Sponsorship

Represents job field similar to this job

Trends of Total Sponsorships

2023 (1)

2021 (1)