Wolters Kluwer · 1 week ago
Lead Quality Engineer - AI
USA - Riverwoods, IL
Full-time
Hybrid
Senior Level
$90K/yr - $157K/yr
5+ years expWolters Kluwer is seeking a Lead AI Quality Engineer to ensure the quality, reliability, and trustworthiness of AI-powered product experiences in their Tax and Accounting division. This role involves designing tests to confirm system behavior, developing automated tests, and collaborating with teams to define quality metrics.
BankingFinanceInformation ServicesInformation TechnologyLegalMobilePublishingSoftware
Responsibilities
Design and implement evaluation harnesses to measure retrieval accuracy, citation correctness, response quality, and overall system behavior
Develop automated tests for APIs, ingestion pipelines, and chat workflows
Collaborate with developers and product managers to define quality metrics (accuracy, latency, cost, hallucination rate)
Analyze logs, traces, and feedback signals to identify root causes of failures in AI-driven responses
Create regression suites to ensure changes to prompts, chunking, or embeddings don’t break existing behavior
Validate REST APIs and service integrations for resilience, correctness, and security
Contribute to observability by instrumenting metrics and dashboards for system performance
Participate in sprint planning and retrospectives, ensuring testability is built into features from day one
Qualification
Required
Bachelors Degree in Computer Science or equivalent
5+ years of experience in software testing, quality engineering, or equivalent engineering roles with a focus on validation and reliability
Experience with AI evaluation frameworks (e.g. LlamaIndex evals, OpenAI Evals, Ragas, TruLens, or custom harnesses)
Strong skills in Python testing frameworks (Pytest, unittest, or equivalent)
Experience testing web applications and APIs
Familiarity with AI/ML or non-deterministic system testing
Knowledge of CI/CD pipelines, Git, and automated regression testing
Strong analytical skills: able to define metrics and success criteria where outputs aren't deterministic
Comfortable working in a fast-paced Agile environment with weekly sprints, pairing, and close collaboration with PM/UX/Dev
Preferred
Knowledge of retrieval-augmented generation (RAG) pipelines
Experience with metrics/observability tooling (Grafana, Prometheus, Datadog)
Familiarity with containerized environments (Docker, Kubernetes)
Exposure to performance/load testing tools (Locust, k6, JMeter)
Benefits
Medical, Dental, & Vision Plans
401(k)
FSA/HSA
Commuter Benefits
Tuition Assistance Plan
Vacation and Sick Time
Paid Parental Leave
Company
Wolters Kluwer
Wolters Kluwer is an information services company specializing in software solutions and services for the healthcare and legal sectors.
10001+ employees
H1B Sponsorship
Wolters Kluwer has a track record of offering H1B sponsorships. Please note that this does not
guarantee sponsorship for this specific role. Below presents additional info for your
reference. (Data Powered by US Department of Labor)
Distribution of Different Job Fields Receiving Sponsorship
Represents job field similar to this job
Trends of Total Sponsorships
2025 (94)
2024 (22)
2023 (23)
2022 (10)
2021 (3)
Funding
Current Stage
Public CompanyTotal Funding
$1.78B2025-06-23Post Ipo Debt· $578.76M
2025-03-13Post Ipo Debt· $542.74M
2024-03-11Post Ipo Debt· $655.84M
Leadership Team
Jason Marx
CEO, Wolters Kluwer Tax & Accounting
Nancy McKinstry
CEO
Recent News
Business Wire
2026-01-19
thefly.com
2026-01-12
Company data provided by crunchbase