Apply on Employer Site

Crowe · 22 hours ago

AI Governance Testing & Validation Manager

Livingston, NJ

Full-time

Hybrid

Mid, Senior Level

$102K/yr - $204K/yr

5+ years exp

Crowe is a leading public accounting, consulting, and technology firm that focuses on delivering innovative solutions. The AI Governance Testing & Validation Manager will lead independent testing and operational monitoring of AI systems, ensuring alignment with business goals and regulatory standards while mentoring consultants and evolving testing methodologies.

AccountingAdviceConsultingFinanceFinancial ServicesInformation TechnologyProfessional ServicesTax Consulting

No H1B

Responsibilities

Independent Testing: Design and execute independent test plans for classical ML and LLMs/GenAI (functional accuracy, robustness, safety, toxicity, jailbreak/prompt-injection, hallucination/error rates); define acceptance criteria and go/no-go recommendations

Sales enablement: Partner with teams to qualify opportunities, shape solutions/SOW/ELs, develop proposals and pricing, and contribute to pipeline reviews. Build client-ready collateral

Offering development: Evolve Crowe’s AI Governance methodologies, accelerators, control libraries, templates, and training. Incorporate updates from standards/regulators into our playbooks (e.g., NIST’s GAI profile)

Thought leadership: Publish insights, speak on webinars/events, and support marketing campaigns to grow brand presence

People leadership: Supervise, coach, and develop consultants; manage engagement economics (scope, timeline, budget, quality) and support recruiting

Bias/Fairness: Plan and run bias/fairness assessments using appropriate population slices and fairness metrics; document mitigations per NIST guidance on identifying/managing bias

Evaluate Explainability: Produce model explainability/transparency artifacts (e.g., model cards, method docs) and apply techniques (SHAP, LIME, feature attributions) aligned to NIST’s Four Principles of Explainable AI

Qualification

AI governance experiencePythonSQLNIST AI RMF 1.0Bias/Fairness assessmentsExplainability methodsML OpsCloud platformsNetworkingRelationship managementWritten communicationVerbal communication

Required

3+ years hands-on AI governance/Responsible AI experience (policy, controls, risk, compliance, or assurance of AI/ML systems)

5+ years in compliance, risk management, and/or professional services/consulting with client-facing delivery and team leadership

Strong Python and SQL (evaluation pipelines, data prep, metric computation, scripting CI jobs)

Demonstrated experience designing fairness/bias tests and applying explainability methods; ability to translate results for non-technical stakeholders

Practical knowledge of NIST AI RMF 1.0 (and GenAI profile), ISO/IEC 42001, and awareness of EU AI Act obligations for high-risk systems

Prior experience should include progressive responsibilities, including supervising and reviewing the work of others, and project management, including self-management of simultaneous work-streams and responsibilities

Strong written and verbal communication and comprehension both formally and informally to our clients and our teams, in a variety of formats and settings, including in interviews, meetings, calls, e-mails, reports, process narratives, presentations, etc

Networking and relationship management

Willingness to travel

Preferred

Experience operationalizing LLM/GenAI evaluations (adversarial/red-team testing, toxicity/harm scoring, retrieval/grounding, hallucination measurement, safety policies) consistent with NIST guidance

Hands-on with ML Ops/observability (e.g., model registries, data validation, drift detection), cloud (AWS/Azure/GCP), and containerization

Familiarity with governance and compliance platforms (e.g., GRC systems) and collaboration with privacy/security/legal

Bachelor's degree required; advanced degree a plus (CS, statistics, data science, information systems, or related)

Certification: AIGP – Artificial Intelligence Governance Professional (IAPP) or equivalent credential in AI governance/privacy/risk (e.g., CIPP/CIPM/CIPT with AI coursework, ISO/IEC 42001 implementer/auditor)