Senior Manager of Meta Evaluation & Quality Assurance jobs in United States
cer-icon
Apply on Employer Site
company-logo

Apple · 10 hours ago

Senior Manager of Meta Evaluation & Quality Assurance

positionSeattle, Washington, United States
timeFull-time
remoteOnsite
senioritySenior Level, Lead/Staff
money$217K/yr - $326K/yr
date8+ years exp

Apple Services Engineering (ASE) powers many AI and LLM features across App Store, Music, Video and more. As these systems increasingly rely on human-in-the-loop evaluation systems, the quality of our decisions is constrained by the quality of our evaluation systems.

AppsArtificial Intelligence (AI)BroadcastingDigital EntertainmentFoundational AIMedia and EntertainmentMobile DevicesOperating SystemsTVWearables
check
Comp. & Benefits
check
H1B Sponsor Likelynote

Responsibilities

Lead, mentor, and develop a multidisciplinary team of Data Scientists and Machine Learning Engineers, fostering a culture of rigorous scientific inquiry, technical excellence, and accountability
Define and drive the strategic roadmap for Meta Evaluation methodology and standards across Apple Services
Oversee the development of ML-based quality validation systems. You will guide ML Engineers in building models that utilize human-in-the-loop workflows to audit evaluators, identifying anomalies, disagreement, and ambiguity in evaluation data
Establish data quality validation standards and define the statistical processes for measuring confidence, calibration, and inter-rater reliability
Partner with Model Engineering and Data Science teams to validate new AI Judges (autograders) and Agents pre-production, ensuring they meet prescribed performance standards before deployment
Collaborate with Operations teams to build active learning loops where human experts adjudicate discrepancies flagged by your validation models, creating a continuous cycle of system improvement
Monitor the health of the evaluation ecosystem, identifying risks such as evaluator drift, bias, or silent agent failures, and reporting decision-readiness signals to leadership
Stay current with industry best practices in evaluation science, active learning, and hybrid human-AI quality control, bringing innovative validation methods to Apple’s evaluation stack
This is a highly collaborative leadership position that requires working across Engineering, Quality, Training, and Production Ops. Most of all, you are able to manage and lead change effectively while maintaining Apple culture and standards. Interpersonal skills, strategic thinking, and technical product knowledge are essential for success in this role

Qualification

Data ScienceMachine LearningMeta EvaluationHuman-in-the-Loop systemsStatistical analysisPythonAnalytical skillsCommunication skillsOrganizational skills

Required

8+ years of experience in Data Science, Machine Learning, or Evaluation Science, with 3+ years leading technical teams
Strong background in Meta Evaluation, AI/ML measurement, statistics, or quality assurance methodologies
Demonstrated success in designing Human-in-the-Loop (HITL) machine learning systems or active learning pipelines
Masters degree in Statistics, Data Science, Machine Learning or related field

Preferred

PhD in Statistics, Computer Science, Machine Learning, or related field
Deep understanding of evaluation pipelines, calibration techniques, and statistical process control
Experience building ML models specifically designed for quality estimation, anomaly detection, or disagreement modeling
Proficiency in Python or R for statistical analysis and reasoning about evaluation data
Experience defining governance gates or certification processes for AI systems
Proven ability to manage complex methodological and technical programs in dynamic, fast-paced environments
Exceptional communication, organizational, and analytical skill

Benefits

Comprehensive medical and dental coverage
Retirement benefits
A range of discounted products and free services
Reimbursement for certain educational expenses — including tuition
Discretionary bonuses or commission payments
Relocation

Company

Apple

twittertwittertwitter
Glassdoor4.2
company-logo
Apple is a technology company that designs, manufactures, and markets consumer electronics, personal computers, and software.
dateFounded in 1976
location
Cupertino, California, USA
people-size10001+ employees
web-link
https://www.apple.com

H1B Sponsorship

Apple has a track record of offering H1B sponsorships. Please note that this does not guarantee sponsorship for this specific role. Below presents additional info for your reference. (Data Powered by US Department of Labor)
Distribution of Different Job Fields Receiving Sponsorship
Represents job field similar to this job
Trends of Total Sponsorships
2025 (6998)
2024 (3766)
2023 (3939)
2022 (4822)
2021 (4060)
2020 (3656)

Funding

Current Stage
Public Company
Total Funding
$5.67B
Key Investors
Berkshire HathawayMicrosoftSequoia Capital
2025-05-05Post Ipo Debt· $4.5B
2025-01-16Post Ipo Debt· $0.31M
2021-04-30Post Ipo Equity

Leadership Team

leader-logo
Tim Cook
CEO
leader-logo
Craig Federighi
SVP, Software Engineering

Recent News

Venrock
Venrock Portfolio
2025-12-01
IndiaTimes
Apple TV+ pulls Jessica Chastain’s "The Savant" just days before premiere
2025-09-25
Mac Daily News
Jessica Chastain: I respect Apple’s decision to pause the release of ‘The Savant’ or something
2025-09-25
Company data provided by crunchbase