Khan Academy · 2 weeks ago
Senior Platform Engineer I, AI Evaluation (24 months fixed-term)
United States
Contract
Remote
Senior Level
$138K/yr - $172K/yr
5+ years expKhan Academy is a nonprofit with the mission to deliver a free, world-class education to anyone, anywhere. They are seeking a Senior Platform Engineer to evolve and extend their internal evaluation framework for assessing AI-driven experiences, requiring software development skills and domain experience with AI evaluation.
AppsE-LearningEducationInternet
Responsibilities
Evolve and extend our internal evaluation framework for assessing the quality of our AI-driven experiences at Khan Academy
Work with ML data engineers and platform developers to help internal teams adopt an eval-driven development process incorporating offline benchmark tests and online experiments
Be fluent in the range of offline and online evaluation strategies, and when to apply the techniques over the lifecycle of development
Have intuitions about how to specify eval pipelines succinctly using declarative syntax
Understand the role of stratified datasets and ground truth labeling
Appreciate the range of eval scoring schemes from human raters to automated LLMs-as-judge
Qualification
Required
Bachelor's or Master's degree in Computer Science, Data Engineering, related field, or equivalent professional experience
5 years of Software Engineering including significant time working on the evaluation of generative AI systems or other evaluations of ML model quality
Strong programming skills in Go, Python, SQL, and at least one data pipeline framework (e.g., Airflow, Dagster, Prefect)
Familiarity with the architecture of large language models and their industry-standard APIs
Preferred
Experience with labeling platforms (e.g., Label Studio, Scale AI, Toloka) and human-in-the-loop concerns such as rubric development and inter-rater agreement
Exposure to MLOps practices such as model registry, feature store, or continuous evaluation
Background in education technology or other human-centered AI applications
Benefits
Competitive salaries
Ample paid time off as needed – Your well-being is a priority
8 pre-scheduled Wellness Days in 2026 occurring on a Monday or a Friday for a 3-day weekend boost
Remote-first culture - that caters to your time zone, with open flexibility as needed, at times
Generous parental leave
An exceptional team that trusts you and gives you the freedom to do your best
The chance to put your talents towards a deeply meaningful mission and the opportunity to work on high-impact products that are already defining the future of education
Opportunities to connect through affinity, ally, and social groups
401(k) + 4% matching & comprehensive insurance, including medical, dental, vision, and life
Company
Khan Academy
Khan Academy is a nonprofit organization that provides a free world class education for anyone, anywhere.
51-200 employees
H1B Sponsorship
Khan Academy has a track record of offering H1B sponsorships. Please note that this does not
guarantee sponsorship for this specific role. Below presents additional info for your
reference. (Data Powered by US Department of Labor)
Distribution of Different Job Fields Receiving Sponsorship
Represents job field similar to this job
Trends of Total Sponsorships
2025 (2)
2023 (2)
2022 (1)
2021 (2)
2020 (5)
Funding
Current Stage
Growth StageTotal Funding
$16.21MKey Investors
Amgen FoundationOmidyar Network
2022-01-01Series A· $0.01M
2020-07-21Grant· $3M
2017-10-12Grant· $3M
Leadership Team
Caroline Hu Flexer
CEO & Co-Founder of Khan Academy Kids & Duck Duck Moose
Boris Lau
Senior Software Engineer
Recent News
Hindu Business Line
2025-12-15
Tech Funding News
2025-10-31
Company data provided by crunchbase