Reflection AI · 1 month ago
Member of Technical Staff - Evaluations
SF
Full-time
Onsite
Mid LevelReflectionAI is focused on building open superintelligence and making it accessible to all. The role involves conducting critical comparative analysis, building evaluation systems, and collaborating with various teams to enhance model capabilities.
Computer Software
Responsibilities
Conduct critical comparative analysis to advance our understanding of model capabilities
Build and refine evaluation systems and processes that create tight feedback loops between data, evals, and model behavior
Develop generalizable evaluation frameworks that capture what matters for reasoning, alignment, and usefulness
Collaborate closely with pre-training, post-training, and applied teams to translate insights into model improvements
Push the boundaries of what’s measurable, from synthetic evals to human feedback and real-world interaction data
Qualification
Required
Strong statistical analysis and experimental design skills to rigorously measure model improvements
Familiarity with LLM evaluation methodologies: static benchmarks, human preference evals, and/or agentic tasks
High agency and thrive in a fast-paced startup environment; bias for impact over process
Excited to work in a new frontier lab, defining how we measure and accelerate progress toward more capable models
Collaborative, detail-oriented, and motivated by building the feedback loops that make models truly improve
Benefits
Top-tier compensation: Salary and equity structured to recognize and retain the best talent globally.
Health & wellness: Comprehensive medical, dental, vision, life, and disability insurance.
Life & family: Fully paid parental leave for all new parents, including adoptive and surrogate journeys. Financial support for family planning.
Benefits & balance: paid time off when you need it, relocation support, and more perks that optimize your time.
Opportunities to connect with teammates: lunch and dinner are provided daily. We have regular off-sites and team celebrations.
Company
Reflection AI
Frontier open intelligence accessible to all. Our team previously built frontier LLMs at labs like DeepMind, OpenAI, and Anthropic.
11-50 employees
H1B Sponsorship
Reflection AI has a track record of offering H1B sponsorships. Please note that this does not
guarantee sponsorship for this specific role. Below presents additional info for your
reference. (Data Powered by US Department of Labor)
Distribution of Different Job Fields Receiving Sponsorship
Represents job field similar to this job
Trends of Total Sponsorships
2025 (5)
Funding
Current Stage
Early StageCompany data provided by crunchbase