Apply on Employer Site

Meta · 4 hours ago

Staff Research Engineer, MetaAI Assistant Measurement

Seattle, WA

Full-time

Onsite

Lead/Staff

8+ years exp

Meta is a technology company that builds platforms to connect people and empower communities. They are seeking a Staff Research Engineer to provide technical leadership in AI Assistants, focusing on architecting measurement and evaluation paradigms that enhance user value and drive technical success across teams.

Computer Software

Comp. & Benefits

Responsibilities

Architect Scientific Strategy: Define and lead the execution of the scientific roadmap for AI Assistant measurement, ensuring methodologies are rigorous, scalable, and aligned with product goals

Innovate & Build: Spearhead the research and development of novel offline and online evaluation metrics, automated benchmarks, and synthetic data generation pipelines to close the loop between model training and deployment

Cross-Functional Technical Leadership: Serve as the primary scientific liaison to pre-training, post-training, and product teams, ensuring that measurement insights directly influence model architecture and training recipes (the "evaluation flywheel")

Mentorship & Influence: Provide technical mentorship and guidance to senior research engineers and applied scientists, fostering a culture of scientific rigor and code without direct management responsibilities

Hands-on Contribution: Remain hands-on in code and research, building prototypes for new evaluation frameworks and validating novel measurement theories

Qualification

Generative AI expertiseMeasurement systems designPython proficiencyDeep learning frameworksLarge language models experienceEvaluation frameworks developmentData synthesis techniquesCross-functional leadershipTechnical mentorshipCommunication skills

Required

Bachelor's degree in Computer Science, Computer Engineering, relevant technical field, or equivalent practical experience

Track record of driving technical strategy and landing large-scale research or product impacts in a time-sensitive environment

Proven technical vision regarding the future trajectory of Generative AI, specifically in how model performance translates to user utility

Expertise in designing and implementing online and offline measurement systems, benchmark building, and data synthesis techniques

Experience leading complex, cross-functional technical initiatives, driving consensus across engineering, research, and product boundaries

Proficiency in Python and deep learning frameworks (e.g., PyTorch), with the ability to prototype and implement complex methodologies

Preferred

PhD in Machine Learning, Statistics, Econometrics, Causal Inference, Computer Science, or related fields

8+ years of experience, or PhD + 6 years of hands-on experience in large language models, NLP, and Transformer modeling, within both research and engineering development settings

Significant experience developing comprehensive evaluation frameworks for LLMs, multimodal models, or agentic systems

Expertise in online measurement, user-behavior modeling, counterfactual analysis, and A/B testing platform design

Demonstrated background in AI system benchmarking, Reward modeling, RL, content evaluation, or human-AI interaction

Track record of publication in top-tier conferences (NeurIPS, ICLR, ICML, ACL) or significant contributions to open-source AI ecosystems

Ability to communicate complex scientific concepts clearly to executive leadership and non-technical stakeholders to influence high-level decision-making

Experience building and curating large-scale datasets and ontologies used for training or evaluation