Apply on Employer Site

Walmart Canada · 4 weeks ago

(USA) Principal, Data Scientist | Conversational AI

(USA) Crossman Respect Building CA SUNNYVALE Home Office

Full-time

Onsite

Senior Level, Lead/Staff

$143K/yr - $286K/yr

7+ years exp

Walmart Canada is seeking a Principal Data Scientist for their Next Gen Commerce team, which focuses on developing intelligent agents for conversational shopping. The role involves leading the design and measurement of AI systems, collaborating with engineering and product teams to improve model performance and ensure safe deployment.

DeliveryRetailShopping

Responsibilities

Develop Evaluation Architectures: Design and implement state-of-the-art evaluation pipelines for conversational agents using LLM-as-a-judge, and hybrid scoring frameworks

Prompt Engineering & Calibration: Develop high-precision prompts for evaluator models and rigorously test them against human judgment to ensure high inter-rater reliability

Model Distillation & Optimization: Lead the fine-tuning of smaller, cost-effective models to act as scalable 'Judge' models, balancing trade-offs between accuracy, latency, and cost

Dataset Curation: Work with large-scale conversation logs to curate 'Golden Set' datasets and design annotation instructions that standardize ground truth for subjective tasks

Cross-Functional Integration: Collaborate with Engineering teams to integrate quality signals into CI/CD pipelines, enabling automated regression testing and production monitoring

Failure Mode Analysis: Conduct deep-dive analyses on agent failures (hallucinations, tool misuse, safety violations) and define actionable feedback loops for the modeling team

Insight Discovery & Strategic Influence: Leverage evaluation data to discover systemic weaknesses and root causes, actively influencing sub-agent modeling teams and cross-functional partners to prioritize and drive targeted improvements in overall performance

Thought Leadership: Mentor senior data scientists, standardize best practices for evaluation across the org, and maintain world-class credentials through patents, publications, or conference presentations

Qualification

Large Language ModelsData ScienceMachine LearningPythonNLPDeep LearningMetric DesignModel DistillationDataset CurationCross-Functional IntegrationFailure Mode AnalysisThought LeadershipPublicationsMentoring

Required

Advanced degree (Master's or PhD) in Computer Science, Statistics, Mathematics, Computational Linguistics, or a related field

7+ years of experience in Data Science or Machine Learning with a focus on NLP, Deep Learning, or AI evaluation

Deep understanding of Large Language Models (LLMs), including prompt engineering, chain-of-thought reasoning, and instruction tuning

Solid understanding of Python and expertise with core data science packages (NumPy, Pandas, PyTorch, Scikit-learn)

Proven experience designing metrics for non-deterministic outputs (e.g., evaluating summarization, relevance, or helpfulness)

Experience building scalable data pipelines and familiarity with distributed training/inference frameworks

Option 1: Bachelors degree in Statistics, Economics, Analytics, Mathematics, Computer Science, Information Technology or related field and 5 years' experience in an analytics related field

Option 2: Masters degree in Statistics, Economics, Analytics, Mathematics, Computer Science, Information Technology or related field and 3 years' experience in an analytics related field

Option 3: 7 years' experience in an analytics or related field

Preferred

PhD in Machine Learning, NLP, or a related quantitative field

Experience with conversational AI, chatbots, summarization, retrieval-augmented generation, or recommendation evaluation in an e-commerce context

Knowledge of model distillation, LoRA, instruction tuning, or parameter-efficient adaptation techniques

Familiarity with evaluating open-ended outputs where ground truth is subjective or contextual

Publications, patents, or open-source contributions in LLM evaluation or applied AI

Data science, machine learning, optimization models, PhD in Machine Learning, Computer Science, Information Technology, Operations Research, Statistics, Applied Mathematics, Econometrics

Publications or active peer reviewer in related journals or conference

Successful completion of one or more assessments in Python, Spark, Scala, or R

Using open source frameworks (for example, scikit learn, tensorflow, torch)

We value candidates with a background in creating inclusive digital experiences, demonstrating knowledge in implementing Web Content Accessibility Guidelines (WCAG) 2.2 AA standards, assistive technologies, and integrating digital accessibility seamlessly

The ideal candidate would have knowledge of accessibility best practices and join us as we continue to create accessible products and services following Walmart's accessibility standards and guidelines for supporting an inclusive culture

Benefits

Health benefits include medical, vision and dental coverage.

Financial benefits include 401(k), stock purchase and company-paid life insurance.

Paid time off benefits include PTO (including sick leave), parental leave, family care leave, bereavement, jury duty, and voting.

Other benefits include short-term and long-term disability, company discounts, Military Leave Pay, adoption and surrogacy expense reimbursement, and more.

Live Better U is a Walmart-paid education benefit program for full-time and part-time associates in Walmart and Sam's Club facilities. Programs range from high school completion to bachelor's degrees, including English Language Learning and short-form certificates. Tuition, books, and fees are completely paid for by Walmart.