Fact Finders Pro ยท 6 hours ago
Applied AI Statistics Analyst (Volunteer Role-Unpaid)
Seattle, WA
Full-time
Onsite
Mid LevelFact Finders Pro is seeking a highly analytical and methodologically rigorous AI Statistical Analyst to enhance the reliability of outputs from machine learning and generative AI models. The role involves developing methodologies for validating probabilistic outputs, designing experiments, and collaborating with cross-functional teams to integrate statistical validation into AI workflows.
Computer Software
Responsibilities
Develop methodologies for validating probabilistic outputs from LLMs using classical statistical techniques
Design experiments and statistical tests to evaluate the calibration and reliability of probabilistic claims made by AI models
Work cross-functionally with ML engineers, data scientists, and prompt engineers to integrate statistical validation into generative AI workflows
Apply statistical inference techniques (e.g., regression analysis, Bayesian inference, significance testing) to model outputs and ground-truth data
Construct and test hypotheses about LLM behaviors under different prompt and context configurations
Communicate findings and methodologies clearly to both technical and non-technical audiences
Qualification
Required
Strong foundation in probability theory and its applications to uncertainty quantification in AI systems
Proven experience with regression analysis, including linear, logistic, and multivariate techniques in real-world machine learning or AI projects
Deep understanding of hypothesis testing and statistical significance, with practical experience in applying these concepts to AI/ML evaluation
Familiarity with generative AI models such as LLMs (e.g., GPT-4, Claude, Gemini), including experience in crafting, refining, and interpreting prompts
Proficiency with Python and statistical tools such as statsmodels, scipy, R, or similar
Familiarity with Bayesian inference frameworks and uncertainty quantification methods (e.g., Monte Carlo simulation, posterior predictive checks)
Ability to design and interpret statistical experiments involving LLM-generated outputs
Excellent communication skills, including the ability to convey statistical findings in clear, actionable terms
Preferred
Graduate degree (MS or PhD) in Statistics, Data Science, Applied Mathematics, Econometrics, or a closely related field
Prior experience in validating or auditing generative AI systems or developing statistical evaluation frameworks for ML outputs
Experience working with LLM APIs and evaluating probabilistic reasoning in LLM-generated content
Company
Fact Finders Pro
At Fact Finders Pro, we're on a mission to tackle one of the most pressing challenges of our time: disinformation.
Founded in 2023
51-200 employees
Funding
Current Stage
Growth StageCompany data provided by crunchbase