Cohere · 1 day ago
Staff Software Engineer, Inference Infrastructure
San Francisco
Full-time
Hybrid
Senior Level, Lead/Staff
5+ years expCohere is a company dedicated to scaling intelligence for humanity by training and deploying frontier models for AI systems. They are seeking a Staff Software Engineer to join the Model Serving team, responsible for developing and operating the AI platform that delivers large language models through API endpoints, while ensuring high performance and reliability in production environments.
Artificial Intelligence (AI)Foundational AIGenerative AIMachine LearningNatural Language Processing
Responsibilities
Develop, deploy, and operate the AI platform delivering Cohere's large language models through easy to use API endpoints
Work closely with many teams to deploy optimized NLP models to production in low latency, high throughput, and high availability environments
Interface with customers and create customized deployments to meet their specific needs
Qualification
Required
5+ years of engineering experience running production infrastructure at a large scale
Experience designing large, highly available distributed systems with Kubernetes, and GPU workloads on those clusters
Experience with Kubernetes dev and production coding and support
Experience with GCP, Azure, AWS, OCI, multi-cloud on-prem / hybrid serving
Experience in designing, deploying, supporting, and troubleshooting in complex Linux-based computing environments
Experience in compute/storage/network resource and cost management
Excellent collaboration and troubleshooting skills to build mission-critical systems, and ensure smooth operations and efficient teamwork
The grit and adaptability to solve complex technical challenges that evolve day to day
Familiarity with computational characteristics of accelerators (GPUs, TPUs, and/or custom accelerators), especially how they influence latency and throughput of inference
Strong understanding or working experience with distributed systems
Experience in Golang, C++ or other languages designed for high-performance scalable servers
Benefits
An open and inclusive culture and work environment
Work closely with a team on the cutting edge of AI research
Weekly lunch stipend, in-office lunches & snacks
Full health and dental benefits, including a separate budget to take care of your mental health
100% Parental Leave top-up for up to 6 months
Personal enrichment benefits towards arts and culture, fitness and well-being, quality time, and workspace improvement
Remote-flexible, offices in Toronto, New York, San Francisco, London and Paris, as well as a co-working stipend
6 weeks of vacation (30 working days!)
Company
Cohere
Cohere is an enterprise AI firm developing secure and private AI technology to address real-world business challenges.
201-500 employees
H1B Sponsorship
Cohere has a track record of offering H1B sponsorships. Please note that this does not
guarantee sponsorship for this specific role. Below presents additional info for your
reference. (Data Powered by US Department of Labor)
Distribution of Different Job Fields Receiving Sponsorship
Represents job field similar to this job
Trends of Total Sponsorships
2025 (11)
2024 (14)
2023 (13)
2022 (5)
2021 (2)
Funding
Current Stage
Late StageTotal Funding
$1.71BKey Investors
Government of CanadaTiger Global ManagementIndex Ventures
2025-09-24Series D· $100M
2025-08-14Series D· $500M
2025-06-17Secondary Market
Leadership Team
Aidan Gomez
cofounder + ceo
Ivan Zhang
Co-Founder
Recent News
Beyond Bylines
2026-01-11
2026-01-06
Crunchbase News
2026-01-06
Company data provided by crunchbase